Since its official launch in 2004, Scopus has been used globally in many large-scale analyses. There are three types of large studies where Scopus data are used in a central role.
The first group of large-scale analyses deals with national research assessments. The first national assessment supported by Scopus was the Excellence in Research for Australia (ERA) of the Australian Research Council (ARC) in 2010, later repeated in 2012 and 2015. The first edition of the Research Excellence Framework (REF 2014) national assessment in the UK also used Scopus data. The REF 2014 was held by the Higher Education Funding Council for England (HEFCE) and the funding bodies for Scotland, Wales, and Northern Ireland. Up to four research outputs per active researcher were submitted by the UK’s Higher Education Institutions (HEIs) and matched to the corresponding records in Scopus for 11 out of 36 “units of assessment” (UoAs; i.e., broad subject areas). For each of these 11 UoAs, Scopus citation counts of the submitted outputs were compared with that UoA’s citation benchmarks (also obtained from Scopus as contextual data) and used as an additional assessment criterion by the REF’s expert panels, besides peer review of the scientific content of the outputs. Prior to the announcement of the REF 2014 assessment results by HEFCE, the REF Results Analysis Tool provided by Elsevier allowed the UK’s HEIs to compare and evaluate their own REF performance across several benchmarks and metrics. More examples of national assessments where Scopus data were used include the 2013–2014 assessment in Portugal held by the FCT (“Fundação para a Ciência e a Technologia”), the ASN (“Abilitazione Scientifica Nazionale”) national accreditation rounds (2012–2013, 2016–2018, and 2018–2020), VQR (“Valutazione della Qualità della Ricerca”) national assessment in Italy (2012–2013, 2016–2017), and National University Corporation Evaluation (NUCE) national assessment in Japan held in 2016–2017 and to be held in 2020 by the National Institution for Academic Degrees and Quality Enhancement of Higher Education (NIAD-QE).
A second type of analysis supported by Scopus data is government science policy evaluations. The UK department of Business, Innovation and Skills (BIS) commissioned a report that entailed a comparative study of the UK’s international research base, in 2011 (Elsevier, 2011) and 2013 (Elsevier, 2013). In 2016, another refresh of this report was issued by the newly renamed Department for Business, Energy, & Industrial Strategy (BEIS) (Elsevier, 2016). More examples of the use of Scopus data for research policy reports include those of the European Research Council (ERC) and other large government bodies, often dealing with program evaluations and research landscape analyses. Many of these evaluations include different data sources of various types (macroeconomic data, for instance), as well as deep qualitative evaluations in which Elsevier works in consortia.
As a further example of this second type of analysis, Scopus data have also been used in the production of bibliometric indicators for the US National Science Foundation’s Science and Engineering Indicators (SEI), for the 2016 (National Science Board, 2016) and 2018 (National Science Board, 2018) editions, and will be used in future editions of the SEI reports up to 2022. Scopus data are also the source of bibliometric indicators for the European Research Area in the context of the 2010–2014 study “Analysis and Regular Update of Bibliometric Indicators for the European Commission” (Science-Metrix, 2014) and have recently been selected for the continuation and improvement of this study, now called “Provision and Analysis of Key Indicators in Research and Innovation,” for the three coming years.
The third type of analysis is that of university rankings. University rankings are often composed of combinations of evaluations for which only part is a bibliometric resource. Different ranking bodies have variations of subjective and objective data sources to provide ranked lists of universities. These rankings and the media attention they draw provide a platform for academia to engage with the public. Elsevier provides publication output, citation, and international collaboration data from Scopus for each university to organizations in the field of university rankings. These include the World University Rankings and its various derived Regional, Global Subject, Young University, and World Reputation Rankings, as well as the Wall Street Journal/THE College Rankings, which are all issued by Times Higher Education (THE, since 2014); and the World University Rankings and its various derived rankings issued by Quacquarelli Symonds (QS, since 2015); as well as various other regional and subject-specific rankings, such as the Best Chinese Universities Ranking issued by ShanghaiRanking Consultancy (since 2015), Perspektywy in Poland, Maclean’s in Canada, National Institutional Ranking Framework (NIRF) in India, the Financial Times Global MBA Ranking, and the Frankfurter Allgemeine Zeitung Economists Ranking in Germany.
The large-scale analyses supported by Scopus are reports in the government and commercial space. In contrast to peer-reviewed scientific studies, where the focus is on access to the data and open reproducibility of results, reports in the evaluation space focus on accountability. Key aspects in those engagements are therefore about how data providers deal with quality concerns, quality assurance, and risk mitigation.