Bibliometrics is the quantitative analysis of scientific output and the networks behind that output. It can be performed at different levels. At the Institut Pasteur, for example, bibliometric research can be performed on the scientific output of a scientist, a unit, a department or a project. But there are also large-scale bibliometric studies for entire institutions, countries, continents, etc.
Three people in the CeRIS library work on bibliometrics, in collaboration with scientists but also with the management team and support services. The library can help Institut Pasteur scientists calculate their indicators and carry out comprehensive bibliometric studies.
The main aim of bibliometrics is to plan research. It enables scientists:
There are various methods and indicators to meet each of these needs. It is important to understand what each represents so that you can choose the most appropriate indicators for you.
Bibliometrics can also be taken into account when assessing scientists. However, we would recommend:
For several years now, a growing number of institutions have begun to criticize the negative effects of using indicators to assess scientists. For example:
Following these initiatives, a number of organizations, including the French High Council for Evaluation of Research and Higher Education (Hcéres), are calling for more qualitative evaluation methods that do not use the impact factor or h-index but take into account all the activities carried out by researchers, including teaching, applied research and scientific outreach.
Impact factor: a heavily criticized indicatorThis indicator is no longer used for its intended purpose. Although it was created to measure the impact of journals and help libraries select which journals they should subscribe to, it has come to be used for researcher evaluation, with potential for manipulation. The Institut Pasteur does not use impact factor to assess its scientists.
All indicators have advantages and disadvantages. It is important to choose the appropriate indicators depending on the aim of the bibliometric analysis.
Use with precautionSince bibliometric indicators only give a snapshot at a specific moment in time, they always need to be placed in context and several should be used at once. An indicator is not an absolute score, and it is always important to check any values that seem strange by going back to the publications.
The need to improve existing indicators and the use of social media has led to the creation of new indicators.
Databases are used to compile a corpus of references for analysis. All databases have limits and drawbacks (user-friendliness, topics covered, number of journals and books indexed and time period). It is therefore important to choose a database that corresponds with the aim of the bibliometric analysis.
The two subscription databases, both available at the Institut Pasteur, are:
Created in the 1960s, this is the historical database for bibliometrics on which several international rankings are based, including the Academic Ranking of World Universities (ARWU) also known as the Shanghai ranking. The limits of its coverage are well known, meaning that its inherent biases are clear, such as the fact that humanities, social sciences and economics are not well represented.
This database is more recent (2004) and its use is becoming increasingly widespread. Some international rankings use it like the Times Higher Education (THE). Its coverage is wider than WoS in thematic terms and with regard to the number of journals and books indexed. But its temporal coverage is not precisely known.
There is no real alternative to these two databases
Always cite the source databaseAnyone can calculate their indicators using databases. But from one database to another, the value of these indicators may change (especially depending on the database coverage), which is why you should always indicate the database and the filters used.
Once the corpus of references has been compiled from the database, the references are sent to an analysis tool to be processed, generating either figures or visual representations.
The two most popular tools have been developed by the publishers of two commercial databases:
These two tools are used at the Institut Pasteur. They analyze reference corpora retrieved from databases and generate bibliometric indicators.
These tools enable users to visualize data in a way that is often more meaningful and clear, using maps. The most well-known free programs are VOSviewer and Gephi. They can be used to:
analyze large quantities of words (full text of publications);
search for co-occurrences of words and produce word clusters;
display results in the form of a map.
The maps generated by these tools are often a useful addition to traditional bibliometric analyses, and they can be of as much interest to scientists themselves as to decision-makers, as a means of identifying competitors, potential collaboration opportunities or topics.
Firstly because the library staff know which indicators to use. They also take the time to correct the data. To produce a quality analysis, there are several little traps that need to be avoided (homonyms, changes of unit, etc.), requiring a great deal of care and attention. Finally, the library staff provide the scientist with a full package of results, containing the indicators and also the raw data, maps and a methodology report.
