|Year : 2015 | Volume
| Issue : 3 | Page : 783-786
Using scientific indices to evaluate the journal quality
Saurabh R Shrivastava, Prateek S Shrivastava, Jegadeesh Ramasamy
Department of Community Medicine, Shri Sathya Sai Medical College and Research Institute, Chennai, Tamil Nadu, India
|Date of Submission||27-Feb-2015|
|Date of Acceptance||16-Apr-2015|
|Date of Web Publication||22-Oct-2015|
Saurabh R Shrivastava
3rd Floor, Department of Community Medicine, Shri Sathya Sai Medical College and Research Institute, Ammapettai Village, Thiruporur, Guduvancherry Main Road, Sembakkam Post, Kancheepuram 603108, Tamil Nadu
Source of Support: None, Conflict of Interest: None
From a researcher's point of view, it is quite difficult to ascertain which of the available journals is the best for publication of their research work, as well as identify the quality indices and determine ways and means of enhancing the international reach of the article. The objective of the article is to explore the various indicators that can be used to scientifically assess the quality of the journal. Keywords used in the search included impact factor, Eigenfactor score, article influence score, SCImago journal rank indicator, and H-index. Each of these indices have some positive aspects and at the same time some associated limitations. In conclusion, in order to assist the researchers to make an informed decision when selecting an appropriate and a good-quality journal, so that their work does not go unrecognized, it is necessary that researchers ascertain the quality of the journal and then submit their research work.
Keywords: Article influence score, Eigenfactor score, H-index, impact factor, index Copernicus value
|How to cite this article:|
Shrivastava SR, Shrivastava PS, Ramasamy J. Using scientific indices to evaluate the journal quality. Menoufia Med J 2015;28:783-6
|How to cite this URL:|
Shrivastava SR, Shrivastava PS, Ramasamy J. Using scientific indices to evaluate the journal quality. Menoufia Med J [serial online] 2015 [cited 2022 Jul 3];28:783-6. Available from: http://www.mmj.eg.net/text.asp?2015/28/3/783/165824
| Introduction: dilemma of a researcher|| |
From a researcher's point of view, it is quite difficult to ascertain which of the available journals is the best for publication of their research work, as well as identify the quality indices and determine ways and means of enhancing the international reach of the article . To further complicate the matter for a researcher, on each of the potential research areas, multiple national/international journals or free/paid journals or variety in the type of articles accepted for publication (original article, short communication, case report, technical note, review article, mini review, editorial, letter to the editor, book review, commentary, theme-based issues, etc.) exists ,.
In recent times, some of the sections of the research community have reached a conclusion that if their research work gets published in a journal under open access section, the scientific visibility of their work gets augmented by a multiple number of times, as other people do not have to pay any money to access the work . The objective of the article was to explore the various indicators that can be used to scientifically assess the quality of the journal. This will help the scientific community to select an appropriate journal or reject a substandard journal when publishing their research work.
Crude indicators to evaluate journal quality
As a matter of fact, until the last decade, only crude indicators were popular to assess the quality of the journal. These crude indicators included the following parameters:
- Circulation - Total number of issue copies sold per month;
- Readership - Total number of readers per copy;
- Regular readers - Total number of people who regularly access the current issues;
- Total number of downloads for an article during a specified period of time; and
- Scientometrics - The quantitative attributes of a journal, such as the type of articles published in a particular journal, etc. ,,.
However, the problem with most of them was that none of the indices were computed scientifically, and there was ample scope of bias in the estimates (for instance, in order to enhance the number of downloads or the number of hits for a specific article, the contributors of the work may themselves download the copy of the article on the web or like the article for a number of times) ,,.
Scientific indicators to evaluate journal quality
In order to overcome the shortcoming of the crude indicators, and even to complement the findings of the crude indicators, it is mandatory that the journal/editorial board of the journal should resort to using scientific indicators ,. Although many scientific indicators are being used now-a-days by different journals to enhance their reputation among the scientific community, we discuss only the well-acclaimed indices in this article. These indicators include the following.
Globally, impact factor has been acknowledged as one of the most commonly used indicators to evaluate the journal quality ,. The impact factor of any journal can be computed by dividing the total number of articles in the journal, which is cited by any journal, with the total number of research articles published in that journal during the last 2 years ,,.
The New England Journal of Medicine has a very high impact factor in excess of 53, and thus is highly rated among the scientific community. However, the information pertaining to impact factor about any journal can be easily obtained from the journal website ,. Specific agencies, such as International Scientific Indexing, evaluate the impact factor for different journals, provided the journal has completed the prerequisites (such as the journal being more than 2 years old, journal pays the desired fee to the evaluating agencies, etc.) ,.
Although the impact factor is one of the key indicators in assessing the quality of the journal, its usage as a single measure has some inherent limitations ,. These deficiencies include problems such as reporting of high impact factor for a specific journal because of the following reasons:
- Self-citation (authors citing their own work on the same topic in different journals to augment the impact factor);
- Journals publishing more number of review articles (as review articles generally have more number of references ranging from 50 to 100 or even more, it gives an opportunity to cite the articles more often) compared with journals publishing only editorials/case report, etc.
- Being a specialized journal (for instance, a specialized journal covering the scope of breast cancer alone allows different authors to cite the same articles published in that journal itself in the past years, and thus a higher impact factor for the journal is obtained);
- Journals publishing theme-based issues;
- Bias attributed to English language, etc. ,,,,.
However, despite the presence of these shortcomings, the impact factor still remains the most often used indicator probably because of the absence of any other measure to reach to a conclusion about the quality of the journal ,. In fact, on the basis of the impact factor of a journal, the quality of the research work performed by a researcher is evaluated (as it is presumed that any journal with high impact factor will not accept any substandard quality article, in contrast to a journal that has either less or no impact factor at all) ,. In addition, some of the employers or even the funding agencies use the impact factor as the criterion to give jobs to the professional or even approve the project proposal for financial support ,. Furthermore, some of the institutes (like Shri Sathya Sai Medical College & Research Institute, Kancheepuram or Mahatma Gandhi Medical College & Research Institute, Pondicherry) or universities (for instance, Sri Balaji Vidyapeeth, Pondicherry) recognize the research work of their faculties on the basis of the impact factor of the journal in which faculties publish their research work and reward them proportionately ,,.
The H-index is a useful tool for measuring the scientific output of an investigator by considering various attributes such as the total number of contributors in a specific article, the number of citations received by a scientist, etc. . The H-index negates the limitations of other quality indices (namely, the total number of papers - it cannot be concluded that if a journal publishes more papers then its quality is much better than that of other journals and vice versa - or the total number of citations - this parameter can be significantly influenced by contributing in a single publication of wide importance) ,,. However, the H-index also has some shortcomings: it discards the information on the author placement in the authors' list, limited predictive accuracy and precision, scope of manipulation because of self-citations, etc. ,. The range of the H-index has even been extended to the assessment of a particular department or a university or even a nation ,. The scientific experts from different fields have reached to a conclusion that the H-index should be used to complement the impact factor so that the value of impact factor can provide decisive information about the quality of a journal ,. For instance, the H-index for the Rural and Remote Health journal was calculated to be as 19 .
This indicator quantifies the total number of times research articles published in a specific journal during the last 5 years have been cited in the Journal Citation Reports ,. The Eigenfactor score enables the ranking of a journal depending on the total number of citations. Furthermore, any citation from a highly ranked journal increases the Eigenfactor score significantly and vice versa ,,. The Eigenfactor scores for a specific journal can be obtained from the website eigenfactor.org . The Eigenfactor score differs from the impact factor as it acknowledges citations to journals in the field of both science and social sciences, and even neutralizes the concern of self-citations ,,,. For instance, the Eigenfactor score for the journal International Journal of Epidemiology has been recorded to be 0.037468 .
Article influence score
The article influence score is computed by obtaining the ratio of a journal's Eigenfactor score and the fraction of articles published in that journal ,. This score provides direct evidence about the relative importance of any particular journal based on individual articles ,. The article influence score for a specific journal can be obtained from the website eigenfactor.org . The mean article influence score is 1; a score more than 1 reflects that each article in the specified journal has above-average influence, and a score less than 1 reflects that each article in the specified journal has below average influence ,. For instance, the article influence score for the journal International Journal of Epidemiology has been recorded to be 2.8896 .
SCImago journal rank indicator
SCImago journal rank indicator assesses the quality of only those journals that have been indexed in Scopus ,. The rank of the journals is established by utilizing the same approach as that of Eigenfactor score. However, instead of 5 years (Eigenfactor score), SCImago journal rank indicator considers only 3 years ,.
Index Copernicus value
Index Copernicus is a Poland-based indexing agency that provides a rank to those journals that are indexed in Indexed Copernicus . The Index Copernicus value is estimated by considering a wide range of parameters:
- Scientific quality (percentage of original papers published during the last 2 years; scientific papers published during the previous 2 years; and international indexing);
- Internationalization (this includes attributes such as language of the titles/abstracts/published articles; international editorial board; cooperation with foreign reviewers; and the number of published papers submitted from foreign centers);
- Stability (to evaluate the effectiveness of publishing/editorial management);
- Technical quality (comprises parameters such as possession of digital object identifier number; internet availability; availability of information, editorial, table of contents, abstracts, full-text articles, and tools for finding scientific content; formats in which scientific content is available on the website of the journal; and tools used to manage bibliographies);
- Standards (to assess the standards being followed to ensure scientific reliability such as display of procedures for reviewing a manuscript in both electronic and print version of the journal, details about the type of review adopted by the journal - double-peer reviews, unnamed peer reviews, open reviews - mandatory presence of statements about the originality of the article/conflict of interest/author contributions. financial support, and standards to regulate the legal aspects of the journal functioning);
- Editorial quality (includes particulars such as the first page of the cover - information about the title, International Standard Serial Number, frequency of publication, volume/number/part number, month/year, etc.; title page of journal; editorial board information including their affiliations and biography; information about the institutions that finance and support the journal; particulars about contacting the editor and publisher; instructions for authors; uniform structure of the article, etc.); and
- Print and website point ,.
For instance, the Index Copernicus value of the journal Asian Pacific Journal of Tropical Disease for the year 2013 was 8.43 . However, this index cannot be relied upon much as a solitary indicator .
As the name suggests, immediacy index means the total number of times a specific published article has been cited in the same year during which it has been published . In simple terms, it indicates the frequency with which an article published in a journal gets cited . However, more often than not, the immediacy index gives higher value for those journals that have weekly/monthly issues, specialized journals, specialty with more number of professionals, articles published on the same topic in a single issue (for instance - issues published on Ebola virus disease outbreak in West-Africa), selective journal self citation (some journals have a policy to make it mandatory for the researchers to cite any work that has been published earlier), short publication period (journals with very short review time or very short time interval to publish the manuscript after its acceptance), and for those journals that publish more number of review articles ,,.
Quality of references
In order to assess the quality of references cited in an article, indicators such as citation density (average number of references cited in the new article) and half-life (number of past years required to find 50% of the cited references) are used in the field .
| Conclusion|| |
In order to assist the researchers to make an informed decision when selecting an appropriate and a good-quality journal, so that their work does not go unrecognized, it is necessary that researchers ascertain the quality of the journal and then submit their research work.
| Acknowledgements|| |
Conflicts of interest
There are no conflicts of interest.
| References|| |
Olff M. Are we happy with the impact factor? Eur J Psychotraumatol 2014; 5
Pan RK, Fortunato S. Author impact factor: tracking the dynamics of individual scientific impact. Sci Rep 2014; 4
Yang ZG, Zhang CT. A proposal for a novel impact factor as an alternative to the JCR impact factor. Sci Rep 2013; 3
Kodumuri P, Ollivere B, Holley J, Moran CG. The impact factor of a journal is a poor measure of the clinical relevance of its papers. Bone Joint J 2014; 96-B
Kianifar H, Sadeghi R, Zarifmahmoudi L. Comparison between impact factor, Eigenfactor metrics, and SCImago journal rank indicator of pediatric neurology journals. Acta Inform Med 2014; 22
Elsaie ML, Kammer J. Impactitis: the impact factor myth syndrome. Indian J Dermatol 2009; 54
Garfield E. The history and meaning of the journal impact factor. JAMA 2006; 295:90-93.
Ascaso FJ. Impact factor, Eigenfactor and article influence. Arch Soc Esp Oftalmol 2011; 86
Brown T. Journal quality metrics: options to consider other than impact factors. Am J Occup Ther 2011; 65
Smith R. Commentary: the power of the unrelenting impact factor - is it a force for good or harm? Int J Epidemiol 2006; 35
Rawat S. How is impact factor impacting our research? Biomed J 2014; 37
Casadevall A, Fang FC. Causes for the persistence of impact factor mania. MBio 2014; 5
Hirsch JE. An index to quantify an individual′s scientific research output. Proc Natl Acad Sci USA 2005; 102
Bornmann L, Daniel HD. The state of H index research. Is the H index the ideal way to measure research performance?. EMBO Rep 2009; 10
Bergstrom CT, West JD. Assessing citations with the Eigenfactor metrics. Neurology 2008; 71
Ramin S, Sarraf Shirazi A. Comparison between Impact factor, SCImago journal rank indicator and Eigenfactor score of nuclear medicine journals. Nucl Med Rev Cent East Eur 2012; 15
Oosthuizen JC, Fenton JE. Alternatives to the impact factor. Surgeon 2014; 12
Index Copernicus International. Evaluation methodology; 2014. Available at: http://journals.indexcopernicus.com/page.php?page=3
. [Accessed 19 February 2015]
Luciano M. The new impact factor and immediacy index of World Psychiatry. World Psychiatry 2012; 11
Venable GT, Shepherd BA, Roberts ML, Taylor DR, Khan NR, Klimo P Jr. An application of Bradford′s law: identification of the core journals of pediatric neurosurgery and a regional comparison of citation density. Childs Nerv Syst 2014; 30