Year : 2019 | Volume
| Issue : 5 | Page : 2-8
Can the similarity index predict the causes of retractions in high-impact anesthesia journals? A bibliometric analysis
Mohamed R El-Tahan
Anaesthesiology Department, College of Medicine, Imam Abdulrahman Bin Faisal University, Al Khubar, Saudi Arabia
Prof. Mohamed R El-Tahan
Anaesthesiology Department, College of Medicine, Imam Abdulrahman Bin Faisal University, P.O. 40289 Al Khubar 31952
Source of Support: None, Conflict of Interest: None
|Date of Web Publication||21-Feb-2019|
Background: The overall similarity index (OSI) and highest similarity scores (HSSs) from a single source might help to predict the potential reasons for the retraction from the anesthesia journals.
Methods: Retracted publications, from five highest impact anesthesia journals, were retrieved from the MEDLINE and journal archives and analyzed using a plagiarism detection software (iThenticate) and manually verified for citation characteristics, OSI, HSS, and the presence, extent, and location of the duplicate text. The validity of the OSI including and excluding quotations and references and the HSS in predicting the potential reasons for retraction were tested using the receiver operating characteristic curves.
Results: Of the total 138 retracted original and corresponding articles identified, 131 articles were analyzed. Most of them had the HSS more than 40% arising from a single source. Extensive degree of plagiarism (OSI score >35%) was identified through the main text of all analyzed retracted articles. The areas under the curves indicate that the OSI including and excluding quotations and bibliography and the HSS had reasonable ability to predict plagiarism and fabrication with a perfect sensitivity rate and low specificity but were weaker at distinguishing ethical misconduct or inconsistent or erroneous contents.
Conclusions: The study highlights the presence of significant plagiarism in the retracted anesthesia publications irrespective to the reasons for retraction. The high OSI and the HSS could be useful tools to identify the potential manuscripts with high risks for plagiarism and fabrication.
Keywords: Anesthesia; CrossCheck; misconduct; plagiarism; retractions; similarity index
|How to cite this article:|
El-Tahan MR. Can the similarity index predict the causes of retractions in high-impact anesthesia journals? A bibliometric analysis. Saudi J Anaesth 2019;13, Suppl S1:2-8
|How to cite this URL:|
El-Tahan MR. Can the similarity index predict the causes of retractions in high-impact anesthesia journals? A bibliometric analysis. Saudi J Anaesth [serial online] 2019 [cited 2022 May 23];13, Suppl S1:2-8. Available from: https://www.saudija.org/text.asp?2019/13/5/2/252641
| Introduction|| |
The high-impact anesthesia journals have the commitments to maintain the integrity of the clinical research for offering safe and effective patients' care and building the public trust in healthcare systems. The editors have the responsibilities to retract the flawed publications to correct the scientific literature. There is a significant increase in the number of retracted scientific publications which could reflect the changes in the behavior of journals' editors, authors, and institutions. Retractions in the medical literature are usually due to compromised peer review, ethical misconduct, plagiarism, duplicate publications, fabrication, falsification, inappropriate data management, authorship issues, journal issues, review process, conflict of interest, or unknown reasons.,
The World Association of Medical Editors defines plagiarism as “the use of others' published and unpublished ideas or words (or other intellectual property) without attribution or permission and presenting them as new and original rather than derived from an existing source.” The commercial plagiarism-checking software is regularly used by the journals for detecting plagiarism. Currently, the Crossref Similarity Check (iThenticate) is one of the most commonly used software for examination of the similarity index (SI) of the text.
There is no current consensus about the cutoff SI score to detect plagiarism. Of note, the low SI score does not necessarily mean eliminating plagiarism. Higgins and others found good sensitivity and specificity for the cutoff SI score of 15% (84.8% and 80.5%, respectively) to detect plagiarism among the manuscripts submitted to a single specialized medical journal (Genetics in Medicine) over a 1-year period. Taylor has developed an effective screening algorithm for the detection of plagiarism in the manuscripts submitted to the American Journal of Roentgenology including (1) the overall SI including quotations and references, (2) the highest similarity score (HSS) from a single source, and (3) to submit manuscripts with an average value of more than 12% for further review. I believe that Taylor's screening algorithm could be extrapolated to the anesthesia journals not only for detecting plagiarism but also predicting the reason for retractions.
Up to the best of my knowledge, no previous publications have studied the predictability of the overall SI score for the reasons of retraction from high-impact anesthesia journals.
I hypothesized that the overall similarity index (OSI) score and HSS from a single source might help to predict the potential reasons for the retraction from the high-impact anesthesia journals.
The investigator aimed to examine the overall SI scores, the HSS from a single source, and location of the duplicate text in the retracted publications from the five high-impact anesthesia journals.
| Methods|| |
Ethics committee approval for this research was waived. Five nonspecialized anesthesia journals with the highest impact factors including Anesthesiology, British Journal of Anaesthesia, Anesthesia and Analgesia, European Journal of Anaesthesiology, and Anaesthesia were selected based on the Journal Citation Reports by Clarivate Analytics in 2017.
All retracted articles from these five journals were included in the subsequent analyses. The retracted articles, which were left blank or only available in the scanned forms, were excluded from the analysis because the iThenticate software is unable to identify duplication in images.
The full search strategy aimed to include any retracted article from the selected five journals for any reason including plagiarized, erroneous, inconsistent, falsified, or fabricated content or ethical misconduct. Pertinent articles were independently searched in BioMedCentral, PubMed, and the journals' archives (updated November 3, 2018) by an expert librarian familiar with the literature search. The databases were searched using the following search keywords: “retracted,” “retraction,” “retract,” “note,” “notice,” “notification,” “misconduct,” “fabrication,” “falsification,” “duplicated,” and “plagiarism.” No time or language restrictions were imposed. Subsequently, all the abstracts and full-text articles of the included retracted publications and pertinent correspondences were reviewed.
The recruited retracted articles were analyzed using the iThenticate plagiarism detection software [http://www.ithenticate.com] and then verified with a detailed manual review.
The primary outcome included the OSI including and excluding the quotes and bibliography. The OSI was defined as the percentage of duplicated text from all sources whether it was originated from a single source or multiple sources.
Secondary outcomes were the type of retracted article, reason of retraction (plagiarism, duplicate publication, fabrication, ethical misconduct, or inconsistent or erroneous contents), HSS, defined as a SI greater than 10% arising from a single source, whether the publication contains copied portions of text from one or more sources, extent of plagiarism, and the location of the duplicated text in the manuscript. Similar to Taylor, plagiarism was defined as “an OSI equal or higher than 25% with the HSS from a single source equal to or higher than 10%.”
The validity of the data
Searching bias was assessed by a second investigator independently assessed compliance to selection criteria and selected publications for the final analysis.
Data were tested for normality using the Kolmogorov–Smirnov test. Categorical and ordinal data were analyzed using the Chi-square test followed by a modified Bonferroni correction. One-way analysis of variance was used to compare the continuous and normally distributed values in the five journals. Corrections for family-wise multiple comparisons using the Student–Newman–Keuls test were performed.
Univariate variables included the type of retracted article, the OSI including and excluding quotations and references, the HSSs more from a single source, the location of the duplicated text in the manuscript, and reasons for retraction. To identify the independent predictors that influenced reasons for retraction, these variables were examined in a stepwise manner in a multivariate logistic regression model, with entry and retention set at a significance level of P < 0.05.
The receiver operating characteristic (ROC) curves were generated to evaluate the accuracy of the OSI including and excluding quotations and references and the HSS in predicting the potential causes for retraction. A ROC area of 1.0 is characteristic of an ideal model, whereas an area of 0.5 indicates a model of no diagnostic value. The cutoff values for the OSI and HSSs with the best predicting sensitivity and specificity were derived from the ROC curve analyses.
Data were expressed as mean (standard deviation) or number (proportion). A value of P < 0.05 was considered to be statistically significant for continuous, categorical, and ordinal data, for which a Bonferroni correction was used.
| Results|| |
Database searches identified a total of 138 retracted articles from the five high-impact anesthesia journals. Seven articles were excluded from analysis because the original contents were removed, or the article is only available in a scanned form.
All remaining 131 articles were analyzed. The characteristics of the retracted articles, including the type of retracted publications and reasons for retraction, are presented in [Table 1]. Most of the retracted publications were original articles and were mainly retracted because of fabrication or ethical misconduct.
The OSI scores including and excluding the quotes and bibliography were shown in [Figure 1]. The retracted articles from the European Journal of Anaesthesiology differed significantly from the Anaesthesia journal in terms of the OSI scores including the quotes and bibliography(P = 0.025). Compared with the British Journal of Anaesthesia, the Anesthesia and Analgesia and European Journal of Anaesthesiology had significantly higher OSI scores excluding the quotes and bibliography (P < 0.044).
|Figure 1: The overall similarity index (OSI) (%) including and excluding the quotes and bibliography in the five journals. Note. Data are presented as mean (SD). BJA: British Journal of Anaesthesia; EJA: European Journal of Anaesthesiology. *P = 0.025 significant compared with the Anaesthesia journal, †P < 0.044 significant compared with the British Journal of Anaesthesia|
Click here to view
The retracted articles from the five journals did not differ significantly in terms of the HSS from one source, origin of duplicated texts, or extent or location of plagiarism [Table 2]. The vast majority of the analyzed retracted publications had the HSSs more than 40% arising from a single source. Major degrees of plagiarism (OSI score >35%) were identified through the main text of all analyzed retracted articles irrespective of the reasons for retraction [Table 2].
The type of retracted article, the OSI including and excluding quotations and references, the HSS more from a single source, and the location of the duplicated text in the manuscript showed no significant correlation with the reasons for retraction [Table 3].
The ROC curves of validating the OSI including and excluding quotations and references and the HSS in predicting the potential causes for retractions are shown in [Table 4] and [Figure 2]. The areas under the curves indicate that the OSI including and excluding quotations and bibliography and the HSS had reasonable ability to predict plagiarism and fabrication with a perfect sensitivity rate and low specificity but were weaker at distinguishing ethical misconduct or inconsistent or erroneous contents. The optimal cut points are presented in [Table 4].
|Table 4: Sensitivity and specificity of different outcome measures for the identification of the causes of retraction|
Click here to view
|Figure 2: The Receiver Operating Characteristic (ROC) curves analyses for validation of the overall similarity index including and excluding quotations and references and the highest similarity score in predicting the potential causes for retractions|
Click here to view
| Discussion|| |
Plagiarism is a grave unethical problem that could negatively impact the integrity of scientific research. That has raised the attention of the journals' editors to use different online software solutions for the detection of plagiarized contents in the submitted manuscripts. The SI, generated with the well-developed iThenticate software, could be used for plagiarism detection.
The present study showed that the analyzed retrieved retracted publications from the five high-impact anesthesia journals included a major degree of plagiarism (OSI greater than 35%) and HSSs more than 40% arising from a single source, irrespective the reasons for retraction. The OSI including and excluding quotations and bibliography and the HSS could be able to predict both plagiarism and fabrication.
In the present study, the identified significant degree of plagiarism in the all analyzed retracted anesthesia publications may be partially inflated by our tight definition of plagiarism similarly to Taylor, including “an OSI ≥25% with the HSS from a single source ≥10%.” However, interestingly most retracted articles had OSI scores >35% and the HSSs >40% arising from a single source. Most of the retracted publications were original research which could be explained by the fact that the original articles are one of the most powerful types of publications. Compared with the Anesthesia and Analgesia and European Journal of Anaesthesiology, the retracted publications from the United Kingdom anesthesia journals (the British Journal of Anaesthesia and Anaesthesia) had lower OSI scores.
The reasons for retraction of scientific publications vary significantly among different medical specialties. Article retractions occur across the biomedical field because of errors, plagiarism, duplicate publication, fraud, and invalid peer review. The commonest reasons from retraction from the surgical journals are duplication, ethical violations, and falsified data. The present study has shown that most of the retracted publications from the five anesthesia journals were because of fabrication or ethical misconduct. Similarly, Yan and others found that fraudulent data, plagiarism, and duplicate publication are the commonest reasons for 110 retracted publications from the orthopedic journals. Wang and colleagues have identified 97 retracted neurosurgical publications over a 5-year period because of duplicate publishing and plagiarism.
Similar to the previous study, an extensive amount of similarity was detected throughout the main text and abstract of the retracted publications.
Similarly to Taylor, the results of the present study demonstrated that the use of OSI including and excluding quotations and bibliography and the HSS from a single source might be useful to identify plagiarism and fabrication with a good sensitivity and a low specificity. Additionally, the cutoff points for these outcome variables have been described. Further studies are required to test whether these findings generalize to all impacted anesthesia journals.
The present study might have an impact in terms of drawing the attention of the editors to subject the submitted manuscripts with the high OSI and highest SI from one source scores for further editorial review for the possibilities of plagiarism or fabrication.
The present study had several limitations. First, the retracted anesthesia articles were retrieved only from five high-impact anesthesia journals. Thus, our results could not be generalized to the other anesthesia journals. Second, the iThenticate software was unable to examine 6% of the retracted anesthesia publications because either the original contents were removed or they were available in a scanned form. Of note, Baždarić and others demonstrated the ability of a combined use of the plagiarism detection software solutions including the TBLAST, CrossCheck, and WCopyfind and manual verification to detect plagiarized manuscripts. Similarly, I used the iThenticate software in conjunction with manual verification to examine the identified retracted publications. Third, the present study was not powered to test the hypothesis. Finally, the OSI score was not validated earlier to identify plagiarism.
In conclusion, the study highlights the presence of significant plagiarism in the retracted anesthesia publications irrespective to the reasons for retraction. The high OSI and the HSS could be the useful tools to identify the potential manuscripts with high risks for plagiarism and fabrication.
The author wishes to express their appreciation to Dr. Alaa Khidr, assistant professor in Anaesthesiology, Anaesthesiology Department, College of Medicine, Imam Abdulrahman Bin Faisal University, Al Khubar, Saudi Arabia, for his participation in this study.
Financial support and sponsorship
Conflicts of interest
I received free airway device samples from Ambu in April 2014 and from Airtraq in March 2015 for use in another three published studies. He has no direct financial or other interests in Ambu, Airtraq, or any other industry (in the context of this and other studies).
Mohamed R. EL-Tahan designed the study, employed searching, recruiting, analysis, and interpretation the data, and wrote and revised the manuscript.
| References|| |
Petrini C. Some considerations on integrity in scientific research. Clin Ter 2014;165:e186-90.
Steen RG, Casadevall A, Fang FC. Why has the number of scientific retractions increased? PLoS One 2013;8:e68397.
Moylan EC, Kowalczuk MK. Why articles are retracted: A retrospective cross-sectional study of retraction notices at BioMed Central. BMJ Open 2016;6:e012047.
Campos-Varela I, Ruano-Raviña A. Misconduct as the main cause for retraction. A descriptive study of retracted publications and their authors. Gac Sanit 2018 Jun 5. pii: S0213-9111(18)30072-4. doi: 10.1016/j.gaceta.2018.01.009. [Epub ahead of print].
Carlisle JB. Data fabrication and other reasons for non-random sampling in 5087 randomised, controlled trials in anaesthetic and general medical journals. Anaesthesia 2017;72:944-52.
Masic I, Begic E, Dobraca A. Plagiarism detection by online solutions. Stud Health Technol Inform 2017;238:227-30.
Park S, Yang SH, Jung E, Kim YM, Baek HS, Koo YM. Similarity analysis of Korean medical literature and its association with efforts to improve research and publication ethics. J Korean Med Sci 2017;32:887-92.
Higgins JR, Lin FC, Evans JP. Plagiarism in submitted manuscripts: incidence, characteristics and optimization of screening case study in a major specialty medical journal. Res Integr Peer Rev 2016;1:13.
Taylor DB. JOURNAL CLUB: Plagiarism in manuscripts submitted to the AJR: Development of an optimal screening algorithm and management pathways. AJR Am J Roentgenol 2017;208:712-20.
Masic I, Begic E, Dobraca A. Plagiarism detection by online solutions. Stud Health Technol Inform 2017;238:227-30.
Wang T, Xing QR, Wang H1, Chen W. Retracted publications in the biomedical literature from open access journals. Sci Eng Ethics 2018 Mar 7. doi: 10.1007/s11948-018-0040-6. [Epub ahead of print].
King EG, Oransky I, Sachs TE, Farber A, Flynn DB, Abritis A, et al.
Analysis of retracted articles in the surgical literature. Am J Surg 2018;216:851-5.
Yan J, MacDonald A, Baisi LP, Evaniew N, Bhandari M, Ghert M. Retractions in orthopaedic research: A systematic review. Bone Joint Res 2016;5:263-8.
Wang J, Ku JC, Alotaibi NM, Rutka JT. Retraction of neurosurgical publications: A systematic review. World Neurosurg 2017; 103:809-14.e1.
Baždarić K, Bilić-Zulle L, Brumini G, Petrovečki M. Prevalence of plagiarism in recent submissions to the Croatian Medical Journal. Sci Eng Ethics 2012;18:223-39.
[Figure 1], [Figure 2]
[Table 1], [Table 2], [Table 3], [Table 4]
|This article has been cited by|
||Screening for plagiarism in psychiatric research: Similarity scores are not all the same
| ||Brian J. Miller |
| ||Journal of Psychiatric Research. 2020; 131: 31 |
|[Pubmed] | [DOI]|
||Journal Retraction Rates and Citation Metrics: An Ouroboric Association?
| ||Amrutha B Nagella, Venkatesh S Madhugiri |
| ||Cureus. 2020; |
|[Pubmed] | [DOI]|