Prevalence of nonsensical algorithmically generated papers in the scientific literature - SIGMA
Article Dans Une Revue Journal of the Association for Information Science and Technology Année : 2021

Prevalence of nonsensical algorithmically generated papers in the scientific literature

Résumé

In 2014 leading publishers withdrew more than 120 nonsensical publications automatically generated with the SCIgen program. Casual observations suggested that similar problematic papers are still published and sold, without follow-up retractions. No systematic screening has been performed and the prevalence of such nonsensical publications in the scientific literature is unknown. Our contribution is 2-fold. First, we designed a detector that combs the scientific literature for grammar-based computer-generated papers. Applied to SCIgen, it has a 83.6% precision. Second, we performed a scientometric study of the 243 detected SCIgen-papers from 19 publishers. We estimate the prevalence of SCIgen-papers to be 75 per million papers in Information and Computing Sciences. Only 19% of the 243 problematic papers were dealt with: formal retraction (12) or silent removal (34). Publishers still serve and sometimes sell the remaining 197 papers without any caveat. We found evidence of citation manipulation via edited SCIgen bibliographies. This work reveals metric gaming up to the point of absurdity: fraudsters publish nonsensical algorithmically generated papers featuring genuine references. It stresses the need to screen papers for nonsense before peer-review and chase citation manipulation in published papers. Overall, this is yet another illustration of the harmful effects of the pressure to publish or perish.
Fichier principal
Vignette du fichier
CabanacAndLabbé2021.pdf (2.87 Mo) Télécharger le fichier
Origine Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-03242216 , version 1 (30-05-2021)

Licence

Identifiants

Citer

Guillaume Cabanac, Cyril Labbé. Prevalence of nonsensical algorithmically generated papers in the scientific literature. Journal of the Association for Information Science and Technology, 2021, 72 (12), pp.1461-1476. ⟨10.1002/asi.24495⟩. ⟨hal-03242216⟩
267 Consultations
154 Téléchargements

Altmetric

Partager

More