Generative AI is threatening scholarly publishing along two fronts at once: authors are using large language models to produce papers that look like research, and reviewers are quietly using the same tools to shortcut the evaluation meant to catch them. Automated detection offers no reliable rescue. The defense that remains is trained human judgment — and reviewers need a concrete set of red flags to exercise it.
AI's Dual Impact: Authoring Fraud and Compromised Review
The first threat sits in the manuscript itself. Writing in the Bulletin of the Atomic Scientists, an analysis of AI use in scholarly publishing notes that researchers who have used LLMs in their writing produce roughly a third more preprints than colleagues, citing a 2025 study by Kusumegi and co-authors. Speed is the lure; the hazard is overuse. Some researchers, the analysis reports, have fallen into the trap of having an LLM generate large portions of papers, or rewriting a draft so extensively that the meaning may unintentionally change. The result is prose that is fluent, confident and superficially convincing — but, in the analysis's words, "might potentially just turn out to be so much smoke and mirrors."
The second threat is subtler: the review process itself. Peer review is time-consuming and often thankless, and LLMs have begun to creep into it despite most publishers now carrying explicit warnings against reviewing papers with these tools. The Bulletin analysis, citing Sugiyama and Eguchi's 2025 finding, reports that some authors have even been discovered embedding invisible comments in their drafts instructing the LLM they expect will review the paper to skip straight to approval. When the verification layer can be gamed, the confidence that peer review is meant to provide collapses from both directions.
Disclosure is not the norm, either. Research by Geng and Trotta cited in the same analysis found that once distinctive markers of AI writing were publicly reported, they became less common in newer publications, while less-publicized markers kept spreading — strong evidence, the analysis argues, that many authors actively disguise their AI use.
The Limits of AI Detection Software
The obvious technological answer — run the manuscript through a detector — does not hold up. A systematic literature review published in Information in 2025, surveying research on ChatGPT's impact on academic integrity, reports that researchers including Denny and colleagues, Khalil and Er, and Susnjak have all documented the limitations of current plagiarism detection tools, which fail to recognize original essays produced by ChatGPT. The review's cited authors call for advances in detection technology and updated institutional policies.
The problem runs in both directions. An editor writing on the AAUP's Academe blog in June 2024 observed that antiplagiarism programs in general offer no effective bulwark against AI-generated writing, and that software purporting to detect AI prose risks being used to falsely accuse students of academic dishonesty. The same editor noted that Turnitin's AI-checking feature was still in development at the time. A false accusation is as corrosive to academic trust as an undetected fake paper — which is why the human reviewer, not the detector, remains the load-bearing element of the system.
