As discussed by Drs. Miller and Young, a crisis is underway in scientific research. “Paper mills” publish fake or manipulated articles that resemble genuine scientific research; scientific journals are offered money for each paper published in the journal, and there are instances of more common fraud, including plagiarism, omitting negative results, and statistical manipulation of data.
The Alzheimer’s Case
In 2006, a paper published in the distinguished journal Nature reported the discovery of a protein, which forms plaques in human brains, that, when inserted into the brains of rats, caused memory impairment, as demonstrated by tests in mazes.
This finding was touted on the author’s website as “the first substance ever identified in Alzheimer’s research that has been shown to cause memory impairment.” It was cited in about 2300 scientific articles (a considerable number) about Alzheimer’s research. This article boosted the National Institute of Health (NIH)-supported research on this protein as a cause of Alzheimer’s from zero to $287 million in 2021. Numerous articles followed, culminating in the development of an Alzheimer’s drug in 2021 based on this theory.
Dr. Matthew Schrag, a neuroscientist and professor at Vanderbilt University, was contacted by an attorney representing two prominent neuroscientists who believed some research related to the Alzheimer’s drug may have been fraudulent. [1] He was asked to investigate the images in the published papers that supported the theory that the specific protein caused Alzheimer’s. He found that the authors
“appeared to have composed figures by piecing together parts of photos of different experiments.”
Some scientists now believe that these experiments misdirected Alzheimer’s research for 16 years.
“The immediate, obvious damage is wasted NIH funding and wasted thinking in the field because people are using these results as the starting point for their experiments.”
- Thomas Sudhof, MD, PhD, Stanford University
This is a dramatic illustration of why fraud detection is critical.
Detecting Fraud
AI Software: AI tools are being used to detect plagiarism, statistical manipulation, and image fraud in scientific publications. According to one scientific publisher, image fraud was the #1 reason papers were retracted from journals. Duplication of images, where the same image is copied, flipped, rotated, or cropped, is common. In some cases, doctored images make it look like the scientists carried out more experiments than were done.
The publisher Science recently announced that they will use the AI detection software Proofig to check all the images on a paper before publication. Proofig counts the number of features shared by different images within a paper and presents a graphical representation of the shared features. Although this software can catch image fraud involving data duplication, it cannot detect other types, such as images built on falsified data.
Human Fraud Detectives: Dutch microbiologist Dr. Elisabeth Bik has single-handedly caught numerous cases of fraud in scientific papers. She uses the image-detecting software Image Twin to help her detect doctored images. Still, she believes the human eye is often better than machines in detecting irregularities in images, such as the Western blot, used to separate and identify proteins based on molecular weight. These were the images that were determined to be fraudulent by Dr. Matthew Schrag. Ironically, there is no process to pay fraud detectives like Dr. Bik, so she uses crowdfunding to support her work.
AI authors: Another type of scientific fraud is using AI to create fake scientific papers. It is not known how widespread this practice is. A recent study investigated the use of AI to generate high-quality fraudulent medical articles. The authors used ChatGPT to generate a fraudulent article about neurosurgery. Here are some of the prompts they used to create an article in under an hour [2]:
- Suggest relevant randomized controlled trial in the field of neurosurgery that is suitable for publication and has a high chance of acceptance.
- Give me an abstract and the whole article, section by section, using scientific language.
- Give me materials, methods, and a detailed results section, including patient data.
- I need a discussion. Compare the results with published articles.
- Give me all the references.
- Suggest tables and charts to go with the results section.
- Provide datasheets for creating charts.
The authors then used AI detection software to check their article. They used the AI detector software “Content at Scale,” which states it has a 98% accuracy rate of telling whether the text is human or AI-generated. It rated the probability of AI content of this article at 48%, far from a convincing result. Another software tool, AI Text Classifier by Open AI, rated the AI generation of the article as “unclear.”
If researchers are no longer able to distinguish real from fake science, we could be on the brink of a nightmare scenario where
“generative AI empowers dishonest scientists to forge data and papers with such ease and at such a rate that self-correction mechanisms are overwhelmed and the scientific process enters a new, dysfunctional state.”
Potential Solutions
Science publishing needs to change. Traditional journals need to work more quickly; the peer review process typically takes months, and it is not uncommon for articles to be published up to a year after they are submitted. Much of fraud detection depends on scientific journals and academic institutions, which have an inherent conflict of interest to avoid the bad publicity that comes with retracted papers.
One solution is to require independent audits of any paper that gets federal funding. Independent auditing is not a new concept; it is a standard for the financial results of publicly traded companies. The U.S. spent $191 billion in 2023 on scientific research. Surely, a small slice of that money could be used to set up an audit system to develop better fraud-detecting software and to fund human fraud detectives. When fraud is detected and confirmed, there needs to be transparency, where the names and the institutions are rapidly made public in a database so scientists and the public can see the consequences of their actions. The credibility of scientific research and the scientific process is at stake.
[1] The neuroscientists were short sellers of stock of the company making the claims.
[2] As an author of several scientific articles, I find it almost impossible to comprehend how fast AI works. Preparing each section of an article is a laborious task, taking hours to days, and the discussion section typically involves countless hours of work researching previous results. Putting together the reference section is enough to make most scientists consider another profession!
