Fake science is getting faker — thanks, AI


The practice of science involves trying to find things out about the world by using rigid logic and testing every assumption. Researchers then write up any important findings in papers and submit them for possible publication. After a peer-review process, in which other scientists check that the research is sound, journals publish papers for public consumption.

You might therefore reasonably believe that published papers are quite reliable and meet high-quality standards. You might expect small mistakes that got overlooked during peer review, but no major blunders. It’s science, after all!

You’d be wrong in expecting this, though. Real and good science does exist, but there’s a worrying amount of bogus research out there, too. And in the last few years, it has increased in volume at lightning speed, as evidenced by the skyrocketing number of paper retractions.

Fake science

A number of practices currently threaten to undermine the legitimacy of scientific research. They include made-up authors, the addition of scientists who had nothing to do with a paper as a co-writer and even more nefarious practices like swamping journals with submissions from low-quality, AI-written junk.

This process is similar to a recall at the grocery store. If a previously sold product is bad or dangerous for some reason, the store might decide to recall it and ask all customers not to use it. Similarly, a journal can recall a published paper that, in hindsight, turned out to be bogus.

Of course, sometimes papers get retracted because the authors made an honest mistake in their research. In more than half the cases, however, it’s because of academic misconduct or fraud. Up until a decade ago, this sort of behavior was more or less limited to researchers falsifying experimental data or skewing results to favor their theory. The more sophisticated technology has become, however, the more things have gotten a lot more complicated.

One simple solution would be to just ignore bogus papers. The problem, though, is that they’re often hard to identify. Also, once a paper is retracted from a publication, that tarnishes the entire journal a bit. Let this happen often enough, and the public’s confidence in science as a whole goes down. Therefore, the scientific community as a whole needs to take this problem seriously.

Camille Noûs

Some of the problem is analog. Camille Noûs doesn’t have much to do with AI, but it deserves a mention nevertheless. Born in March 2020, Noûs has already co-authored more than 180 papers in fields as diverse as astrophysics, computer science and biology

I’m saying “it” because Noûs is not a real person; rather, it’s an artifact invented by French research advocacy group RogueESR. It carries the gender-neutral French first name Camille and a conflation of the ancient Greek word “νοῦς,” meaning reason or cognition, and the French word “nous,” meaning “us.”

Noûs was created in response to a heavily criticized new law (source in French) to reorganize academic research in France. Although the law’s objective was to make research better, its critics think that scientists’ jobs will be unfairly precarious and dependent on external funding under its requirements. In particular, the funding a scientist gets must depend on their own previous achievements, although research is often a community effort.

To make this concern visible, many researchers chose to add Noûs as a co-author. The journals and peer reviewers who were in charge of checking those papers weren’t always informed, however, that Noûs isn’t a real person.

Although the research portion of all these papers so far seems legitimate, it’s cause for concern that one can so easily add a co-author that doesn’t even have an ID card. Although highlighting communal efforts with authors like Noûs is an honorable goal, the idea that scientists can be invented out of thin air in this day and age is quite alarming.

Credit: Author provided