Let's Talk About "Pre-Prints"
How should we interpret early, unvetted research?
Happy New Year to all of my readers! 🥳 I hope you had a great holiday break with your loved ones and are starting 2025 rested and refreshed.
Today, I want to talk about the growing trend of “pre-print” research publications. They have become increasingly popular in recent years, fundamentally changing how scientific research spreads. Remember the early days of the COVID-19 pandemic, when it seemed like our understanding of the virus changed daily? Much of that rapid-fire information came from pre-print research articles—scientific studies shared online before going through the traditional peer-review process. While this quick sharing of information helped accelerate our response to the pandemic, it also led to confusion and premature conclusions (including a few scandals, discussed below).
What exactly are pre-prints, and do their benefits outweigh the potential drawbacks? How should we interpret these preliminary research findings when we read about them in the news?
Let’s dig in!
—Eric
What ARE pre-prints?
Traditionally, when researchers complete a study, they submit their findings to an academic journal where it undergoes peer review—a process where multiple scientists unaffiliated with the study review the methods and results, then after asking questions and probing the data, each can recommend it be accepted, revised, or rejected (and several layers of editors usually weigh in above the reviewers). As you can imagine, this is a lengthy process that can take months or even years. Pre-prints bypass this initial waiting period by allowing researchers to share their findings immediately on specialized websites like arXiv, bioRxiv, or medRxiv.
Think of pre-prints as the scientific equivalent of a movie test screening—they’re not the final cut, but they give people a chance to see and discuss the work while it’s still being refined. The feedback from test audiences can significantly change the ultimate film. For example, when I lived in Los Angeles in the early 2010s, I was invited to view an early cut of The Life of Pi. The version I saw differed substantially from the theatrical release: the narrator was originally played by Tobey Maguire, but the director Ang Lee recast a less known actor for that role when test audiences didn’t react well to him.
For pre-prints, this immediate sharing of preliminary results has several compelling advantages. First, it dramatically speeds up the spread of new discoveries. In fast-moving fields like artificial intelligence or infectious disease research, waiting months for peer review could mean missing crucial opportunities to build on important findings. During the COVID-19 pandemic, pre-prints allowed researchers worldwide to share their discoveries about the virus almost in real-time, helping accelerate our understanding and response to SARS-CoV-2.
Pre-prints also level the playing field in science. These documents are free to all, so anyone can access the findings instead of having to pay expensive journal fees. Researchers from smaller institutions or developing countries can share their work immediately, rather than waiting for acceptance from prestigious journals. Young scientists can also use pre-prints to generate “buzz” and build their reputations; one study found that manuscripts that were initially released as a pre-print went on to get more citations and media attention.
How Fast is TOO Fast?
However, this speed and openness comes with risks. Without peer review’s quality control, pre-prints may contain errors or overstate their conclusions. We've all seen headlines proclaiming breakthrough discoveries that later turn out to be premature or incorrect (especially during the pandemic). Often, these come from journalists or social media users who don't fully grasp the preliminary nature of pre-print research.
How common is this problem? A few studies have looked at the question. In general, peer-reviewed studies tend to be higher in quality than their pre-print form, although the differences appear to be small on average. A bigger issue than changes between initial and final form is the fact that about HALF of research manuscript pre-prints are never ultimately published in a peer-reviewed journal at all! This tends to be due to rejection from multiple journals, which could imply the original research was not up to par:
“The reason most often given for nonpublication of the preprint in a peer-reviewed journal was rejection by at least one journal (58.3%, 7/12), followed by lack of time (25%, 3/12). […] Among those preprints that were submitted, 57.1% (4/7) got rejected 3–4 times. The official reasons given by journals for rejecting the preprints were manyfold and included lack of novelty (n = 3), too few figures/tables (n = 2), and not meeting the journal’s scope (n = 2), among others.”
Other issues can include a lack of funding or conflict of interest disclosures, not being indexed in scientific search engines like PubMed, and questions over whether and how to include this unvetted “gray literature” in meta-analyses and systematic reviews. The table below does a great job summarizing the pros and cons:
One of the most dramatic examples of the perils of the pre-print process was early hype for the malaria drug hydroxychloroquine during the COVID-19 pandemic. A small study suggesting it was an effective treatment by French microbiologist Didier Raoult was published as a pre-print and immediately gained traction on television news and social media. While it was later published in a journal, there were questions about the quality of peer-review, and years later it was ultimately retracted as fatally flawed and inaccurate. It was the second most cited study of all time to be retracted (and the most cited COVID retraction). Multiple other pre-print papers on hydroxychloroquine were also retracted.
Communicating Uncertainty
While fellow scientists generally understand these limitations, problems can arise when pre-prints capture public attention before they’re fully vetted. One survey found that almost 70% of people misunderstood what pre-prints are. This should not be surprising when only half of mainstream news stories about pre-prints mention the findings are preliminary and could change. So how do we harness the benefits of pre-prints while minimizing their risks? The scientific community has developed several approaches.
Many pre-print servers now include prominent disclaimers about the tentative nature of their content. Studies have found that such statements let non-scientists (appropriately) view pre-prints with less certainty. Some servers require basic screening to filter out obviously problematic submissions. Scientists are also working to improve science communication, helping journalists and the public better understand how to interpret these findings.
For readers of pre-prints, especially the general public, it’s crucial to maintain a healthy skepticism. Look for subsequent peer-reviewed versions of the research, and pay attention to how other experts in the field respond to the findings. Remember that science is a process of continuous learning, not a set of facts carved in stone.
The Future of Scientific Publishing
Despite concerns over how they might be interpreted, pre-prints appear to be here to stay. They've become an integral part of modern science, promoting faster, more open research while challenging us to think carefully about how we communicate scientific findings to the public.
For the average person interested in research, pre-prints offer an exciting window into the scientific process as it unfolds. However, this window comes with a responsibility to understand what we're seeing—not final, peer-reviewed conclusions, but rather science in its raw, dynamic state. And for scientists and medical professionals, it is on us to provide context around the reliability of breaking news and call out excessive hype.
As we continue to navigate this new landscape, perhaps the key is to embrace both the traditional and modern approaches: leveraging the speed and openness of pre-prints while maintaining the crucial quality control of peer review. After all, in science as in many things, the goal isn't just to move fast—it’s to move as quickly as possible while still getting things right.
I was impressed with the qualities of the pre-prints during the pandemic. great for looking at the data. There were even a few occasions that I read some of the reviews asking questions and minor corrections. Now they are now endemic and lost a lot of the qualities that made them worth reviewing.
Love the pre-print overview! Thank you