I recently interviewed the Dutch scientist, Dr. Elisabeth Bik, a microbiome and science integrity consultant. After years of working as a research associate at Stanford University and as the scientific editorial director at uBiome, she turned consultant to follow her passion of scientific integrity work. The following transcript is an edited version of comments and our conversation.
Becoming an Ornithologist Was My Dream
Bik: “Already in elementary school I pictured myself as a biologist, studying birds. Becoming an ornithologist was my dream – I knew that word since I was eight years old. I was already then a bit of geek or nerd! I really wanted to study Biology, and that’s what I did. Though when I went to college, I did not really see myself becoming a teacher, I rather enjoyed doing research. Following the “natural path” and in line with what is common when studying Biology, I extended my studies by pursuing a PhD in Microbiology.”
“Scientific Integrity Work Is a Passion of Mine – Coincidences Led Me to It”
It all started with reading about plagiarism. The result was doing detective work that started with taking one sentence out of a paper that Bik had written several years back. She searched in Google Scholar – in-between quotes – which led to another paper that incorporated the same sentence, a paper written by someone else. As Bik investigated that paper, she quickly realized that all these sentences were “stolen” from various papers. This paper was a patchwork of different papers and being passed off as their own, new review paper. After investigating each sentence within the “stolen” paper, she found some sentences had been used in yet a third paper. The “stolen sentences” from the original papers were reused yet again for a third plagiarized paper. “It turned out to be this hair ball and once you started pulling on it, a bunch of papers came up that all had plagiarized text,” Bik stated. Bik was consumed by this major specific science integrity research project for a year, as she was researching every single sentence via Google Scholar to find the original source. It turned out that for 80 to 90 percent of these papers, the text seemed to have originated from other papers. She assumed that the sentences that she couldn’t find were simply not indexed by Google Scholar. Bik highlighted each sentence noting the original source, before sending the findings back to the editors. As a result, some of these papers got retracted.
Another coincidence, involved a PhD thesis with plagiarized text and image duplications. Her sharp eyes and good memory recognized that a particular western blot, with a little smear, was also published somewhere else purporting to be a different experiment. As she decided to check a few PLOS ONE publications, using the search term “western blots”, she quickly identified additional examples. As often in life, these little coincidences can have a big impact– it completely changed Bik’s life. Bik was hooked and turned her hobby into her daily, full-time work.
“It turned out to be this hair ball and once you started pulling on it, a bunch of papers came up that all had plagiarized text,”
Most of her work is unpaid volunteer work. Occasionally, she works as a consultant for universities whose faculty members might have been accused by others on PubPeer for image irregularities. These universities want her to look into these papers (e.g., 10 papers that have been flagged) and see whether Bik agrees with the concern(s) raised. She only takes on consultancy work for cases that have not been raised by her, which often involves comparing raw images to published ones. In addition, she also worked for some publishers to look at various papers from a particular research group. All these consultancy jobs are dealing with 10-20 papers, not hundreds, and they usually entail a careful, detailed review of all images and a report of the findings which can take hours, even days, to produce.
Paper Mills Are the Organized Crime of Scientific Misconduct
Paper mills have a following partly because the Chinese system demands at least one publication as a graduation requirement for medical doctors. Since students are often not given the time to do research, they “buy” a paper. Essentially the students buy authorship to fulfill the graduate requirement on time and to get a hospital appointment. Although the publications appear real and valid, they are completely fake, including falsified and recycled images and experimental data found across multiple papers. These papers often have a central focus on small non-coding RNAs affecting a particular pathway in a particular cancer type. Since they are niche papers, one needs to be an expert to follow and understand the content, and with little impact on a particular field, they fly under the radar. Paper mill organizations generate numerous similar papers that simply replace gastric cancer with colon cancer, cervical cancer, or prostate cancer. By replacing the name of the non-coding RNA with something else a new paper is created. A paper produced by a paper mill can cost $8,000 to $10,000 USD. As forgeries are identified by individuals like Elisabeth Bik, the publication is retracted without any protest or consequence. “So it seems that our hypothesis is correct, and that these papers are indeed fake,” Bik stated. By March 2021, Bik and some paper mill detectives had collectively listed more than 1,300 articles as possibly being generated by paper mills.
“the scientific writing industry is not enforced by any legal laws.”
It turns out paper mill services are hard to pinpoint, as they often work with English language editing services. While they seem to have a virtual backdoor that you have to know (i.e., vaguely worded Chinese advertisements offer these types of services). Unfortunately, the scientific writing industry is not enforced by any laws, but it’s certainly unethical.
“Some journals seem to be completely overtaken by these productions, and we are unable to concretely discern if the editors are aware”, Bik stated. Although they are predominantly low-impact factor, albeit legitimate journals, they are nevertheless included in the Scientific Citation Index (SCI). “When we identify these fake papers, some journals are genuinely shocked. However, we also believe some journals might have been in the loop or, at least, aware but do not take any actions. As these journals publish on the open access model, which is a great model, they receive money from the authors to publish papers. So, I’m afraid they might just not care enough about the quality but more about the quantity/money”, Bik commented.
China as a Nation Wants to Be a High Contributor to Scientific Literature
It appears that the Central Committee of the Communist Party of China (CCCPC) – a political body comprised of top party leaders – enforces this graduation publication requirement, maybe without fully understanding the detrimental effects. Furthermore, since the papers have already been published and are fully accessible, they are included in the statistics, meaning the retractions have minimal impact. The focus is, unfortunately, on quantity not quality, so this focus fosters the business of paper mills.
Indian University Ranking
Of course, this can be seen in other countries as well. A particular University in India, for example, pushes publications, which carry a heavy weight in the calculation of the Indian University ranking scale. This specific university appears to enforce the rule that professors publish as much as possible. Bik found at least 257 papers from that particular university with plagiarized text and manipulated images. Interestingly, only 15 of these papers have been retracted so far.
Paper Mills Are Also Found in Russia and Iran
While paper mills and the drive to increase university rankings are clearly more fraudulent examples taking place predominantly in India and China, scientific misconduct is not systemic. It is mostly only seen with specific individual researchers or specific labs throughout the world, with paper mills also found in Russia and Iran. Though, unlike Chinese and Indian paper mills, the publications there tend to be not in English language, making them harder to identify and allowing them to remain under the radar.
Scientific Fraud Doesn’t Happen Universally but Is Centered Around Specific Researchers or Labs
To-date Bik has identified about 5,000 papers with problems and this just by looking at duplicated images. Specifically, these papers have an image used twice within the same paper, or two images have overlaps, or have images that have been completely manipulated where, for example, one cell is visible ten times in the same photo. Bik estimates, based on her own findings, that about 4 percent of all papers that include images (e.g., an image of cells, a western blot, or a DNA gel) are problematic. This is, for obvious reasons, quite problematic in itself, as it only points out papers that have image duplications/issues within the same paper. Papers that have duplicated images across other publications are harder to trace, but one would imagine that the percentage of problematic publications of this type of fraudulence is likely much higher. Bik in her work only focuses on papers with image duplications within the same publication (see an example in Figure 1).
Figure 1: An example of a retracted, open access paper with image issues posted on PubPeer.
Then there are of course the publications that contain plagiarized text, which is yet another ethical problem or a conflict of interest, which Bik believes are the minority of papers.
“I’m only really finding the tip of the iceberg. The rest is under water. I would not be able to find those just by looking at images. So that’s just a scary thought.”
Only 10% of Papers With Duplicated Images Have Been Retracted
To-date only about 10 percent of all papers from the initial stack of 800 problematic papers Bik has identified have been retracted after a period of five-six years, and in some cases even longer. 30% have been corrected, which, as Bik stated, is positive as she believes some are honest errors, whereas with overlaps in different images, it’s hard to know what the true situation is. Of course, this could be corrected with a new set of images and a correction statement.
“If the same cell is visible multiple times within the same photo, then I don’t believe ‘this is an error’ type of excuse. So those, in my opinion, should be retracted,” Bik said. “Interestingly, in many cases these issues have not been addressed by the journals. The journals did not take any action in 60% of these cases, which is horrible, disappointing, and frustrating, and, of course, a shame for scientific publishing. Interestingly, this seems journal-dependent as groups of papers from the same journal where the editor does nothing about the misconduct, and simply states, ‘Okay. Thank you. We will take a look at this.’” Editors that have not taken any action are not just with one publisher, but includes well recognized publications, such as PNAS, Science, Wiley, Elsevier, or Spandidos publications.
There Is no Good Software yet for Scaled Image Scanning and Comparison to All Published Images
Currently, there is no good software on the market that can scan images in a submitted publication and compare them to all the images published. The computational effort of such a comparison would take hours to complete, and with likely several hundred submissions a day, no publisher would want to undertake such an effort. Technically, it should be a task that can be accomplished, but before this sort of task can successfully be implemented the computational aspect needs to be improved with regard to task size and speed. Eventually, computational capacity and algorithm optimization will improve to make this technically feasible.
“For now, some software tools can scan the images within a paper and compare them to each other, thus finding duplications that I did not. But there are also instances where this approach can’t find the duplications I have found,” Bik pointed out. The software still requires some improvement. Building the right software tool is much harder than most people think, with an exceptional challenge being the extraction of images from a PDF document. For example, the software is challenged with ignoring labels within an image, that are not part of the image – a task the human brain is very good at. Bik was part of a DARPA challenge three years ago, for which Bik provided several duplicate images to train the software tool. The challenge included several teams. Though the data collected is interesting, the teams have not yet been able to solve the problem.
One software Bik uses and seem to work well, is ImageTwin, which was developed by Markus Zlabinger, a computer scientist at the Technische Universität in Vienna, Austria.
The Didier Raoult Story
The Didier Raoult case started when Bik questioned the experimental design, study size, and conclusions from the infamous COVID-19 related hydroxychloroquine study. As a result of Bik’s work and findings, Raoult threatened Bik with a lawsuit, most certainly with the goal to silence her. Bik stated in our conversation that she does not really have a thick skin, so the attack took her by surprise. Bik first questioned the hydroxychloroquine paper by Raoult, and subsequently identified other Raoult papers with image problems. In total, she identified about 20 papers with image problems plus 40 additional published papers by his group with potential ethical concerns. In particular, Bik noticed a set of papers where studies were performed involving patients in African countries with no wording about local authority involvement or necessary permissions and approvals in place. It certainly seems that Raoult and his team of researchers went to the country where the study was to be conducted, sampled individual people, and then took the results back to get them published. Bik compared this study approach to neo-colonial science. Clearly, it’s not just the hydroxychloroquine study that is questionable when it comes to the Didier Raoult research and publication activities, it goes far beyond (see also the Didier Raoult callout box) and even includes image duplication issues identified across several of Raoult’s papers.
While Bik only learned of Raoult’s complaint via a YouTube video in which Raoult’s lawyer accuses Bik of harassment, blackmailing, and extortion, Bik never received an official complaint, nor has she been told of the exact wrongdoing that led to the accusations in the YouTube video. Certainly Bik’s active presence on social media, including Twitter, has helped uncover and disseminate some of her scientific fraud findings to the wider scientific community. While she is actively sharing her findings, they can certainly not be labeled as extortion or an attempt of blackmailing. The scientific community should be able to freely speak up and raise criticism if scientific integrity is at stake. Interestingly, Raoult to-date has still not addressed any of Bik’s questions raised on PubPeer about the hydroxychloroquine study or the image duplications identified. Instead he has chosen the legal path by threatening with a lawsuit. Since, it was announced the Raoult will be replaced by a successor in June 2022 by the IHU – Méditerranée Infection.
“The scientific community should be able to freely speak up and raise criticism if scientific integrity is at stake.”
Unfortunately, following the Raoult case, Bik pretty quickly received additional letters from a couple of lawyers representing another group of researchers associated with a set of 6-7 papers completely unrelated to the Raoult case. This set of authors – from institutions across North America (including Harvard) and Canada – includes a Qigong Master. These authors published a whole set of papers which repeatedly used the same technique. “Unfortunately, the technique was not clearly explained,” as Bik pointed out. “It basically described how the cancer cells were placed in a closed room while the Qigong Master transferred his energy into the cells, thus irradiating them and causing them to die. Since this type of experiment is not reproducible outside the very same room, this is not considered a scientific study.” Stephanie Lee wrote a great article on Buzzfeed about this issue.
Bik criticized the Qigong study in 2019 and has now received threats of lawsuits from a couple of law firms. Bik maintains that her criticism is warranted, and the researchers and/or publishers can publicly respond to her comments.
The U.S. Office of Research Integrity (ORI) Only Investigates a Couple Of Cases Per Year
In the U.S we have the Office of Research Integrity (ORI), which is meant to oversee and implement the guidelines for publishing scientific papers. Specifically, the ORI oversees and directs Public Health Service (PHS) research integrity activities on behalf of the Secretary of Health and Human Services. However, they unfortunately only investigate a couple of cases a year, per Bik’s statement. Besides, they have never replied to any of Bik’s cases. Even within the ORI, there seems not enough monetary support or manpower for these investigations.
The PHS is composed of several offices and agencies (e.g., Office of Public Health and Science, National Institutes of Health, The Centers for Disease Control and Prevention, and The Food and Drug Administration to name a few) and provides nearly $38 billion for health research and development, primarily in the biomedical and behavioral sciences through its extramural and intramural programs
There is also the forum for editors of peer-reviewed journals called COPE (Committee of Publication Ethics) which promotes and exists to address the integrity of the scientific record. It supports and encourages editors to report, catalogue, and instigate investigations into ethical problems in the publication process. Most journals are members of COPE. Bik noted: “even though COPE has clear guidelines, many journals seem not to follow them very well. Unfortunately, the usefulness of these guidelines is questioned quickly, as the process laid out in flow charts quickly seem to break down when an author doesn’t reply to concerns. As a result, there are many dead ends in these flow charts and the journals can’t effectively work with them. Not to mention that authors change institutions and, with that, their contact information. Hence, it often can be difficult to reach authors for specific fraud claims.”
The Call to Action for the Scientific Community
Bik:
- “That’s a pretty big question. In general, I’m frustrated with the lack of responses of journal editors or research institutions to claims of scientific fraud and when raising serious concerns about published research papers. While I can show proof of image manipulation in hundreds of examples, scientific journals often don’t take any action which is very frustrating as it doesn’t serve the reader. The reader should be made aware if there is a potential flaw within a paper. This is particularly important when citing such a paper and/or basing one’s own research on such a paper.”
- “When it comes to paper mills, we need much better safeguards to perform peer reviews to catch fake papers. Unfortunately, that’s a huge undertaking as the paper mills make excellent forgeries that are hard to recognize. I can recognize some of these by eye, but we need much better technological support to find these fake papers. I believe we need more proof that the research within the manuscript has actually been executed. Publishers and institutions need to better investigate these cases, instead of protecting their star researchers or colleagues. Too many conflicts of interest surround the investigation of these cases.”
- “I would love to be involved in creating better guidelines. I would love to see that happening at the international level since science is conducted globally, with researchers moving or research conducted by multiple researchers from multiple countries. These cases are complex and require the attention of many to investigate them. Besides we need “neutral” people to investigate fraud cases, as there are many with conflicts of interest that need to be avoided or overcome. It shows that affected institutions often don’t do anything either. Unfortunately, in such situations there are too often no real consequences. Only an international organization that has no conflict of interest can address this issue.”
For now, Bik works on one case at a time. All the work she is doing, she is doing alone, including prepping her talks, providing interviews, and acting as an administrator, which takes time from her central mission of uncovering scientific misconduct. Her work hopefully empowers institutions and publishers to become less gullible and more committed to initiating necessary action that includes corrections and/or reactions.
What Does the Future Hold?
Bik seems to enjoy what she is doing and the compensation she receives does allow her to do this for now full time. She can do this whenever while contributing to an important facet of scientific research and integrity. While she isn’t planning to return to an industry job, she would consider a grant-supported academia position. Though, she sees the challenge with such a position which might impact her freedom and come with conflicts of interest.
“Right now, I’m not employed and I have all the freedom I need to do my work – this is priceless! I am not held back by any limitations.”
