The Journalology newsletter helps editors and publishing professionals keep up to date with scholarly publishing, and guides them on how to build influential scholarly journals.
Journalology #53: Secrecy vs transparency
Hello fellow journalologists, This week’s issue covers paper mills, the rise of Chinese research, and the increasing use of generative AI in scholarly publishing. The same themes keep coming up week after week, but with new twists. Enjoy! NewsHow big is science’s fake-paper problem?An unpublished analysis shared with Nature suggests that over the past two decades, more than 400,000 research articles have been published that show strong textual similarities to known studies produced by paper mills. Around 70,000 of these were published last year alone. The analysis estimates that 1.5–2% of all scientific papers published in 2022 closely resemble paper-mill works. Among biology and medicine papers, the rate rises to 3%. Nature (Richard van-Noorden) Are there times when secrecy is better than transparency? This week Nature ran a news story claiming that 1.5-2.0% of papers published in 2022 could be produced by paper mills. However, the results haven’t been published as a preprint or as a journal article, so the wider community can’t assess whether the methodology and conclusions are valid. Rather, Adam Day privately shared the data with Nature’s journalists, who used them as the basis of the news story. There are good reasons not to share the data publicly — doing so could help the fraudsters evade detection in the future. Is that need for secrecy more important than having openly available data available for wider discussion? For example, if you’re a physics publisher, the “Subject Breakdown” figure in the news story suggests that paper mills are less of a problem. However, it’s impossible to know if that’s the case without seeing the data. In 2005 Nature published an editorial about dual-use research (e.g. research that could be used by terrorists to weaponise a pathogen), which concluded: It is important to develop clear guidelines about what research is considered sensitive, what is expected of researchers whose work produces dual-use outcomes, and how the government should in practice respond without losing the priceless virtues of open scientific scrutiny. Paper mills are a significant challenge for our industry. However, at a time when openness and transparency are being championed, is secrecy acceptable? Should Nature be running a news story about an individual’s claims when the data aren’t public, and when he could gain financially from having the publicity that Nature provides? To be clear, I’m not criticising Adam for his approach. I don’t know whether ‘open’ or ‘private’ is best for this kind of data set (FWIW, my gut instinct is that Adam is right to keep the data locked away, but I’d also like to be able to see the numbers for myself). How to make research reproducible: psychology protocol gives 86% success rateIn a bid to restore its reputation, experimental psychology has now brought its A game to the laboratory. A group of heavy-hitters in the field spent five years working on new research projects under the most rigorous and careful experimental conditions possible and getting each other’s labs to try to reproduce the findings.
Published today in Nature Human Behaviour, the results show that the original findings could be replicated 86% of the time — significantly better than the 50% success rate reported by some systematic replication efforts.
The study, the authors say, shows that research in the field can indeed be top quality if all of the right steps are taken.
Nature (David Adam) Biochemical Society launches innovative Subscribe to Open model for its journalsThe Biochemical Society (and its trading arm, Portland Press Ltd) is delighted to announce the launch of Subscribe to Open (S2O) for five of its world-leading research and review journals. This move marks another significant step in the Society's commitment to making research accessible while maintaining the highest standards of quality. Biochemical Society (press release) JB: This week the Biochemical Society also announced a Read & Publish agreement with the Big Ten Academic Alliance. Open access: Plan S 2.0 bold but ineffective?Robert-Jan Smits, one of the architects of Plan S while he was director general of research and innovation at the European Commission, said the tougher line was the result of the fact that large commercial publishers had not “pulled their weight” in the shift towards open access. “They want to get things done and are fed up with the delays,” he said of the funders’ perspective. “The large commercial publishers can only blame it on themselves that there is now a Plan S 2.0.” Times Higher Education (Alicia Kowaltowski) JB: Is anyone else’s blood boiling after reading this? The truth that funders don’t want to hear is that many researchers don’t much care about open access; it’s certainly a lower priority for them than impact factors, according to the author surveys that I’ve seen over the years. To blame the slow uptake on commercial publishers is utterly ridiculous. As the story notes: He [Rick Anderson] said that Coalition S had yet to convince the global research ecosystem to adopt its vision and seemed outraged by this, adding that the group’s slow growth in recent years suggested that anything close to global adoption remained unlikely. Plan S has lost a lot of credibility over the past 12 months, in my opinion. Publishers are less likely to cooperate collegially with Plan S in the future, which means that we’re going to end up with an even more adversarial environment. This is bad news for scholarly communication. Project MUSE Accelerates Move to Open Access with Publisher S2O CommitmentsLeading humanities and social sciences platform Project MUSE announces that many of our university press and related scholarly publisher partners have already committed to participate in the launch of our Subscribe to Open (S2O) program for journals in 2025. Fifty journals from more than 20 publishers are confirmed for participation to date, with more expected to join before the end of the year. Project MUSE (press release) Clarivate Report Reveals China Challenging U.S. Research Dominance, Signaling Future Research Ambitions
China’s research landscape draws on data from the Web of Science™ citation index of scientific publications and charts China’s remarkable research trajectory over the last 40 years. The report shows an acceleration in published research output, which increased five-fold between 2009 and 2021, well outpacing the U.S. and E.U. Analysis from the Institute for Scientific Information suggests this trajectory is likely to continue. Clarivate (press release) JB: This week Nature Index published a series of articles on the theme of Rising Stars, which covers similar ground. China’s rapid rise in the Nature Index is unparalleled, but where does it go to from here? Can the United States and India partner up to reach new heights? Nature Index Rising Stars 2023 explores the countries and institutions that have exceeded expectations in their high-quality research output. Broadening audience, increasing understandingToday we are launching a new pilot project aimed at using LLMs to increase accessibility of content on bioRxiv. Every bioRxiv preprint will now be posted with three AI-generated summaries, each created for a different kind of reader: someone with little or no scientific training; a scientist with expertise in a different field; and someone whose expertise equals the author's. The summaries are created from the full text of the preprint, not just the abstract. They can be found under the Automated Services tab on the dashboard, which can be launched by the widget under the list of authors. bioRxiv (announcement) JB: This sounds great, as long as the summaries are accurate and useful. There’s no mention of any kind of quality control process in the announcement. Is anyone reading them to make sure they make sense prior to publication? Alarm bells are ringing for me. Am I right to be concerned? ‘ChatGPT detector’ catches AI-generated papers with unprecedented accuracyA machine-learning tool can easily spot when chemistry papers are written using the chatbot ChatGPT, according to a study published on 6 November in Cell Reports Physical Science. The specialized classifier, which outperformed two existing artificial intelligence (AI) detectors, could help academic publishers to identify papers created by AI text generators.
“Most of the field of text analysis wants a really general detector that will work on anything,” says co-author Heather Desaire, a chemist at the University of Kansas in Lawrence. But by making a tool that focuses on a particular type of paper, “we were really going after accuracy”.
Nature (McKenzie Prillaman) Taylor & Francis Pilots Duplicate Submission Detection ToolThe duplicate submission tool is part of a suite of initiatives in the STM Integrity Hub, developed by members to safeguard research integrity. Taylor & Francis is also testing a paper mill detection tool, which uses innovative technology to spot signs of possible paper mill characteristics in submitted papers.
Alongside Taylor & Francis, publishers already taking part in the duplicate submission tool pilot include IEEE, IOP Publishing, ACS Publications, Sage, and PeerJ.
Taylor & Francis (press release) OpinionWho Is an Author? Finding the Balance Between Contribution and AccountabilityWe currently offer our authors the opportunity to use the free Paperpal Preflight service, which uses AI to review a submission for common errors and concerns. Beginning this fall, we will use AI to write patient summaries for each full-length research article published in JU. Authors will be asked to sign off on the content before we post the summaries on our AUANews digital platform. We will also use AI to “read” each author-written JU Insight so global readership can consume our content by listening to it. These AI-driven improvements allow our authors to concentrate on the research while we drive engagement to their important work. Journal of Urology (Jennifer Regala and D. Robert Siemens) Bringing Generative AI to the Web of ScienceWe are implementing AI in the Web of Science with a focus on automating tasks that are labor intensive for researchers but do not require expertise, such as extracting, restructuring, summarizing, and presenting requested information from our meticulously structured metadata. The next step in our AI journey is the Web of Science AI Research Assistant, a generative-AI-powered tool that opens new modes of utility for users. Researchers can ask questions and more easily uncover the right answers from Web of Science data, with the AI Research Assistant cutting through the complexity of data and making connections between articles, as well as helping them to browse concise summaries of articles and results sets. With this new conversational capability, the Web of Science user will be more agile than ever, freeing up valuable time for activities that add new knowledge to the ecosystem. Clarivate (Francesca Buckland) JB: This follows on from the announcement back in June that Clarivate had partnered with AI21 Labs. Kitchen Essentials: An Interview with Juan Pablo Alperin and John Willinsky of PKPI think that the most promising opportunities ahead for the infrastructure community involve the current initiatives underway with publisher organizations to improve how we address issues of research integrity at multiple levels. Most of the attention to date has been on data falsification, paper mills, and reviewer scams. But there’s a related opportunity in this age of misinformation, which recognizes that growth of open access (however slow some of us find the pace) carries with it a responsibility for infrastructure developers and publishers to help inform and educate the public about the scholarly publishing standards that distinguish journals as a trusted information source in these trying times. The Scholarly Kitchen (Alice Meadows interviews Juan Pablo Alperin and John Willinsky) Correction is courageousResearchers are human and therefore have opinions. They also are fallible. The same holds true for journal editors and peer reviewers of research papers. A better approach is to strengthen the process for correcting errors in the scientific record with expedience and transparency. A recent study by Kathleen Hall Jamieson and colleagues suggests that public trust of such a system would go a long way in building trust across ideologies. Unfortunately, the sluggishness of journals to correct errors, the silence of research institutions regarding alleged errors, and the defensiveness of researchers have created an environment that is difficult to change. Science (H. Holden Thorp) Journal ClubThe Oligopoly’s Shift to Open Access. How the Big Five Academic Publishers Profit from Article Processing ChargesUsing publication data from WoS, OA status from Unpaywall and annual APC prices from open datasets and historical fees retrieved via the Internet Archive Wayback Machine, we estimate that globally authors paid $1.06 billion in publication fees to these publishers from 2015–2018. Revenue from gold OA amounted to $612.5 million, while $448.3 million was obtained for publishing OA in hybrid journals. Among the five publishers, Springer-Nature made the most revenue from OA ($589.7 million), followed by Elsevier ($221.4 million), Wiley ($114.3 million), Taylor & Francis ($76.8 million) and Sage ($31.6 million). With Elsevier and Wiley making most of APC revenue from hybrid fees and others focusing on gold, different OA strategies could be observed between publishers. Quantitative Science Studies (Leigh-Ann Butler et al). JB: We’re now at the end of 2023 so it’s not clear to me why the authors used data from 2015 to 2018. Here’s the table showing the estimates for the publishers. It’s a shame no one copy edited the table prior to publication. This chart grabbed my attention too, for obvious reasons; it shows the commercial benefits of developing a portfolio strategy. N.B. Only journals from the 5 commercial publishers are included. The graph would look very different now, I suspect, with MDPI and Frontiers journals dominating. As an aside, Nature Communications has published 6865 research articles year-to-date, which means that it may hit 8000 research articles by the end of the year if the run rate continues (data source: Nature Communications website). At the current list price of $6490 that means the journal would generate $51.6m in revenue in 2023. That figure is almost certainly an overestimate because the Christmas holidays, currency effects, and waivers will mean the actual figure is likely lower than that. But it shows that OA journals can be editorially excellent and also be commercially successful. Nature Communications has never experienced exponential growth, which is one of the secrets of its success, in my opinion. It’s important to remember that this output is only possible because of the in-house editorial team. There are 147 editors listed on the website (yes, I counted them...) who are all permanent members of staff. You can do your own calculations on what the overhead cost for these staff might be (remember to include national insurance, pension, healthcare, office costs etc in your per-head cost). Then you need to factor in the support staff who work alongside these editors (production, editorial assistants, technology, legal, HR, finance, marketing etc) and the cost base starts to get very large indeed. Recruitment for Vice President, Publishing, Nature Communications, Comms and NPJs Portfolio closed on November 8. It will be interesting to see who takes on this challenge; there’s a strong legacy to build upon. And finally...One of the big news stories outside of scholarly publishing was the keynote from the first ever OpenAI DevDay. You can watch the keynote here (1.3 million had viewed it within 3 days of it going live on YouTube). Ben Thompson from Stratechery was suitably impressed. Ian Mulvany provided insight into what the announcements might mean for scholarly publishing (I love the “magic incantation” analogy). Until next time, James P.S. If you got this far presumably you found this newsletter to be helpful. Please do forward it on to your colleagues and encourage them to sign up. I have lots of exciting things planned for 2024 and beyond. |
Journalology
The Journalology newsletter helps editors and publishing professionals keep up to date with scholarly publishing, and guides them on how to build influential scholarly journals.
Subscribe to newsletter Hello fellow journalologists, I tend to write these newsletters quickly and sometimes I make a mistake. Last week I said: Pablo Gómez Barreiro’s commentary was published on March 23 and presumably hadn’t been seen by Stefan Tochev, the MDPI CEO, by the time he published his latest newsletter on April 30, which used the original graphs. Somehow I missed this sentence from Stefan’s newsletter: An alternative approach, using weighted average by publication volume, shows...
Subscribe to newsletter Hello fellow journalologists, Normal service returns this week, with a plethora of news, opinion and journal articles for your delectation. Share and enjoy. A coaching testimonial from an Editor-in-Chief I can recommend the coaching program without reservation. It’s been invaluable to be able to discuss the many issues that come up for editors. James is not only insightful but also well-organized so that the coaching is a great investment. Professor Kathryn Phillips...
Subscribe to newsletter Hello fellow journalologists, There’s two weeks’ worth of news to catch up on, as I was on vacation with my family over Easter. So I’ve adopted the same format as the previous issue, primarily to make this newsletter as concise as possible. As before, the text that follows has been pasted from news sources and is not my own. I’ve mixed in a few opinion pieces that I enjoyed and grouped similar topics (research integrity, AI etc.) together. But first, I hope you’ll...