Winners of the BNA Credibility Prizes 2024
17th April 2024
BNA responds to Westminster inquiry into research reproducibility
24th Nov 2021
In September, the BNA responded to a Parliamentary inquiry on reproducibility in research, highlighting that a wide variety of stakeholders need to take action to strengthen the overall system, support research careers, and boost credibility. With the Committee this week publishing all its written submissions, we want to share some of the key parts of our response.
Background to the inquiry
The House of Commons Science and Technology Committee put out a call over the summer for evidence on the scale of issues around reproducibility in research, and on how this could be tackled.
It builds on a past inquiry its predecessor Committee carried out in 2018 on research integrity, and is in response to concerns that many organisations and individuals have raised that have highlighted issues around reproducibility across research disciplines. Part of this results from the ‘publish or perish’ research culture, where questionable research practices have emerged that undermine reproducibility - including forming a hypothesis after results are known (HARKing), analysing data in multiple ways to reach significance (P-hacking) and failing to share research data.
The BNA's own Credibility in Neuroscience campaign has added to these voices by highlighting some of the reproducibility challenges within neuroscience, and how tackling these requires a focus on research culture, skills, and the broader research environment.
The importance of preregistration
Preregistering studies in an independent registry provides a clear time-stamped account of the experimental rational, hypothesis, methods, n=numbers and intended statistical analyses. It helps to combat some of the practices that undermine credibility.
This is why we are recommending that preregistration must become a standard requirement for all hypothesis-testing research, and that funders should add preregistration to their terms and conditions of grant funding.
The need for new publishing models
The publishing landscape is changing, with preprints becoming part of a broader publishing system and new publishing innovators shifting focus towards quality standards and fairer article-based metrics.
The BNA is a supporter of DORA, which discourages use of journal-based metrics such as Journal Impact Factor.
We believe that publishers need to switch the emphasis away from novel results and more towards complete reporting free from biases, which can help improve the reproducibility of the work reported. One such model to achieve this is through Registered Reports, to which the BNA Journal has recently strengthened its commitment.
Transforming incentive structures in research
We need to move away from the ‘publish or perish’ culture and incentivise researchers to make their work as reproducible as possible. Incentive structures in research need to be rethought to shift emphasis away from researchers’ publications and towards rewarding other contributions to research that contribute to reproducibility – such as publishing alternative research outputs, reviewing, training, and Team Science.
The Future Research Assessment Programme is reviewing what a healthy research system and assessment model looks like, and how this can have a positive impact on research culture.
We recommend that future research assessment should add preregistration and open data/materials to its requirements for eligibility of research outputs, building on the open access requirements of REF 2021.
Supporting people to support reproducibility
From our survey of neuroscience researchers on Journal Impact Factor (JIF), we learned that JIF is considered important for a substantial majority, and that other surveys have also highlighted its negative influence over decisions at institutional level on funding, hiring and promotion decisions.
We have recommended that institutions be encouraged to introduce hiring and promotion policies that value reproducibility, Open Science, and other factors which support credibility of research.
Institutions also need to ensure that researchers can be trained in methods to strengthen reproducibility of their work, coupled with the creation of new career pathways to provide support for managing and curating the research outputs produced.
Supporting change through added investment
Many of the supporting actions needed, for example in providing training, infrastructure, and replication studies, require funding.
The UK Government has in the past 6 months published an R&D People and Culture Strategy, aiming to address some of the concerns raised on research culture, and the results of the recent Spending Review that will raise research funding to £20 billion by 2024/25. With the funding confirmed, the UK Government needs to add teeth to its Strategy and help to provide funding that can make a difference to strengthen reproducibility in the sector.
Next steps
The Committee is currently considering all the written evidence submitted before holding further hearings and then writing its report - this is expected to be concluded in 2022, and is required to be responded to by the Government.
Interested in the BNA's responses to government? Catch up with the latest in our Policy & Advocacy section.