Policy: Publication of UKRI Report into Peer Review
The report brings together evidence on the use and effectiveness of different peer review processes.
UK Research and Innovation (UKRI) has published a report reviewing interventions to the peer review processes used in research and innovation award funding. The report provides an extensive review of evidence of different types of interventions in peer review processes, their aims, drawbacks, and impact.
The review was conducted by Technopolis and was based on an extensive literature review (encompassing both academic and ‘grey’ literature), a survey of UKRI staff, and a programme of 22 interviews with representatives of UK and international research funders and a range of other stakeholders and experts in the field.
The study assessed 38 interventions, which range from small process ‘tweaks’ such as increasing/decreasing the number of reviewers per application and shortening application sections, to more fundamental changes such as partial randomisation and complete bypass of peer review. The aim of the study was to assess these 38 interventions, to establish what each of them might be useful for, and what disadvantages or hazards each might involve.
The report finds that there is a clear need to coordinate use of the interventions with the context and aims of each specific funding opportunity in question, and advises that creating bespoke funding processes tailored to the needs and aims of each funding opportunity is a clear ‘direction of travel’ for the future of R&I funding.
The report’s key recommendation is that process design should always be a constituent part of scheme design. Every funding scheme has specific aims and characteristics, and so the design of the application, review and decision-making process should be considered for each individual funding opportunity.
Further recommendations for all funders are as follows:
- Information technology (IT) systems need to have the necessary flexibility and function.
- Some interventions have the potential to become a ‘new normal’ in order to save burden and reduce bias across the board.
- Funders should monitor any interventions they undertake.
- Investigations into wider research culture must continue alongside the process interventions discussed in the report.
Professor Dame Melanie Welham, UKRI People and Culture Champion, said:
“This report and associated tool for exploring the data brings together evidence about what works in peer review.
“We hope the synthesis will be useful to research and innovation funders across the globe when designing and delivering peer review.
“UKRI will be using the findings and recommendations from the report to reflect on and evolve our own peer review processes over the coming years.”
The report can be accessed in full at the UKRI website.
(This report was the subject of a ResearchConnect Newsflash.)