The evidence behind pandemic policies

UNCOVER is a network of hard-working students and staff who carry out rapid evidence reviews to support policy makers with key covid-19 policy decisions. Article by Lucy Saddler

As the nation stayed at home and hospital beds filled up with coronavirus patients in March 2020, Ruth McQuillan – a senior lecturer at the University of Edinburgh’s Usher Institute – was pondering how staff and students could respond to the emerging pandemic.

“Governments were having to make decisions about things like masks and they didn’t have the evidence, or they didn’t know whether covid transmitted more indoors or outdoors.

“There were so many questions, but people didn’t know the answers to them,” Ruth says.

She recognised that students and staff from the institute had the fundamental skills to conduct evidence reviews to provide policymakers with the concise and reliable information they needed to make those all-important public health decisions.

Alongside fellow lecturer at the Usher Institute, Professor Harry Campbell, she contacted colleagues to gauge interest. A volunteer army of both students and staff formed and UNCOVER – the Usher Network for Covid-19 Evidence Reviews – was born, with the team responding to requests for evidence reviews from key policy-making networks immediately.

During the very early stages of the pandemic, the team sometimes had less than 48 hours to complete the reviews.

“Before the pandemic, a systematic review would take you more than six months. But the world had to change because people couldn’t wait for that amount of time, so we had to find ways of doing it more quickly” Ruth adds.

From its genesis in the midst of a global pandemic, Ruth says that UNCOVER has “continued to grow and grow”.

UNCOVER is based at the Usher Institute, one of five data-driven innovation (DDI) hubs, which aims to deliver data-driven advances in the health and social care sectors.

The DDI has directly provided UNCOVER with funding that has allowed them to pay students for their work and employ postgraduates as interns.

The first step was to refine the question and then register the study online, to avoid duplication. Next is to write a search strategy – so for a question about the association between alcohol and compliance with covid regulations, the search strategy would list terms related to alcohol and other terms such as social distancing and mask wearing.

A database search produces a long list of relevant papers, and the screening process begins.

A piece of software then randomly assigns team members a paper to look at and based on a screen of the title and abstract, it is either accepted or rejected.

Each paper is screened twice by a different team member to avoid bias. If there is a conflict, a third person reviews the paper and takes the final decision.

The team might then end up with a list of 200 papers. They perform a similar process as before but this time, they screen the whole text before deciding if the paper is rejected or accepted.

From the papers that are left, information and evidence is then extracted before being summarised in a report for policymakers.

At all stages of the process, the team have to balance the need for speed with the need for accuracy and rigour.

Ruth McQuillan explains that the key to achieving this balance is total transparency:

“It’s about caveats… being transparent and not claiming things beyond what is possible to claim.

“If the answer to the question is that there is no evidence, then that is the answer.”

With an emphasis on collaboration between students and staff at the university, the UNCOVER working model is unique and has many benefits, as Ruth explains:

“You get people who are really committed and doing it for the right reasons.

“Giving up time to do extra work, in the middle of the night…that is a special sort of student and we have got special people.”

 

Invaluable experience

Jodie Fryer became involved with UNCOVER during the fourth year of her undergraduate infectious diseases degree.

“To be honest, I didn’t know a great deal about UNCOVER. I was told that it would be good experience and they did reviews – none of it was anything I’d done before,” says Jodie.

Jodie’s first review with UNCOVER was looking at how to make buildings more pandemic proof and she is currently working on a review for Public Health Scotland, that examines the effect the pandemic has had on student mental wellbeing.

Jodie believes that the experience and skills she has gained from UNCOVER will be invaluable as she progresses in her studies.

“I have a better understanding of what I’ll be doing practically, and I might have a head start on other people, having done the process so much,” she explains.

UNCOVER has expanded to support self-directed student research projects – for example Student Selected Component (SSC) projects that medical students take in fifth year.

 

Rewarding research

Peter Sloan is doing his SSC project through UNCOVER and, since October 2020, has been examining the changes in a local authority’s reproduction number when students return to university.

“It’s quite easy to see that case rates of covid in students around the times of September, October and November increased – but the extent to which that is impacting the wider community in terms of transmission of covid is less clear,” he explains.

Although the project is self-led, Peter is supported by Ruth McQuillan, Professor Harry Campbell and Dr Li You (Leo), a research fellow.

Peter has found acquiring new skills and discovering new ways of conducting research to be the most rewarding element of his involvement with UNCOVER.

“I didn’t think that I’d ever be able to get involved with statistical analysis. I found the whole concept intimidating. When I began this project, it wasn’t with the aim of doing any mathematical analysis. I was expecting to do some sort of systematic review, but it evolved in its own way.”

 

Ensuring progress

As the pandemic continues, UNCOVER will continue to complete rapid reviews as well as support student research projects and dissertations.

Ruth especially hopes to establish long-term links with agencies such as Public Health Scotland, with whom the network is currently working on two research projects.

“We could have internships where UNCOVER students graduate and go there to work and staff secondments too. That would be nice to develop, but that will take time,” she remarks.

The network has often struggled to find appropriate funding, with most academic funding prioritising research projects over the network’s infrastructure and capacity building.

The Data Driven Innovation initiative (DDI) has nonetheless supported the network from its infancy onwards wherever possible. UNCOVER received funding through the DDI’s ‘Building Back Better’ open funding call, which enabled the network to undertake those two research projects with Public Health Scotland.

“The DDI saw the potential in what we do, and they have supported us. Finding an organisation like that, with a good vision, is quite difficult.  So that’s a challenge for us. How do we keep this going?”

But for now, Ruth adds, “we take it one step at a time.”

 

The project is funded by the Data-Driven Innovation initiative (DDI), delivered by the University of Edinburgh and Heriot-Watt University for the Edinburgh and South East Scotland City Region Deal. DDI is an innovation network helping organisations tackle challenges for industry and society by doing data right to support Edinburgh in its ambition to become the data capital of Europe.