Prioritising the Research Agenda for Trial Retention – Birmingham, 23rd October 2018

I’ve been in Birmingham today for the final consensus meeting of the PRioRiTy II project, so I thought I’d write a quick blog post before dinner so that you can find out a bit more about the process of research prioritisation.

Image credit: Prof Shaun Treweek

I’ve spoken on this blog before about the first PRioRiTy project, which was a prioritisation of questions around trial recruitment. That project took the same shape, though I’ve been more heavily involved in PRioRiTy II because it’s led by Dr Katie Gillies, who was one of my PhD supervisors; she’s now one of my line managers and she’s fantastic to work with. Katie was a participant in the consensus meeting for PRioRiTy, and as soon as she came back from that meeting (almost 2 years ago!), she set to work on PRioRiTy II – she moves fast because she is a complete trial methods nerd, just like me (I’ve found my people!).

So, I said she moves fast, but you might ask why it’s taken 2 years to get here. As with anything in research the process is not as simple as it looks. This prioritisation exercise was not simply a group of people getting together in a room today and deciding what we thought was most important; the process has involved many, many more people than just those in the room, and is based on the method that the James Lind Alliance (JLA) use for priority setting partnerships. The JLA method is designed to take into account the views and opinions of all stakeholders that have an interest in a specific research area. For us, that meant: trialists, methodologists, ethics committee members, researchers, patients, funders, clinicians and more, but the JLA method has been used for lots of prioritisation activities and the people involved are tailored each time to fit with the aims of the project.

We had a wonderful graphic illustrator with us today, and she captured the ‘story so far’ brilliantly in the image below. Before today we had a survey, followed by a huge amount of data analysis and question searching within the responses from that initial survey, an interim prioritisation process (some of you might have been involved with this because I posted about it here), and then came this face to face consensus meeting – so today was the culmination of a lot of views, opinions, time and effort.

A snapshot of a graphic illustration from the PRioRiTy II consensus meeting showing the story so far.

Below are a few photographs from the day – lots of serious faces, extensive discussion, and some compromises needing to be made too.

The kick off – Katherine Cowan (Senior Advisor to the JLA) did an excellent job chairing the workshop, keeping us all to time and making sure that everyone’s voice was heard within the discussions.

Image credit: Prof Shaun Treweek

In the first session of group work attendees shared their 3 most and 3 least important questions from the list of 21 that we had supplied them with in advance. From the initial survey responses we had 27 questions, which were then narrowed down to 21 questions during the interim prioritisation.

The second session of group work saw the beginning of the ranking process! Coloured tablecloths were used to distinguish questions that were most important (green), least important (red), and somewhere in the middle (yellow). This allowed participants to discuss the ranking of their group as a whole (i.e. based on the feedback from the first group session), and then physically move the questions into a more defined ranking position after discussion.

The final session – questions were laid on the floor so that the entire group could see the ranking. Katherine then went through each question in turn to ensure that the group could reach a consensus; harder than you might think!

Image credit: Trial Forge

We won’t be sharing the top 10 questions around trial retention just yet though; tomorrow we have our final Steering Group meeting (let me know if you’d like to see a blog post about what a Steering Group does within a research project!) where we will go through the top 20 questions and make sure that all the wording is clear.

We then plan on unveiling the top 10 at the Society of Clinical Trials meeting in New Orleans next May. If you’ve been reading the blog for a while you might remember that I went to the Society for Clinical Trials meeting earlier this year when it was in Portland. It’s a brilliant conference that enables trialists from around the world to meet each year to share their work. After that we also plan to do some more conference dissemination at the International Clinical Trials Methodology conference, which takes place in Brighton next October. Keep an eye out for future blog posts too – I’ll be posting the final top 10 when they’re released!

Image credit: Prof Jane Daniels

Publication Explainer: Trial Forge Guidance 1: what is a Study Within A Trial (SWAT)?

This is the third in my ‘Publication Explainer’ series, read the first and second here and here. As I have said previously, these explainers are a place for me to answer some of the most common questions I’ve been asked by the people around me (usually my boyfriend, friends, or colleagues that haven’t been involved with the project).

This post focusses on the paper below: Trial Forge Guidance 1: what is a Study Within A Trial (SWAT)? Read the full paper here.

What is a SWAT?

A SWAT is a Study Within A Trial – i.e. a self-contained research study that is taking place within a clinical trial. Usually SWATs focus on a methodological aspect of a trial, e.g. evaluation of: an intervention that is designed to improve the recruitment of participants to trials; an intervention that is designed to keep participants engaged with the trial (i.e. retention of participants); or an intervention that is designed to find out more about the way that data is collected (e.g. online versus paper). Often

Why are you trying to encourage people to do SWATs?

It is important that we encourage people to do SWATs because they are so often underpowered. Statisticians can calculate the sample size needed for the results to enable us to see a difference between the two interventions; if we hit that target sample size (i.e. recruit enough participants) then the result is less likely to be down to pure chance. As sample size calculations are done for the host trial, and not the SWAT, it’s likely that the SWAT will be ‘underpowered’ – meaning that the effect that we see in the results may not be a real effect; it could be down to chance. That’s ok though, because SWATs are designed to enable the data from them to be pooled with the same SWATs that have been done in other host studies.

What are you aiming to do in this paper?

This paper is the result of a huge amount of discussion, much of which started at a face to face event that was held in Aberdeen last year, the group of authors on this papers is pretty big, and that reflects everyone that took part in that event and the discussions that came after it. As a group, we are very conscious that SWATs are one of the most obvious (and arguably, easiest) ways for us to improve the way that trials are designed and conducted; so it’s important that we encourage people to do them. It is not realistic to think that trial methodologists can do all of the SWATs that we need; there just isn’t enough of us, and we need trialists to help us. By writing and publishing this piece of guidance, we aimed to produce a one-stop paper where people could go to find out what a SWAT is easily.

Within the last few days, we’ve submitted ‘Trial Forge Guidance 2: How to decide if a further Study Within A Trial (SWAT) is needed’ to the same journal, Trials. Trials journal is currently taking part in a pilot along with a number of other journals that fall under the BioMed Central umbrella, when authors submit their papers for publication they have the option of publishing a pre-print of their work. This pre-print edition is published online within about a week, meaning that the peer review process can run along side, but the research is being disseminated more more quickly. Once that pre-print is available, I’ll share it on the blog so you can read that too 🙂