Please note that this is an early draft. There will be daily updates of the data contained within this paper, as well as the analysis and interpretation.
In late 2019 and early 2020, an outbreak of a novel coronavirus named “SARS-CoV-2” causing a disease named “Coronavirus Disease 2019” (Covid-19)1 became a global pandemic that affected lives, families, industries and entire nations. Medical research on humans in clinical trials was also affected, with a great amount of research activity switching to studying the disease. In an effort to safeguard the well-being of those involved and curb the spread of Covid-19, the FDA, the EMA and many individual institutions have published guidance or established policies that have resulted in the suspension, termination or withdrawal of many ongoing clinical trials due to Covid-19.1,2
These interruptions to clinical trial activity affect many different stakeholders in the enterprise of human research. The stopping of clinical trials due to Covid-19 impacts patients who were enrolled in these clinical trials, as they may not receive the treatment they had anticipated. Further, patient expectations regarding their contribution to new medical knowledge may also not be met. The large volume of stopped clinical trials due to Covid-19 also affects research institutions, investigators, researchers and others involved in running clinical trials. The Covid-19 pandemic has also caused setbacks for entire programmes of human research, possibly delaying the advent of new drugs, evidence of drugs’ efficacy in new indications or combinations, or informative negative results.
While many clinical trials were suspended with the intention of restarting after the Covid-19 pandemic, the prospect of starting again is far from certain for any clinical trial that has stopped, and even in cases where a clinical trial does resume after the pandemic, there may be reduced statistical power, more funding needed, or changes required to the protocol or analysis to accommodate for the interruption. Thus, clinical trials that were stopped and do not start again may be unable to answer the questions they set out to answer, and even in cases where they do start again, it may take greater financial resources, patients and time to make up for data that can no longer be included in analysis.
Clinical trials that were stopped and do not start again may be unable to answer the questions they set out to answer, and even in cases where they do start again, it may take greater resources to make up for data that can no longer be included in analysis due to the interruption. The large volume of stopped clinical trials due to Covid-19 also affects research institutions, investigators, researchers and other staff who run such trials. The Covid-19 pandemic has also caused a setback for entire programmes of human research, possibly delaying the advent of new drugs, evidence of drugs’ efficacy in new indications or combinations, or informative negative results.
There are several publications highlighting the extensive closures of clinical trials that have occurred due to Covid-19 policies, however the extent to which this has affected the human research enterprise is unclear.3–6
The following represents an attempt to quantify the number of clinical trials that have been terminated, suspended or withdrawn due to Covid-19 and their impact on different stakeholders in the project of clinical research on humans. We will capture the cohort of trials that stopped during the Covid-19 pandemic, identify ones that cited Covid-19 as the reason for stopping, and compare these against a comparator taken from the same timeframe, two years earlier, from which we will also estimate the baseline rate at which clinical trials that are terminated, suspended or withdrawn are re-started within 1 year, under normal circumstances. Among trials that stopped due to Covid-19, we will use patient enrollment to quantify the impact on patients, the number of trials stopped to estimate the impact on researchers and their institutions, and the number of unique drugs and biologicals listed as interventions as a proxy for the extent of the impact on drug development of terminations, suspensions and withdrawals due to Covid-19.
The Covid-19 arm of this study includes all clinical trial registry entries from ClinicalTrials.gov whose overall status changed to “Terminated”, “Suspended” or “Withdrawn” (“stopped”) since 2019-12-01. To collect this sample, all clinical trial registry records were that were updated since 2019-12-01 were downloaded, and this database was updated daily from ClinicalTrials.gov. Clinical trials that changed status to “Terminated”, “Suspended” or “Withdrawn” after 2019-12-01 were included. Clinical trials that were already stopped before this date were not included. The “why stopped” field for every trial was manually screened for whether it mentions Covid-19 explicitly, including terms such as “Covid-19”, “COVID19”, “Covid”, “NCoV-2019”, “Coronavirus”, “Corona virus”, “Corona”, “The current pandemic”, and variations on “Institution suspended all clinical research not deemed medically essential.” These were also screened for whether the “why stopped” field used language indicating that the trial was expected to resume, including terms such as “expected to resume”, “temporary suspension”, “hold”, “pause”, etc. The “why stopped” field for trials testing a Covid-19 indication that were terminated, suspended or withdrawn was also manually screened for whether it was stopped due to accrual or coordination issues, e.g. “The epidemic of COVID-19 in China has been coming to an end,” “Similar projects have been registered, and it need to be withdrawn,” or “We cannot meet number of subjects as recently published similar studies.” Other trial details were extracted from the downloaded ClinicalTrials.gov XML files, such as title, summary and detailed description, primary completion date, actual or anticipated enrollment, phase number, study type, patient allocation, intervention model, primary purpose, masking, indications, interventions and sponsors.
To provide a baseline for comparison and an estimate of the base rate at which clinical trials re-start after stopping, we captured a comparator cohort of matched length of clinical trial registry entries whose status changed to “Terminated”, “Suspended” or “Withdrawn” from two years earlier, to allow for a full year of followup time that is unaffected by Covid-19 policies for the entire comparator cohort.
All clinical trials in both cohorts were automatically coded for inclusion non-exclusively in four broad indication areas: oncology, neurologic, cardiovascular and pain by a search for the conjunction of the set of trials on ClinicalTrials.gov that match a search for that indication area and the NCT number of the trial in question.
The names of drugs and biologicals listed as interventions in trials that stopped due to Covid-19 were manually curated to remove placebos and “standard of care” arms that do not list a particular drug. Brand names were standardized to generic names where possible, and details of administration were removed (e.g. quantities, routes, schedules) to allow for easier counting of unique drugs and biologicals.
The protocol for this project was registered before data collection.7 The data sets that this report is based on are available at https://covid19.bgcarlisle.com, and will be moved to the Open Science Foundation upon completion. The code for data collection, screening and extraction that was used in this project are freely available.8
All statistical tests were two-sided, and we considered \(p<0.05\) to be significant. We used a z-test to calculate test statistics for proportions of trials in between samples. We regard any tests to be exploratory only, and we did not correct for multiple testing.
Among the 4024 clinical trial registry entries on ClinicalTrials.gov that were suspended, terminated or withdrawn (“stopped”) between 2019-12-01 and 2020-08-31, 1670 (42%) were closed with a reported reason that explicitly mentioned Covid-19. (See Figure 1, Table 1 and Covid-19 arm per-trial data set.) There was a combined enrollment of 103,832 patients in trials stopped due to Covid-19 that reported actual patient enrollment, and a combined enrollment of 4,204,344 patients in trials stopped due to Covid-19 that reported anticipated patient enrollment.
The comparator arm included 2090 clinical trials. (See Figure 2, and Comparator arm per-trial data set) There was a combined enrollment of 155,230 patients in trials in the comparator arm that reported actual patient enrollment, and a combined enrollment of 46,292 patients in trials in the comparator arm that reported anticipated patient enrollment.
Among trials that were terminated, suspended or withdrawn citing Covid-19 as the reason, 637 (38%) included language in the “why stopped?” field that explicitly indicated an intention to start the study again after the pandemic ends. Among trials in the comparator arm taken from two years prior, 212 (10%) were started again within 1 year of the date when they were first stopped.
We tested for a difference in the proportion of clinical trials terminated, suspended or withdrawn among four broad indication areas to see if there was a significant change in the number of clinical trials testing cardiovascular, neurologic, oncologic or pain indications between the clinical trials that were stopped since 2020-12-01 citing Covid-19 as the reason and matched comparator sample drawn from two years prior.
Among trials stopped by Covid-19, there was a significantly smaller proportion of terminated, suspended or withdrawn clinical trials testing cardiovascular (11% vs 13%, \(p\)=0.054) and oncologic indications (23% vs 33%, \(p\)=6.9e-12). There was a significantly larger proportion of terminated, suspended or withdrawn clinical trials testing neurologic indications (8% vs 7.1%, \(p\)=0.15). The difference in the proportion of clinical trials testing pain indications that were terminated, suspended or withdrawn due to Covid-19 vs the comparator from two years prior was not significant. See Table 2.
Among the clinical trials that were stopped with a reason citing Covid-19, 696 trials were testing drugs or biologicals; the remainder were testing procedures, devices, behavioural interventions, laboratory analyses, diagnostic tests, etc. A total of 702 unique drugs or biologicals were captured in our sample of clinical trials. (See Drugs and biologics in stopped trials data set.)
There are 66 interventional trials testing Covid-19 indications that have been terminated, suspended or withdrawn since 2019-12-01. These included 23 trials (35%) that were stopped with reasons citing an inability to accrue sufficient patients, or a coordination issue, such as another project that is too similar. These trials had a combined anticipated enrollment of 5448 patients and an actual enrollment of 464 patients. (See Covid-19 arm per-trial data set.)
These results reflect a massive disruption of clinical trials due to Covid-19. There were 623,669 more patients enrolled in clinical trials that stopped between 2019-12-01 and 2020-08-31 than enrolled among clinical trials that stopped in the time period of the same length starting 2017-12-01, suggesting a high level of impact on patients. There were also 1934 more clinical trials that stopped between 2019-12-01 and 2020-08-31 than clinical trials that stopped in the time period of the same length starting 2017-12-01, suggesting a high level of high level of impact on researchers and their institutions. A total of 702 unique drugs and biologicals were tested among trials that were stopped due to Covid-19, which suggests broad impacts on many programmes of medical research.
While the base rate for stopped clinical trials re-starting again within one year is about 1 in 10, 4 in 10 trials that stopped due to Covid-19 included language in their reported reason for stopping that indicated they anticipated resuming the trial at a later date. The situation here is certainly different, and so it is difficult to say whether the stated intentions of the investigators or the historical base rate will be a better predictor of the rate at which stopped clinical trials will resume. Due to the nature of some clinical trials, it will not be possible for them all to start again. Not all of the stopped clinical trials that do not eventually re-start represent a complete loss, as even underpowered trials may provide value by meta-analysis. Even in cases where a clinical trial that was stopped is able to begin again, there may be factors that increase the costs of trial completion, require more patients, or decrease the statistical power and the scientific value of the clinical trial.
The stopping of clinical trials testing Covid-19 indications is a cause for particular concern, as 8 of 26 (31%) such trials were stopped due to inability to accrue patients or other coordination issues, such as other similar projects.
This project only includes clinical trial registry entries whose overall status was updated to “Terminated”, “Suspended” or “Withdrawn” on ClinicalTrials.gov, and so these numbers do not reflect clinical trials that have not been kept up-to-date. Trials that were registered using other clinical trial registries would also not be captured. There may be more than one sufficient reason to stop a clinical trial, and so trials that explicitly mention Covid-19 may have also stopped for other reasons. Conversely, a trial that did not mention Covid-19 in their reason for stopping, as there was another sufficient reason for stopping, may have also been stopped due to Covid-19.
This project sets me up to come back and see just how many of these trials that were stopped by Covid-19 actually do restart within a year.
The Covid-19 pandemic of 2020 caused major interruptions to human research programmes, researchers and their institutions, and patients involved in clinical testing.
|Trials stopped 2019-12-01 to 2020-08-31 (any reason)||Trials stopped 2019-12-01 to 2020-08-31 (due to Covid-19)||Trials stopped 2017-12-01 to 2018-08-31 (comparator arm)|
|Status when stopped|
|Suspended||1833 (46%)||1512 (91%)||372 (18%)|
|Terminated||1430 (36%)||118 (7%)||1027 (49%)|
|Withdrawn||761 (19%)||40 (2%)||691 (33%)|
|Phase 1||521 (13%)||219 (13%)||280 (13%)|
|Phase 1-2||195 (5%)||48 (3%)||118 (6%)|
|Phase 2||611 (15%)||219 (13%)||370 (18%)|
|Phase 2-3||59 (1%)||12 (1%)||29 (1%)|
|Phase 3||269 (7%)||104 (6%)||163 (8%)|
|Phase 4||282 (7%)||99 (6%)||149 (7%)|
|N/A||1477 (37%)||703 (42%)||674 (32%)|
|(Left blank)||610 (15%)||266 (16%)||307 (15%)|
|Lead sponsor agency class|
|Industry||864 (21%)||312 (19%)||505 (24%)|
|NIH||78 (2%)||14 (1%)||87 (4%)|
|U.S. Federal||39 (1%)||19 (1%)||8 (0%)|
|Other||3043 (76%)||1325 (79%)||1490 (71%)|
|Randomized||2037 (51%)||914 (55%)||987 (59%)|
|Non-Randomized||394 (10%)||157 (9%)||186 (11%)|
|Not stated||1266 (31%)||509 (30%)||917 (55%)|
|Interventional||3414 (85%)||1404 (84%)||1783 (85%)|
|Observational||563 (14%)||243 (15%)||281 (13%)|
|Observational (patient registry)||47 (1%)||23 (1%)||26 (1%)|
|Indication area||Trials stopped by Covid-19||Trials stopped in Comparator||\(p\)-value|
|Cardiovascular||190 (11%)||274 (13%)||0.054|
|Neurologic||134 (8%)||149 (7.1%)||0.15|
|Oncology||389 (23%)||697 (33%)||6.9e-12|
|Pain||100 (6%)||120 (5.7%)||0.37|
I would like to gratefully acknowledge Peter Grabitz, Carole Federico and Delwen Franzen for insightful conversations that inspired and guided this work.
1. Drug Evaluation and Research, C. for. FDA Guidance on Conduct of Clinical Trials of Medical Products during COVID-19 Public Health Emergency. U.S. Food and Drug Administration Available at: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/fda-guidance-conduct-clinical-trials-medical-products-during-covid-19-public-health-emergency. (Accessed: 4th May 2020)
4. Mahase, E. Covid-19: Cancer Research urges mass testing to enable care to continue during pandemic. BMJ 369, (2020).
5. Mullard, A. COVID-19 brings drug development to a halt. Nature Reviews Drug Discovery (2020). doi:10.1038/d41573-020-00069-1
6. Adams, Ben. IQVIA: 80% of our trial sites are ’inaccessible’ due to pandemic, eyes Q4 for turnaround. FierceBiotech Available at: https://www.fiercebiotech.com/cro/iqvia-80-our-trial-sites-are-inaccessible-due-to-pandemic-eyes-q4-for-turnaround. (Accessed: 4th May 2020)
7. Carlisle, B. G. Clinical trials that were terminated, suspended or withdrawn due to Covid-19. (2020). doi:10.17605/OSF.IO/PRAFD
8. Carlisle, B. G. Covid-19 Trial Tracker. Codeberg.org Available at: https://codeberg.org/bgcarlisle/Covid-19TrialTracker. (Accessed: 4th May 2020)