hufengling
commented
4 years ago Note: I wanted to add my take from a clinical trial/statistics point of view. My additions will be bolded. I am a biostatistics MD-PhD student at Penn who is currently studying clinical trial methodology. I believe I have expertise in this area, but want to disclose I am still a student.
Title: A Trial of Lopinavir–Ritonavir in Adults Hospitalized with Severe Covid-19
Please paste a link to the paper or a citation here:
Link: https://www.nejm.org/doi/full/10.1056/NEJMoa2001282
What is the paper's DOI?
DOI: 10.1056/NEJMoa2001282
Please list some keywords (3-10) that help identify the relevance of this paper to COVID-19
- drug-repurposing
- antiviral
- randomized-controlled-trial
Which areas of expertise are particularly relevant to the paper?
- [x] virology
- [ ] epidemiology
- [x] biostatistics
- [ ] immunology
- [x] pharmacology
Questions to answer about each paper:
Please provide 1-2 sentences introducing the study and its main findings
The study is trial of a combination therapy for COVID-19. The intervention did not succeed. This is a small, severely underpowered clinical trial, so we cannot exclude a conclusion that Lopinavir-Ritonavir is useful in treating COVID-19.
Study question(s) being investigated:
Efficacy of a combination of antiviral drugs in COVID-19.
How many/what drugs/combinations are being considered?
A combination of Ritonavir-Lopinavir.
What are the main hypotheses being tested?
The efficacy of Ritonavir-Lopinavir in COVID-19 over placebo. The trial was not placebo-controlled. It compared Lopinavir-Ritonavir + standard of care to just standard of care (not standard of care + placebo). This means the study is non-blinded - physician bias or placebo effect likely influence results.
Study population:
What is the model system (e.g., human study, animal model, cell line study)?
Human study.
What is the sample size? If multiple groups are considered, give sample size for each group (including controls).
199 patients in total.
- number treated with Ritonavir-Lopinavir: 99
- number in standard care: 100
For human studies:
What countries/regions are considered?
Wuhan, China.
What is the age range, gender, other relevant characteristics?
median age: 58 years
What is the setting of the study (random sample of school children, inpatient, outpatient, etc)?
Patients admitted for COVID-19 symptoms.
What other specific inclusion-exclusion criteria are considered?
- Inclusion:
- Male and nonpregnant female patients 18 years of age or older,
- had a diagnostic specimen that was positive on RT-PCR,
- had pneumonia confirmed by chest imaging,
- had an oxygen saturation (Sao2) of 94% or less while they were breathing ambient air or a ratio of the partial pressure of oxygen (Pao2) to the fraction of inspired oxygen (Fio2) (Pao2:Fio2) at or below 300 mg Hg.
- Exclusion:
- physician decision that involvement in the trial was not in the patient’s best interest,
- presence of any condition that would not allow the protocol to be followed safely,
- known allergy or hypersensitivity to lopinavir–ritonavir,
- known severe liver disease (e.g., cirrhosis, with an alanine aminotransferase level >5× the upper limit of the normal range or an aspartate aminotransferase level >5× the upper limit of the normal range),
- use of medications that are contraindicated with lopinavir–ritonavir and that could not be replaced or stopped during the trial period,
- pregnancy or breast-feeding,
- or known HIV infection, because of concerns about the development of resistance to lopinavir–ritonavir if used without combining with other antiretrovirals.
- Patients who were unable to swallow received lopinavir–ritonavir through a nasogastric tube.
Treatment assignment:
How are treatments assigned?
For example, is it an interventional or an observational study? Interventional.
Is the study randomized?
yes.
Provide other relevant details about the design.
Of the 357 patients, 158 were excluded and 199 were included. Among the 99 assigned to treatment group, three died after admission and two could not take the treatment. The three patients who died before receiving treatment suggest that treatment was not given immediately after enrollment/randomization. This clouds results, because for every day a treatment arm patient does not receive Lopinavir-Ritonavir, they are effectively a "control patient." Two could not continue treatment due to adverse GI side effects. They should (and are!) be included in results analysis (Intention-To-Treat analysis).
Outcome Assessment:
Describe the outcome that is assessed and whether it is appropriate.
The outcome is determined by a clinician. Outcome is time to improvement on a 7-point scale (in use by WHO Solidarity trial). Objective observations are used to assign points. Outcome is justifiable, but time to worsening on the 7-point may be more appropriate.
- The outcome was mainly the viral load, respiratory assistance (supplemental oxygen), and clinical improvement score, days of invasive intubation
- The outcome is measured after 7, 14, and 28 days
Are outcome measurements complete?
Five patients were dropped during the study. Five treatment arm patients did not continue Lopinavir-Ritonavir, but were still included in the analysis. This is standard practice and appropriate.
Are outcome measurements subject to various kinds of bias?
I think that accounting for a placebo arm (standard-care) corrects for several bias. No placebo was given, but randomization was done. Outcome is subject to placebo effect and physician bias due to non-blinding (because of lack of placebo.) For example, physicians may have paid more attention to treatment arm patients.
Statistical Methods Assessment:
What methods are used for inference?
Kaplan-Meier plot, hazard ratio.
Are the methods appropriate for the study?
Yes.
Are adjustments made for possible confounders?
No adjustment was considered because no important between-group differences in demographic characteristics, baseline laboratory test results, distribution of ordinal scale scores, or NEWS2 scores at enrollment Even if there are no significant between-group differences, adjusting for potential confounders (decided ad-hoc) is best practice - it has no downsides (even if you are wrong about the existence of confounding!) but increases power. Adjustment should have been done, but was not.
Results Summary:
What is the estimated association?
odds ratio. Primary outcome is hazard ratio.
What measures of confidence or statistical significance are provided?
Treatment with lopinavir–ritonavir was not associated with a difference from standard care in the time to clinical improvement (hazard ratio for clinical improvement, 1.24; 95% confidence interval [CI], 0.90 to 1.72). Mortality at 28 days was similar in the lopinavir–ritonavir group and the standard-care group (19.2% vs. 25.0%; difference, −5.8 percentage points; 95% CI, −17.3 to 5.7).
Interpretation of results for study population:
Can we make a causal interpretation for the individuals in the study of drug -> outcome, such as "taking drug A improves likelihood of survival twofold over taking drug B."
Yes. Yes, we can make inferences, but they should be cautious due to sample size and trial limitations. The study is underpowered (so is expected to find no significant difference) and non-placebo controlled.
Are there identified side effects or interactions with other drugs?
Patients with such characteristics were excluded from the study.
Are there specific subgroups with different findings?
No.
Extrapolation of conclusions to other groups of individuals not specifically included in the study:
If the study is an animal study, which animal and how relevant is that model?
No.
If it is a human study, what characteristics of the study population may support/limit extrapolation?
-I think it is hard to tell if the study extrapolates to other countries with n=199, with all the patients from Wuhan, China. Small sample size and lack of placebo also limit extrapolation.
Summary of reliability
I think it is a remarkable study, with sound design and flawless execution and design. It helps narrow down the search for therapy. Study is a solid first investigation into Lopinavir-Ritonavir, though it suffers from a number of issues. One additional reliability issue is that the pre-registration states a sample size of n = 160, but the sample size is increased to n = 199 without solid, stated rationale. It is also concluded because remdesivir became available, not because of scientific principle. This is not standard practice. This study may provide support that, if effective, Lopinavir-Ritonavir may not be the "breakthrough" drug we are looking for, but could still be beneficial. Lopinavir-Ritonavir should still be included in future drug trials for further study with larger sample size and more rigorous design.
Progress
Check off the components as they are completed. If the component is not applicable, check the box as well.
- [x] 1-2 sentences introducing the study and its main findings
- [x] Describe How many/what drugs/combinations are being considered
- [x] Describe the model system
- [x] What is the sample size?
- [x] What countries/regions are considered
- [x] What is the age range, gender, other relevant characteristics
- [x] Describe study setting
- [x] Describe other specific inclusion-exclusion criteria
- [x] Describe how treatments are assigned
- [x] Describe randomization (or not) and other relavent details about the design
- [x] Describe the outcome that is assessed and whether it is appropriate.
- [x] Describe whether the outcome measurements are complete
- [x] Are outcome measurements subject to various kinds of bias?
- [x] Describe methods used for inference
- [x] Describe whether the methods are appropriate for the study
- [x] Are adjustments made for possible confounders?
- [x] Describe the estimated association
- [x] What measures of confidence or statistical significance are provided?
- [x] Describe whether a causal interpretation can be made
- [x] Are there identified side effects or interactions with other drugs?
- [x] Are there specific subgroups with different findings?
- [x] If the study is an animal study, which animal and how relevant is that model?
- [x] If it is a human study, what characteristics of the study population may support/limit extrapolation?
- [x] Summary of reliability
Title: A Trial of Lopinavir–Ritonavir in Adults Hospitalized with Severe Covid-19
Please paste a link to the paper or a citation here:
Link: https://www.nejm.org/doi/full/10.1056/NEJMoa2001282
What is the paper's DOI?
Citation: doi:10.1056/NEJMoa2001282
Please list some keywords (3-10) that help identify the relevance of this paper to COVID-19
Which areas of expertise are particularly relevant to the paper?
Questions to answer about each paper:
Please provide 1-2 sentences introducing the study and its main findings
The study is trial of a combination therapy for COVID-19. The intervention did not succeed.
Study question(s) being investigated:
Efficacy of a combination of antiviral drugs in COVID-19.
How many/what drugs/combinations are being considered?
A combination of Ritonavir-Lopinavir.
What are the main hypotheses being tested?
The efficacy of Ritonavir-Lopinavir in COVID-19 over placebo.
Study population:
What is the model system (e.g., human study, animal model, cell line study)?
Human study.
What is the sample size? If multiple groups are considered, give sample size for each group (including controls).
199 patients in total.
For human studies:
What countries/regions are considered?
Wuhan, China.
What is the age range, gender, other relevant characteristics?
median age: 58 years
What is the setting of the study (random sample of school children, inpatient, outpatient, etc)?
Patients admitted for COVID-19 symptoms.
What other specific inclusion-exclusion criteria are considered?
Treatment assignment:
How are treatments assigned?
For example, is it an interventional or an observational study? Interventional.
Is the study randomized?
yes.
Provide other relevant details about the design.
Of the 357 patients, 158 were excluded and 199 were included. Among the 99 assigned to treatment group, three died after admission and two could not take the treatment.
Outcome Assessment:
Describe the outcome that is assessed and whether it is appropriate.
The outcome is determined by a clinician.
Are outcome measurements complete?
Five patients were dropped during the study.
Are outcome measurements subject to various kinds of bias?
I think that accounting for a placebo arm (standard-care) corrects for several bias.
Statistical Methods Assessment:
What methods are used for inference?
Kaplan-Meier plot, hazard ratio.
Are the methods appropriate for the study?
Yes.
Are adjustments made for possible confounders?
No adjustment was considered because no important between-group differences in demographic characteristics, baseline laboratory test results, distribution of ordinal scale scores, or NEWS2 scores at enrollment
Results Summary:
What is the estimated association?
odds ratio.
What measures of confidence or statistical significance are provided?
Treatment with lopinavir–ritonavir was not associated with a difference from standard care in the time to clinical improvement (hazard ratio for clinical improvement, 1.24; 95% confidence interval [CI], 0.90 to 1.72). Mortality at 28 days was similar in the lopinavir–ritonavir group and the standard-care group (19.2% vs. 25.0%; difference, −5.8 percentage points; 95% CI, −17.3 to 5.7).
Interpretation of results for study population:
Can we make a causal interpretation for the individuals in the study of drug -> outcome, such as "taking drug A improves likelihood of survival twofold over taking drug B."
Yes.
Are there identified side effects or interactions with other drugs?
Patients with such characteristics were excluded from the study.
Are there specific subgroups with different findings?
No.
Extrapolation of conclusions to other groups of individuals not specifically included in the study:
If the study is an animal study, which animal and how relevant is that model?
No.
If it is a human study, what characteristics of the study population may support/limit extrapolation?
-I think it is hard to tell if the study extrapolates to other countries with n=199, with all the patients from Wuhan, China.
Summary of reliability
I think it is a remarkable study, with sound design and flawless execution and design. It helps narrow down the search for therapy.
Progress
Check off the components as they are completed. If the component is not applicable, check the box as well.