hufengling
commented
4 years ago Note: I am a biostatistics MD-PhD student at Penn who is currently studying clinical trial methodology. I believe I have expertise in this area, but want to disclose I am still a student.
Title: Efficacy of hydroxychloroquine in patients with COVID-19: results of a randomized clinical trial
Please paste a link to the paper or a citation here:
Link: https://www.medrxiv.org/content/10.1101/2020.03.22.20040758v1
What is the paper's [Manubot-style citation]
Citation: @doi:10.1101/2020.03.22.20040758
Please list some keywords (3-10) that help identify the relevance of this paper to COVID-19
- hydroxychloroquine
- randomized clinical trial
- SARS-nCov-2
Which areas of expertise are particularly relevant to the paper?
- [ ] virology
- [ ] epidemiology
- [x] biostatistics
- [ ] immunology
- [ ] pharmacology
Questions to answer about each paper:
Please provide 1-2 sentences introducing the study and its main findings
A small randomized, non-placebo-controlled, non-blinded clinical trial was conducted comparing hydroxychloroquine vs standard of care. Hydroxychloroquine was found to be effective at shortening fever duration and time to cough relief.
Study question(s) being investigated:
How many/what drugs/combinations are being considered?
Hydroxychloroquine is compared to standard of care
What are the main hypotheses being tested?
Is hydroxychloroquine effective at shortening fever duration or time to cough relief?
Study population:
What is the model system (e.g., human study, animal model, cell line study)?
Human
What is the sample size? If multiple groups are considered, give sample size for each group (including controls).
- 31 treated with hydroxychloroquine
- 31 treated with standard of care
For human studies:
What countries/regions are considered?
Wuhan, China
What is the age range, gender, other relevant characteristics?
Average age was 44.7 (sd = 15.3). 46.8% of patients were male.
What is the setting of the study (random sample of school children, inpatient, outpatient, etc)?
Inpatient setting
What other specific inclusion-exclusion criteria are considered?
Inclusion criteria: 1. Age ≥ 18 years; 2. Laboratory (RT-PCR) positive of SARS-CoV-2; 3. Chest CT with pneumonia; 4. SaO2/SPO2 ratio > 93% or PaO2/FIO2 ratio > 300 mmHg under the condition in the hospital room (mild illness); 5. Willing to receive a random assignment. Exclusion criteria: 1. Severe and critical illness patients or participating in another trial; 2. Retinopathy and other retinal diseases; 3. Conduction block and other arrhythmias; 4. Severe liver disease; 5. Pregnant or breastfeeding; 6. Severe renal failure; 7. Possibility of being transferred to another hospital within 72 h; 8. Received any trial treatment for COVID-19 within 30 days.
Treatment assignment:
How are treatments assigned?
Interventional study
Is the study randomized?
Study is randomized, stratified by hospital.
Provide other relevant details about the design.
Trial is non-blinded, non-placebo controlled. (Trial claims researchers and patients were not aware of treatment assignments, but without placebo, this is impossible.) Sample size is small. Trial is pre-registered in Chinese Clinical Trial Registry, but trial design and outcomes assessed differ drastically.
Outcome Assessment:
Describe the outcome that is assessed and whether it is appropriate.
Outcomes are fever duration and time to cough relief (maintained for 72hr). Fever duration is assessed by physician, but cough relief cannot be. Outcomes do not seem appropriate. Secondary outcome was improvement of pneumonia on chest CT.
Are outcome measurements complete?
No mention of whether any individuals were lost to follow-up was made.
Are outcome measurements subject to various kinds of bias?
Trial is non-blinded, meaning physician management of trial participants may differ based on trial arm, thus introducing bias.
Statistical Methods Assessment:
What methods are used for inference?
T-test was used for comparison of primary outcomes (fever duration and cough relief). Chi-sq test was used for comparison of pneumonia improvement.
Are the methods appropriate for the study?
Tests are appropriate for the data, but analysis may not be appropriate for the study. Study design and outcomes differ drastically from pre-registered plan (found here: http://www.chictr.org.cn/showprojen.aspx?proj=48880). This is suggestive of p-hacking. Poor choice of outcomes may support this hypothesis (perhaps more "standard" outcomes showed no statistically significant difference.)
Are adjustments made for possible confounders?
Adjustments are not made for possible confounders, such as age or existence of fever/cough prior to study start. Adjustment for confounders is not necessary for generalizability in the setting of randomization; however, it can increase the study's power.
Results Summary:
What is the estimated association?
Difference in fever duration and cough relief.
What measures of confidence or statistical significance are provided?
Duration of fever after trial start: control (3.2 days, sd = 1.3) and hydroxychloroquine (2.2 days, sd = 0.4). P = 0.008. Note that number of patients with fever was unbalanced at start: control = 17/31, hydroxychloroquine = 22/31 Cough remission time after trial start: control (3.1 days, sd = 1.5) and hydroxychloroquine (2.0 days, sd = 0.2). P = 0.0016. Note that number of patients with cough was unbalanced at start: control = 15/31, hydroxychloroquine = 22/31 Anecdotal: 4/31 patients in control arm progressed to severe illness, 0/31 patients in hydroxychloroquine arm progressed to severe illness.
Pneumonia by CT scan: In control, 9/31 exacerbated, 5/31 unchanged, 17/31 improved. In hydroxychloroquine arm, 2/31 exacerbated, 4/31 unchanged, 25/31 improved. P-value = 0.0476. Analyzed by chi-square, but assumption of "large enough n" is violated.
Interpretation of results for study population:
Can we make a causal interpretation for the individuals in the study of drug -> outcome, such as "taking drug A improves likelihood of survival twofold over taking drug B."
A causal interpretation of the data is somewhat possible, if we assume rigorous statistical inference without inflation of false positive rate (due to p-hacking). However, given drastic differences between publication and pre-registered trial, a possibility of p-hacking (and therefore uninterpretability) could be called into question. Additionally, lack of placebo and lack of blinding could confound results. Finally, inappropriate choice of outcome and uncertainty on how the analysis was conducted raise further issues.
Are there identified side effects or interactions with other drugs?
Safety data on hydroxychloroquine in non-COVID-19 patients is well known.
Are there specific subgroups with different findings?
No subgroup findings were analyzed/reported.
Extrapolation of conclusions to other groups of individuals not specifically included in the study:
If the study is an animal study, which animal and how relevant is that model?
N/A
If it is a human study, what characteristics of the study population may support/limit extrapolation?
- Inclusion/exclusion criteria seem reasonable, which supports extrapolation. Randomization also supports extrapolation.
- Poor choice of outcomes and disparity between publication outcome and pre-registered outcomes limit extrapolation.
Summary of reliability
Strengths of this study include randomization and multi-hospital recruitment. Weaknesses of this study include non-placebo-control, non-blinding, small sample size, poor outcome choice, and disparity between pre-registration and publication. Study should be considered as preliminary and findings should be further studied in a larger, more rigorous clinical trial with better outcome selection.
Progress
Check off the components as they are completed. If the component is not applicable, check the box as well.
- [x] 1-2 sentences introducing the study and its main findings
- [x] Describe How many/what drugs/combinations are being considered
- [x] Describe the model system
- [x] What is the sample size?
- [x] What countries/regions are considered
- [x] What is the age range, gender, other relevant characteristics
- [x] Describe study setting
- [x] Describe other specific inclusion-exclusion criteria
- [x] Describe how treatments are assigned
- [x] Describe randomization (or not) and other relavent details about the design
- [x] Describe the outcome that is assessed and whether it is appropriate.
- [x] Describe whether the outcome measurements are complete
- [x] Are outcome measurements subject to various kinds of bias?
- [x] Describe methods used for inference
- [x] Describe whether the methods are appropriate for the study
- [x] Are adjustments made for possible confounders?
- [x] Describe the estimated association
- [x] What measures of confidence or statistical significance are provided?
- [x] Describe whether a causal interpretation can be made
- [x] Are there identified side effects or interactions with other drugs?
- [x] Are there specific subgroups with different findings?
- [x] If the study is an animal study, which animal and how relevant is that model?
- [x] If it is a human study, what characteristics of the study population may support/limit extrapolation?
- [x] Summary of reliability
Title: Please edit the title to add the name of the paper after the colon
Please paste a link to the paper or a citation here:
Link: https://www.medrxiv.org/content/10.1101/2020.03.22.20040758v1
What is the paper's Manubot-style citation?
Citation: doi:10.1101/2020.03.22.20040758
Please list some keywords (3-10) that help identify the relevance of this paper to COVID-19
Which areas of expertise are particularly relevant to the paper?
Questions to answer about each paper:
Please provide 1-2 sentences introducing the study and its main findings
The main goal of this study is to determine the efficacy of hydroxychloroquine as a treatment for patients with COVID-19. The study partially confirms the potential for hydroxychloroquine as a treatment.
Study question(s) being investigated:
Does hydroxychloroquine effectively treat patients with COVID-19? Does hydroxychloroquine have less adverse effects than chloroquine?
How many/what drugs/combinations are being considered?
1 drug: hydroxychloroquine
What are the main hypotheses being tested?
Studies of chloroquine in vitro have indicated a positive effect on COVID-19. Hydroxychloroquine is thought to be safer than chloroquine
Study population:
patients diagnosed with COVID-19 and admitted to Renmin Hospital of Wuhan University, chest CT showing pneumonia
What is the model system (e.g., human study, animal model, cell line study)?
human
What is the sample size? If multiple groups are considered, give sample size for each group (including controls).
For human studies:
What countries/regions are considered?
Wuhan, China
What is the age range, gender, other relevant characteristics?
29-62 years old, 46.8% male and 53.2% female
What is the setting of the study (random sample of school children, inpatient, outpatient, etc)?
random sample of patients admitted to the hospital diagnosed with COVID-19
What other specific inclusion-exclusion criteria are considered?
For example, do the investigators exclude patients with diagnosed neoplasms or patients over/under a certain age?
Treatment assignment:
How are treatments assigned?
For example, is it an interventional or an observational study?
Is the study randomized?
A study can be interventional but not randomized (e.g., a phase I or II clinical trial is interventional but often not randomized).
Provide other relevant details about the design.
This includes possible treatment stratification (e.g., within litters for animal studies, within hospitals for human studies), possible confounding variables (e.g., having a large age range of individuals), possible risks of bias and how they are addressed (e.g., is there masking in a clinical trial? how are individuals chosen in an observational study?).
Outcome Assessment:
Describe the outcome that is assessed and whether it is appropriate.
For example: Is the outcome assessed by a clinician or is it self-reported? Is the outcome based on viral load or a functional measurement (e.g., respiratory function, discharge from hospital)? What method is used to measure the outcome? How long after a treatment is the outcome measured?
Are outcome measurements complete?
For example, are there individuals lost to follow up?
Are outcome measurements subject to various kinds of bias?
For example, a lack of masking in randomized clinical trials.
Statistical Methods Assessment:
What methods are used for inference?
For example, logistic regression, nonparametric methods.
Are the methods appropriate for the study?
For example, are clustered data treated independently or are clusters adjusted for, such as different hospitals or litters?
Are adjustments made for possible confounders?
For example, adjustment for age, sex, or comorbidities.
Results Summary:
What is the estimated association?
For example, is it an estimated odds ratio, a median difference in detected cases, etc?
What measures of confidence or statistical significance are provided?
For example, confidence intervals, p-values, and/or Bayes factors.
Interpretation of results for study population:
Can we make a causal interpretation for the individuals in the study of drug -> outcome, such as "taking drug A improves likelihood of survival twofold over taking drug B."
For example, with a well-performed animal study or randomized trial it is often possible to infer causality. If is an observational study, does it match up with some of the Bradford Hill criteria? https://www.edwardtufte.com/tufte/hill https://en.wikipedia.org/wiki/Bradford_Hill_criteria
Are there identified side effects or interactions with other drugs?
For example, is the treatment known to cause liver damage or to not be prescribed for individuals with certain comorbities?
Are there specific subgroups with different findings?
For example, do individuals with a specific baseline seem to do particularly well? Be particularly cautious with respect to multiple testing here.
Extrapolation of conclusions to other groups of individuals not specifically included in the study:
If the study is an animal study, which animal and how relevant is that model?
Is the model system appropriate? Is there evidence from past use that it's highly-relevant to therapeutic design in this context?
If it is a human study, what characteristics of the study population may support/limit extrapolation?
Summary of reliability
1-2 sentences on concluding remarks, including summary of strengths, weaknesses, limitations.
Progress
Check off the components as they are completed. If the component is not applicable, check the box as well.