New Paper (Therapeutic): Hydroxychloroquine and azithromycin as a treatment of COVID-19: results of an open-label non-randomized clinical trial

[nafisajadavji]

Title: Hydroxychloroquine and azithromycin as a treatment of COVID-19: results of an open-label non-randomized clinical trial

Please paste a link to the paper or a citation here: : https://doi.org/10.1016/j.ijantimicag.2020.105949

Link: https://www.sciencedirect.com/science/article/pii/S0924857920300996

What is the paper's Manubot-style citation?

Citation: doi:10.1016/j.ijantimicag.2020.105949

Please list some keywords (3-10) that help identify the relevance of this paper to COVID-19

• hydroxychloroquine
• azithomycin
• clinical trial

Which areas of expertise are particularly relevant to the paper?

• [x] virology
• [ ] epidemiology
• [ ] biostatistics
• [ ] immunology
• [ ] pharmacology

Questions to answer about each paper:

Please provide 1-2 sentences introducing the study and its main findings

The study investigated the impact of hydroxychloroquine combined with azithromycin on viral loads in SARS-CoV-2-infected patients.

Study question(s) being investigated:

Does hydroxychloroquine and azithromycin effect SARS-CoV-2-infected patients?

How many/what drugs/combinations are being considered?

2 drugs (hydroxychloroquine and azithromycin) were being considered

What are the main hypotheses being tested?

In vitro and in vivo positive effects of Chloroquine have been reported. Hydroxychloroquine has been demonstrated to have an anti-SARS-CoV activity in vitro and is safer than chloroquine.

Study population:

Hospitalized patients with confirmed COVID-19 that were older than 12 years and PCR documented SARS-CoV-2 carriage in nasopharyngeal sample at admission whatever their clinical status.

What is the model system (e.g., human study, animal model, cell line study)?

Human

What is the sample size? If multiple groups are considered, give sample size for each group (including controls).

20 treated 16 controls

For human studies:

What countries/regions are considered?

Marseille, France

What is the age range, gender, other relevant characteristics?

12 years old Mean age of patients was 45.1 15 patients were male hydroxychloroquine patients were older than controls

What is the setting of the study (random sample of school children, inpatient, outpatient, etc)?

in patient (hospitalized)

What other specific inclusion-exclusion criteria are considered?

12 years old PCR documented SARS-CoV-2 carriage in nasopharyngeal sample at admission Patients were excluded if they had a known allergy to hydroxychloroquine or chloroquine or had another known contraindication to treatment with the study drug, including retinopathy, G6PD deficiency and QT prolongation. Breastfeeding and pregnant patients were excluded based on their declaration and pregnancy test results when required.

Treatment assignment:

oral hydroxychloroquine sulfate 200 mg, three times per day during ten days

Six patients received azithromycin (500mg on day1 followed by 250mg per day, the next four days) to prevent bacterial super-infection under daily electrocardiogram control

How are treatments assigned?

Interventional Patients had to consent to receive hydroxychloroquine, those that did not were included in the control group.

Is the study randomized?

No

A study can be interventional but not randomized (e.g., a phase I or II clinical trial is interventional but often not randomized).

non-randomized clinical trial

This includes possible treatment stratification (e.g., within litters for animal studies, within hospitals for human studies), possible confounding variables (e.g., having a large age range of individuals), possible risks of bias and how they are addressed (e.g., is there masking in a clinical trial? how are individuals chosen in an observational study?).

Outcome Assessment:

PCR positive nasopharyngeal samples

At day 3,4,5, and 6 post-inclusion treated patients had increased virological clearance between patients and controls.

At day 6 post-inclusion, 70% hydroxychloroquine treated patients had increased virological clearance, compared to 12.5% in the control group.

At day 6 post-inclusion, 100% of patients treated with hydroxychloroquine and azithromycin increased virological clearance.

Describe the outcome that is assessed and whether it is appropriate.

The outcome measurement was PCR positive nasopharyngeal samples. The authors said that clinical follow-up and occurrence of side-effects will be described in a further paper at the end of the trial.

For example: Is the outcome assessed by a clinician or is it self-reported? Is the outcome based on viral load or a functional measurement (e.g., respiratory function, discharge from hospital)? What method is used to measure the outcome? How long after a treatment is the outcome measured?

Are outcome measurements complete?

For example, are there individuals lost to follow up?

Are outcome measurements subject to various kinds of bias?

Not a randomized clinical trial.

Statistical Methods Assessment:

What methods are used for inference?

For example, logistic regression, nonparametric methods.

Are the methods appropriate for the study?

For example, are clustered data treated independently or are clusters adjusted for, such as different hospitals or litters?

Are adjustments made for possible confounders?

For example, adjustment for age, sex, or comorbidities.

Results Summary:

What is the estimated association?

For example, is it an estimated odds ratio, a median difference in detected cases, etc?

What measures of confidence or statistical significance are provided?

For example, confidence intervals, p-values, and/or Bayes factors.

Interpretation of results for study population:

Can we make a causal interpretation for the individuals in the study of drug -> outcome, such as "taking drug A improves likelihood of survival twofold over taking drug B."

For example, with a well-performed animal study or randomized trial it is often possible to infer causality. If is an observational study, does it match up with some of the Bradford Hill criteria? https://www.edwardtufte.com/tufte/hill https://en.wikipedia.org/wiki/Bradford_Hill_criteria

Are there identified side effects or interactions with other drugs?

For example, is the treatment known to cause liver damage or to not be prescribed for individuals with certain comorbities?

Are there specific subgroups with different findings?

For example, do individuals with a specific baseline seem to do particularly well? Be particularly cautious with respect to multiple testing here.

Extrapolation of conclusions to other groups of individuals not specifically included in the study:

If the study is an animal study, which animal and how relevant is that model?

Is the model system appropriate? Is there evidence from past use that it's highly-relevant to therapeutic design in this context?

If it is a human study, what characteristics of the study population may support/limit extrapolation?

• Can results extrapolate easily to other similar groups? (e.g., same country, similar age groups)
• What would happen if conditions are extended in terms of dose or duration?
• Can results be extrapolated to other populations or in very different settings? (e.g., different age group, primary care setting vs emergency department etc)

Summary of reliability

1-2 sentences on concluding remarks, including summary of strengths, weaknesses, limitations.

Progress

Check off the components as they are completed. If the component is not applicable, check the box as well.

• [x] 1-2 sentences introducing the study and its main findings
• [x] Describe How many/what drugs/combinations are being considered
• [x ] Describe the model system
• [x] What is the sample size?
• [x] What countries/regions are considered
• [ ] What is the age range, gender, other relevant characteristics
• [ x] Describe study setting
• [ x] Describe other specific inclusion-exclusion criteria
• [x ] Describe how treatments are assigned
• [ ] Describe randomization (or not) and other relavent details about the design
• [x] Describe the outcome that is assessed and whether it is appropriate.
• [x] Describe whether the outcome measurements are complete
• [ ] Are outcome measurements subject to various kinds of bias?
• [ ] Describe methods used for inference
• [ ] Describe whether the methods are appropriate for the study
• [ ] Are adjustments made for possible confounders?
• [ ] Describe the estimated association
• [ ] What measures of confidence or statistical significance are provided?
• [ ] Describe whether a causal interpretation can be made
• [ ] Are there identified side effects or interactions with other drugs?
• [ ] Are there specific subgroups with different findings?
• [ ] If the study is an animal study, which animal and how relevant is that model?
• [ ] If it is a human study, what characteristics of the study population may support/limit extrapolation?
• [ ] Summary of reliability

[SiminaB]

Good job @nafisajadavji for getting us started on looking at this paper that has been making the rounds!

There's a discussion here on this as well https://pubpeer.com/publications/3B1F9EAD4982C64445A60F5E83CCFE @cgreene @rando2: Is there a good way to credit other reviewers, e.g. on PubPeer?

In any case, we should make sure we note the loss to follow-up issue that is highlighted on PubPeer (bold face is added by me): "Six hydroxychloroquine-treated patients were lost in follow-up during the survey because of early cessation of treatment. Reasons are as follows: three patients were transferred to intensive care unit, including one transferred on day2 post-inclusion who was PCR-positive on day1, one transferred on day3 post-inclusion who was PCR-positive on days1-2 and one transferred on day4 post-inclusion who was PCR-positive on day1 and day3; one patient died on day3 post inclusion and was PCR-negative on day2; one patient decided to leave the hospital on day3 post-inclusion and was PCR-negative on days1-2; finally, one patient stopped the treatment on day3 post-inclusion because of nausea and was PCR-positive on days1-2-3."

[agitter]

Is there a good way to credit other reviewers, e.g. on PubPeer?

Maybe we could cite the PubPeer review? It has an author, date, and permalink. We would have to add a manual reference in Manubot. I tried a URL reference to the comment permalink but Manubot extracts metadata from the manuscript, not the review

$ manubot cite --render url:https://pubpeer.com/publications/B4044A446F35DF81789F6F20F8E0EE#2
1. Hydroxychloroquine and Azithromycin as a treatment of COVID-19: preliminary results of an open-label non-randomized clinical trial
Philippe Gautret, Jean Christophe Lagier, Philippe Parola, Van Thuan Hoang, Line Medded, Morgan Mailhe, Barbara Doudier, Johan Courjon, Valerie Giordanengo, Vera ESTEVES Vieira, … Didier Raoult
(2020-03-20) https://pubpeer.com/publications/B4044A446F35DF81789F6F20F8E0EE#2

[hufengling]

Note: I am a biostatistics MD-PhD student at Penn who is currently studying clinical trial methodology. I believe I have expertise in this area, but want to disclose I am still a student. Answers from above are duplicated and additional contributions/edits from me are included.

Title: Hydroxychloroquine and azithromycin as a treatment of COVID-19: results of an open-label non-randomized clinical trial

Please paste a link to the paper or a citation here:

Link: https://www.sciencedirect.com/science/article/pii/S0924857920300996

What is the paper's [Manubot-style citation]?

Citation: @doi:10.1016/j.ijantimicag.2020.105949

Please list some keywords (3-10) that help identify the relevance of this paper to COVID-19

• hydroxychloroquine
• azithomycin
• clinical trial
• therapeutics

Which areas of expertise are particularly relevant to the paper?

• [x] virology
• [ ] epidemiology
• [x] biostatistics
• [ ] immunology
• [ ] pharmacology

Questions to answer about each paper:

Please provide 1-2 sentences introducing the study and its main findings

The study investigated the impact of hydroxychloroquine combined with azithromycin on viral loads in SARS-CoV-2-infected patients. The study argues hydroxychloroquine with or without azithromycin is an effective therapeutic for reducing viral load and hospital stay.

Study question(s) being investigated:

How many/what drugs/combinations are being considered?

The trial arms were: hydroxychloroquine, hydroxychloroquine + azithromycin, and standard of care

What are the main hypotheses being tested?

Of the treatments hydroxychloroquine and standard of care, which is most effective for treating COVID-19 patients?

Is there a synergistic effect of azithromycin on treatment with hydroxychloroquine?

Study population:

What is the model system (e.g., human study, animal model, cell line study)?

Human

What is the sample size? If multiple groups are considered, give sample size for each group (including controls).

• 26 treated with hydroxychloroquine (6 dropped from analysis, 6 additionally treated with azithromycin)
• 16 treated with standard of care

For human studies:

What countries/regions are considered?

Various regions in France

What is the age range, gender, other relevant characteristics?

• Hydroxychloroquine arm: age (51.2, sd = 18.7), male gender (n = 9, 45%), asymptomatic (n = 2, 10%), time from symptoms to trial inclusion (4.1 days, sd = 2.6)
• Standard of care arm: age (37.3, sd = 24.0), male gender (n = 6, 37.5%), asymptomatic (n = 4, 25%), time from symptoms to trial inclusion (3.9 days, sd = 2.8)

What is the setting of the study (random sample of school children, inpatient, outpatient, etc)?

Hospitalized patients at Marseille hospital and various, unnamed hospitals.

What other specific inclusion-exclusion criteria are considered?

Inclusion criteria: age >12 years, PCR documented SARS-CoV-2 carriage in nasopharyngeal sample at admission whatever their clinical status.

Patients were excluded if they had a known allergy to hydroxychloroquine or chloroquine or had another known contraindication to treatment with the study drug, including retinopathy, G6PD deficiency and QT prolongation. Breastfeeding and pregnant patients were excluded based on their declaration and pregnancy test results when required.

Treatment assignment:

How are treatments assigned?

Interventional study

Is the study randomized?

Study is non-randomized, non-placebo-controlled, non-blinded. (Non-placebo-controlled here to mean the control arm is standard of care, not standard of care + placebo pill.)

Provide other relevant details about the design.

Treatment arm is entirely confounded by hospital. All treatment arm patients were recruited from a single center while control arm patients could come from other centers. Patients who refused hydroxychloroquine were placed in control arm.

Six treatment arm patients were dropped due to discontinuation of drug because of ICU placement, death, or leaving the hospital. No Intention-To-Treat analysis was provided.

Bias in treatment could be present due to lack of blinding. Decisions made for treatment arm patients could differ from those made for control arm patients.

Outcome Assessment:

Describe the outcome that is assessed and whether it is appropriate.

Outcome is defined as negative nasopharyngeal PCR by day post-study inclusion. Outcome is assessed by a clinician. Outcome seems appropriate; however, there are discrepancies in results, detailed below.

Are outcome measurements complete?

Six individuals are lost to follow up and not included in analysis. All six patients are from treatment arm - five of the six patients had negative effects.

Data on negative nasopharyngeal PCR is incomplete and imputed for some control patients. There are discrepancies in this reporting: on day 4 post-inclusion, four control patients are virus-free but on day 5, only three are, and on day 6, only 2 are. The data suggests two patients had false negative results.

Are outcome measurements subject to various kinds of bias?

Bias in treatment could be present due to lack of blinding. Decisions made for treatment arm patients could differ from those made for control arm patients.

Statistical Methods Assessment:

What methods are used for inference?

Fisher's exact test and chi-squared test are used for binomial outcomes. Use is not specified, but assumption of "large enough n" for each outcome is likely violated. Multiple comparisons (due to testing sequential data) are not corrected for.

Are the methods appropriate for the study?

Treatment arm is entirely confounded by hospital. This issue cannot be accounted for by analysis method. No Intention-To-Treat analysis was conducted. This is inappropriate.

Are adjustments made for possible confounders?

No adjustments are made, despite imbalance in known confounder of age.

Results Summary:

What is the estimated association?

Estimated association is difference in binary outcome.

What measures of confidence or statistical significance are provided?

Day 3: hydroxychloroquine (10/20 with negative titer), control (1/16 with negative titer). P = 0.005 Day 4: hydroxychloroquine (12/20 with negative titer), control (4/16 with negative titer). P = 0.04 Day 5: hydroxychloroquine (13/20 with negative titer), control (3/16 with negative titer). P = 0.006 Day 6: hydroxychloroquine (14/20 with negative titer), control (2/16 with negative titer). P = 0.001

Note: If Intention-to-Treat analysis is conducted (which is a more appropriate analysis), p-values change to 0.030, 0.21, 0.056, and 0.010, respectively. If p-values are corrected for using Bonferroni correction (which is also appropriate), only the Day 6 results are significant at alpha = 0.05.

Interpretation of results for study population:

Can we make a causal interpretation for the individuals in the study of drug -> outcome, such as "taking drug A improves likelihood of survival twofold over taking drug B."

No causal interpretation can be made.

Are there identified side effects or interactions with other drugs?

Hydroxychloroquine side effects in non-COVID-19 patients is well known. Side effects or interactions in COVID-19 patients are not well characterized.

Are there specific subgroups with different findings?

No subgroup testing was done.

Extrapolation of conclusions to other groups of individuals not specifically included in the study:

If the study is an animal study, which animal and how relevant is that model?

N/A

If it is a human study, what characteristics of the study population may support/limit extrapolation?

• Poor study design, confounding by hospital, lack of randomization, lack of blinding, and lack of placebo limit extrapolation.
• Further study by RCT is necessary.

Summary of reliability

Study provides promising exploratory data on use of hydroxychloroquine to treat COVID-19. Results are not further interpretable due to study design issues. An RCT is necessary to study whether this hydroxychloroquine is useful in this setting.

Progress

Check off the components as they are completed. If the component is not applicable, check the box as well.

• [x] 1-2 sentences introducing the study and its main findings
• [x] Describe How many/what drugs/combinations are being considered
• [x] Describe the model system
• [x] What is the sample size?
• [x] What countries/regions are considered
• [x] What is the age range, gender, other relevant characteristics
• [x] Describe study setting
• [x] Describe other specific inclusion-exclusion criteria
• [x] Describe how treatments are assigned
• [x] Describe randomization (or not) and other relavent details about the design
• [x] Describe the outcome that is assessed and whether it is appropriate.
• [x] Describe whether the outcome measurements are complete
• [x] Are outcome measurements subject to various kinds of bias?
• [x] Describe methods used for inference
• [x] Describe whether the methods are appropriate for the study
• [x] Are adjustments made for possible confounders?
• [x] Describe the estimated association
• [x] What measures of confidence or statistical significance are provided?
• [x] Describe whether a causal interpretation can be made
• [x] Are there identified side effects or interactions with other drugs?
• [x] Are there specific subgroups with different findings?
• [x] If the study is an animal study, which animal and how relevant is that model?
• [x] If it is a human study, what characteristics of the study population may support/limit extrapolation?
• [x] Summary of reliability

[nafisajadavji]

@hufengling thanks for contributing! You made some good additions!

[nafisajadavji]

I did not mean to close it. Still navigating GitHub :S

[SiminaB]

Thank you both!

@hufengling: Great job filling in so many details! Can you include numerical results under Results Summary? For example, you say that only the p-values are given, but can you give the actual p-value? (Same thing here https://github.com/greenelab/covid19-review/issues/123) I should maybe have been a bit clearer in the template... I meant to say that you should include what the measures are + their estimates. Eg if the estimate is of an odds ratio and the 95% CI is also include, state that both the odds ratio and the 95% CI are estimated and give their estimates as well. If the p-value is also included, also give that (even though it can be obtained just from inverting the CI).

Also: I don't think that a study is necessarily problematic just because it's not placebo-controlled or that this is even a weakness. In general it's often not possible or it may even be unethical to run a placebo-controlled study, so comparing to standard-of-care is fine (indeed, this is what you want to do for a new treatment, you want to compare it to the care that patients with that illness will generally get).

Again, thank you so much for your great contributions!

[hufengling]

Sorry, I misunderstood! I've updated my comment (and added in some comments on how results would change given alternate, more appropriate clinical trial analysis.)

Regarding your point about placebo, I guess I should have been more clear. I meant that trial arms should be: 1) hydroxychloroquine + supportive care and 2) placebo + supportive care. The latter is standard of care, but giving control patients an additional placebo pill would make the argument stronger. I've clarified this in my comment above.

Patients can do better if they think they are receiving the "new and better" drug. Physicians may also treat patients differently if they know they are on hydroxychloroquine or not - perhaps they pay closer attention to hydroxychloroquine patients.

[SiminaB]

Thanks for the update and got what you're saying now about placebo-controlled trials!

[agitter]

Retraction Watch reports Hydroxychloroquine-COVID-19 study did not meet publishing society’s “expected standard” based on a statement by the International Society of Antimicrobial Chemotherapy.