reviewer-2-comment-1

[docsteveharris]

@Tim: how do you want to play this

The analogy to drug discovery is a poor fit for this essay. One key premise underlying drug discovery activities is that potentially-actionable targets have already been identified through prior research. For example, the decision to seek JAK2-interacting molecules emerged from the understanding that JAK2 regulates important cytokines in a number of disease states. The algorithmic equivalent of this kind of prior knowledge is not apparent to me. Without a clear understanding of (a) a well-articulated problem to be solved and (b) a reasonable mechanism/logic model in which an ML tool – provided it performs well – might affect stakeholder behavior, translational failure seems essentially guaranteed. Said differently, the bake-off between different ML approaches can really only be held once the model (“ligand”)’s “receptor” has been mapped/described, which I would argue happens infrequently at present. As I read the authors’ proposed pillars, I found myself reflecting on the need for this kind of formative work, informed by stakeholders, to ensure that the CDE can be used for properly-specified development, operations, and evaluation.  

I might suggest omitting or significantly abbreviating this section, which could open space to insert valuable information from the supplement. I also think a paragraph commenting on the relationship of the CDE to the organization’s “problem identification and specification” apparatus would be a very helpful “bookend” to the implementation and continuous evaluation portions of the essay.

My thoughts were to use reviewer 2 comment 2 (readability) and argue that the idea was to keep the main text narrative and then allow the interested reader to hit the ESM for details with the figures providing the bridge; possible compromise is to expand the captions for each so the flow of the text remains roughly the same?

And I've already added some context to the drug discovery analogy in a response to reviewer 3 comment 9.

9. It is not clear to me how the phased trials for testing algorithms relate to the Clinical Deployment Environment. How is the CDE used in each of the phases (if at all)? Which of the pillars are relevant in each of the phases? For instance, there is debate on which phase of model development we should investigate issues of AI safety and fairness (Wawira et al 2021). Please connect this section with the rest of the manuscript. Finally, continuous model evaluation (Pillar 5) could also be viewed as a Phase IV trial, which is common in drug regulation for monitoring adverse events after drug approval is granted.

Response: This section was is not meant to be correlated in a 1:1 manner with the five pillars. We hoped to use this section to convince the reader that algorithm deployment would require the similar effort and infrastructure to that for drug discovery, and that the 'get rich quick' approach which fills the popular and even the scientific press is shortsighted[@bunz2022a], and more thoughtful and careful work is often overlooked.[@sendak2020]

Response: We have added a final paragraph that we hope makes this clear, and similar word of caution in the conclusion.

Response: Section: Drug Discovery Parallels

The phases of drug development are not meant to be matched 1:1 to the pillars of the CDE described here, but the parallel is drawn to hightlight the effort necessary to see ML4H have an impact on the clinical and operatational decision making in the workplace.

Response: Conclusion Paragraph 4

They fundamentally are an argument for a professionalisation of ML4H, and a caution against the 'get-rich quick' headlines in the popular and scientific press.[@bunz2022a]

[docsteveharris]

via Tim

We thank the reviewer for this comment but feel that changing the manuscript as they suggest would not be the right thing to do. The point of the drug development pipeline analogy is to articulate the complexity of the intellectual work required for innovation. No-one expects a drug to be developed from scratch in a 3-5 years. However, in many cases people are disappointed when healthcare AI algorithms do not yield practice-changing results in the timeframe of a single project. The benefit of the Drug Trial Phases is that people are easily able to understand the risk level, the size of trial and the appropriate endpoints. This same thinking needs to be related to algorithms. Similar points have been made by Pepe (2005) and Ferrante di Ruffano with respect to biomarker discovery. To specifically address the JAK-2 example, this work would happen in the pre-clinical phase that we describe as algorithm discovery. The AI equivalent of JAK-2 (a targetable molecule) is a feature (a representation of data that has some utility in classification or prediction that also has a causal relationship with the phenomenon under study)