Revisions: Manage model complexity while preserving value of ML

jaybee84 / ml-in-rd

Manuscript for perspective on machine learning in rare disease

Other

2 stars 1 forks source link

Revisions: Manage model complexity while preserving value of ML #175

Closed jaybee84 closed 2 years ago

jaybee84 commented 3 years ago

Please read this section from the point of view of an audience less familiar with ML. Jot down any questions that occur to you while reading this section.

jaclyn-taroni commented 3 years ago

For the audience, what does "performant" mean?
Similar to my comment on #173, explain why bad things (like overfitting!) are bad in practice.
Talking about "helps estimate population values" seems like either too much or too little information
I'm not sure we need to go into the specifics of LASSO, ridge, and elastic net and in tandem, we might shorten the list of references - i.e., should we concentrate on one specific regularization approach so we have more "real estate" to talk about why you might want to use it?

dvenprasad commented 3 years ago

what is class imbalance?

A pattern I noticed (not only limited to this section), is that we talk about the techniques and then provide examples. I wonder if we could talk about the example first and then the techniques presenting it as Context -> Biological problem -> technique and what it solves and then get into the weeds like the different variations of the technique, so we're transitioning people from something they are familiar with to something unfamiliar but it's okay cause the new jargon is more palatable cause it's contextualized better?

jaybee84 commented 2 years ago

Adding the following just to revisit what we had mentioned to do:

We will revise the sections to include a synthesis of the machine learning concepts before transitioning to a few examples of application of these concepts in rare disease for the benefit of the readers.
We will also refine the headers to provide more clarity to the readers.

jaybee84 commented 2 years ago

Other reminders:

note the limitations of resampling in case of non-deterministic algorithms.
clarify confidence interval estimation through resampling both with and without replacement.
clarify that special attention should be directed towards adjusting for rare classes.
mention different approaches that may be taken to address the presence of class imbalance and label-noise.