PoisotLab / ms_network_prediction

Making better network predictions through ML - done right!
https://poisotlab.github.io/ms_network_prediction/
Other
1 stars 2 forks source link

changes before resubmission #143

Closed gottacatchenall closed 3 years ago

gottacatchenall commented 3 years ago

a running list of changes before resubmission: please feel free to add any changes from coauthors to this list


things to change:

things to rephrase:

gottacatchenall commented 3 years ago

re figure one changes:

  1. in the network illustration repo (PoisotLab/NetworkLearningIllustration@b4c26f3118dbd2e3136e1d7ea0046bdbad418840 ) i've added a file to do many partitions of test/training sets to plot a density of the distribution of roc-pr curves. the code is ugly but i think this could help address review one comment #128 and is generally a good practice to put on display in the paper. the resulting roc-pr curves are below.

The downsides are that we don't have a way to indicate a "best" threshold, as indicated by Youden's J on the current version of figure 1. Also it takes about an hour to run on a single core laptop, which isn't great for example code with a paper. It does look neat though

prbootstrap rocbootstrap

  1. also the network plot (the right side of current figure 1) has empirical interactions in purple and imputed interactions in grey, which may not be clearly visible and should be tweaked to ensure the difference is apparent. perhaps splitting the network plot into two panels (empirical interactions in one and imputed interactions in the other)?
TanyaS08 commented 3 years ago

Okay where I'm at with figures (mostly because I've actually lost track of all the discussions):

Figure 1 (Proof-of-Concept) -> do whatever you feel is best/clearer

Figure 4 (AUC) -> Drop it (although there is enough residual attachment on my side that if you guys have a strong desire to keep it I'm not going to have a breakdown)

TanyaS08 commented 3 years ago

Also @gottacatchenall feel free to work on #144 and make the edits you've suggested here (not all of them are immediately clear to me....)