search

hubverse-org / hubEnsemblesManuscript

https://htmlpreview.github.io/?https://github.com/Infectious-Disease-Modeling-Hubs/hubEnsemblesManuscript/blob/master/analysis/paper/hubEnsembles_manuscript.html
Other
1 stars 2 forks source link

manuscript - case study #1

Closed harryhoch closed 6 months ago

harryhoch commented 9 months ago

A few notes on the case study:

  1. the text mentions as_model_out_tbl(), but this routine is not used in the snippets.
  2. It might help to tell folks where covidHubUtils can be found.
  3. In the table consider bolding the best results.
  4. I assume the vignette will point to all of the code needed to regenerate the graphics?
  5. "Grouping the scores by horizon on its own and horizon in conjunction with season reveal similar patterns in model rankings, with the quantile median achieving the lowest WIS except at the 1 week ahead horizon. " are these results shown?
  6. "Notably, for WIS the linear pools outperform the median ensemble at the one week ahead horizon during these times of rapid change, something not shown in the overall analysis." It's hard for me to see this in the 1 week ahead graph. Three suggested changes: 1. use colors that are more easily distinguishable (I have trouble telling the difference between the median-ensemble line and the lp-lognormal). 2. Consider using different shapes for the different lines. 3. Revise the text to indicate which time periods you are referring to (late April-early May?).
lshandross commented 7 months ago

Addressed by the following commits:

  1. https://github.com/Infectious-Disease-Modeling-Hubs/hubEnsembles/commit/d23a35e370c0073a0ef5542d19b949ecdc9a3935
  2. Remove covidHubUtils dependency https://github.com/Infectious-Disease-Modeling-Hubs/hubEnsembles/commit/d23a35e370c0073a0ef5542d19b949ecdc9a3935
  3. https://github.com/Infectious-Disease-Modeling-Hubs/hubEnsembles/commit/e856a16a3a4cecd79cb0048dd01f5d7ebd3689f0
  4. The code may be relocated to be in the manuscript
  5. The results were not shown but have been removed since they were not directly related to the overall purpose (https://github.com/Infectious-Disease-Modeling-Hubs/hubEnsembles/commit/2fda5ea33ef27228274bc8465f1959465f685bc6)
  6. The figure has been split out for increased readability (https://github.com/Infectious-Disease-Modeling-Hubs/hubEnsembles/commit/ad408830efcc67b6364d2b5b616618a46e472311, https://github.com/Infectious-Disease-Modeling-Hubs/hubEnsembles/commit/5cb594805159da81f0e5f515187e3a32040901db), the colors changed (https://github.com/Infectious-Disease-Modeling-Hubs/hubEnsembles/commit/ffc0ce905c04bcc1020673297b75e094a2ace652), different shapes been added (https://github.com/Infectious-Disease-Modeling-Hubs/hubEnsembles/commit/41e337e622b584457ee5146ef6b12a2915e76310), and the text being more explicit (https://github.com/Infectious-Disease-Modeling-Hubs/hubEnsembles/commit/2e0daae95807f1974d9c0e1dba485baedc21440f, https://github.com/Infectious-Disease-Modeling-Hubs/hubEnsembles/commit/d1e3db5d7c4f3407d8e6b52d8fae641a0997cbe5)
lshandross commented 6 months ago

For (4), more of the code is being shown within the manuscript and reference is made to where to find the code that isn't shown in the manuscript, which was also raised in this issue https://github.com/Infectious-Disease-Modeling-Hubs/hubEnsemblesManuscript/issues/23