ECMWFCode4Earth / ml_drought

Machine learning to better predict and understand drought. Moving github.com/ml-clim
https://ml-clim.github.io/drought-prediction/
90 stars 18 forks source link

Review - Analysis notebook #108

Open shaunharrigan opened 5 years ago

shaunharrigan commented 5 years ago

Hi @tommylees112 and @gabrieltseng,

It is my opinion that the "ml_drought/notebooks/docs/05_Analysis.ipynb" notebook is most important as it brings to full project together for prospective users, and is where you can really demonstrate and show off the fruits of your labour. It would be nice (if you get time) to expand this more.

Point 1: In the project outset you state that a key aim was the interpretability of the ML methods for drought prediction - what is the model really learning. It would be nice if the notebook had a step-by-step demonstration on how this can be done, with some example figures/analysis.

Point 2: It would be nice to see a more complete verification of the method together with evaluation metrics and some figures. It would be cool to see the scores of the ML method against the persistence benchmark, and even better some plots.

Point 3: I may have missed it given missing previous meetings, but to what extent is SEAS5 precipitation forecasts being used? Is there an example using SEAS5 or is it used in the primary model as a predictor anyway? Which other SEAS5 variables are used, if any?

I'm sure you are preparing more analysis material as the project is coming to a close, and perhaps have already plans to include more information in the notebook.

Great work guys :)

PS I can't add the "review" label as I'm not a contributor.