Explore options for creating unified top-level PRd dashboard

[stevehadd]

Should contain items for

• Data
• current activity
• trained models
  • access saved pre-trained models
  • access inference?
• results of previous experiments
• monitor results of models
• monitor resource usage? This could be done through azure ML, but require a more custom option (holoviz? a custom web site based on a framework)?] Potentially discuss with @andrewgryan closer to the time.

[stevehadd]

Also worth having another lab discussion about what lab people generally would like from a dashboard, so as to make it more likely that the work done here is reuable (not reuse of code necessarily, but reuse of the ideas an creation process).

[stevehadd]

It looks like one can create custom dashboards through the MLFlow API. This might be a good option for our unified dashboard which would contain https://www.mlflow.org/docs/latest/rest-api.html

• results display - side-by-side plots of precip from model & radar & ML (plus some differences)
• histograms of precip fractions across the whole domain and key sub domains (climatic zones? counties? maybe based on the summaries in month so far climate summaries, ascend might be useful for cutting out shapes of counties, talk to covid project people about cutting out admin areas).
• XAI metrics - for overall model, for particular outputs (i.e. for a particular precip inference, which features were more or less important), importance of individual training examples
• group metrics by weather type (decider)
• ability to apply to every day for as long as stage level 1 mogreps-g data is available.