Systematic validation of outputs

[HSalat]

Create a workflow allowing users to evaluate the accuracy of the modelling for the area that they have chosen

1. Design an algorithm to ML the typical variable distributions and their heterogeneity for a type of area (region, rural/urban e.g.)​

   • Should learn from the set and from external control data​
   • Try to cluster areas​
   • Should be flexible enough to allow identification of sub-optimal fits if match expected distribution​

2. Decide distribution "measures" that inform the user of the quality of the local modelling​

   • Inform rather than judge​
   • Distribution moments

Resources

• Potential R packages: gamlss, fitdistrplus, univariateML.
• Other: Facebook’s prophet for GAM modeling, SDMetrics (Python).

Tasks

• [ ] Lit rev existing methods to compare probability distributions / identify packages and resources
• [ ] Identify potential control data sources (all GB, focused examples, potentially available to end user)
• [ ] Build API to scrape potential control data sources
• [ ] Build API to mass extract distributions from SPC outputs
• [ ] Implement ML algorithm to explore these datasets
• [ ] Design a flexible probability distribution comparison tool
• [ ] Implement a generic probability distribution comparison tool
• [ ] Implement a user interface for SPC
• [ ] Write methodology/result paper
• [ ] Look back at the work done with the feeling of satisfaction that accompanies a well polished piece of work