How will a model or project interface with a machine learning model?

[BrunoC-L]

Epidemiological modeling] is usually done with a set of equations that represent flows between states of infection. See Compartmental Epidemiological modeling

Assuming thee project has to support interfacing models with Machine Learning algorithms instead of equation solvers, there are a few questions:

• What kind of input and output will be given to the ML models?
• Is the ML model just there for hyper-parameter tuning?
• Does the software need to support outputting states, transitions, equations & hyperparameter usage? If so which format?
• Does the software need to support inputting of data received from an ML algorithm providing values to use in equations?

[simdem]

I will first give an overview of modeling and how ML fits in (recapping also some of the elements I discussed in the last meeting). I will answer the questions in a second post.

Model use and calibration. A model is a set of equations applied to some initial conditions using an ODE solver to obtain predictions of epidemiological variables of interest over time (number of persons in a given compartment, auxiliary measure like new infections). The model contains parameters which in a first step are randomly generated in given intervals (these intervals are assumptions, or derived from the literature). Public health data from the real world (hospitalizations, deaths, ...) can be used to validate the predictions. The prediction error can be minimized by tweaking the parameters (continuous optimization) or the model structure (combinatorial optimization). Parameters can be considered to be constants, or to be time-varying (piecewise constant, for simplicity). Technically, having piecewise constant parameters can give arbitrary flexibility. This is not useful for prediction (same logic as fitting data points with a polynomial of arbitrary order). So we want to allow changes of parameters only at the times of events that can change the dynamic of the epidemic (change in the level of public health interventions, return to school, etc.), or we can decide to allow them every X months (this would this be a model that is recalibrated regularly following a previsible schedule).

Machine learning point-of-view. Prediction of epidemic variables is a multidimensional time-series prediction problem. We would want to predict the new cases / new hospitalizations / etc. For example, if we have (x1,x2,x3,x4,x5,y1,y2,y3,y4,y5,z1,z2,z3,z4,z5) -- that is, data for three variables at times 1,2,3,4,5 -- we would like to predict (x6,x7,..., y6,y7,..., z6,z7,...). This can be done retrospectively (with existing time series for a period of time) or prospectively (gathering new data live as published by public health, in a sort of "active learning" fashion). Any statistical/ML model could to the job (ARIMA, convolutional neural networks, ...). But the pitfall of this approach is that we cannot predict the effect of changes in public health measures (e.g. scenarios of vaccination scale-up). For that, we need an interpretable model (the compartmental model) with which we can represent the effect of public health measures.

Merging compartmental models and ML. Time series data is used as calibration target for the compartmental model, and possibly also to determine the times at which parameter change (in piecewise fashion), or model structure change, would be useful. The compartmental model is used for predicting next values of the time series.

[BrunoC-L]

During said meeting 4 weeks ago, we proposed an MVP, minimal viable product, for the AI/ML part of the project. We agreed that a working algorithm could be grid search over the domain of hyper-parameters.

this means that we must:

1. output the list of parameters along with their domains
2. pass it to the model
3. get a list of values from the model (in this case, a random arrangement)
4. run the simulation
5. pass the results to the optimizer¹ (Most likely each Y_real & Y_pred)
6. if the stopping criteria is met, pick the result with the highest validation score, else go back to step 3

¹ The MVP could be achieved without passing the results since random search cares not for the scores or losses to generate new predictions, but it is still a required step in the general process.