Is total population information inherent to the initial conditions, or is there a separate parameter?
Can a model be made scale free? i.e. scaled to a population value of 1
Check at different time points to ensure conservation of total population
“Aggregate_parameter” interpretation
Check that a Mapping between model parameters described in literature and the corresponding aggregate parameters used for model representation/simulation
--> This should help interpret the parameters that are calibrated, as well as develop the mappings below in 5.
Scale/units test for parameters
In cases of “aggregated parameters” visible at the syntactic level, do the units agree when combining the original model parameters?
Can be checked for individual parameters: agreement with each other
Can be checked for aggregated parameters: agreement with state variables & corresponding equation
Check if parameters are explicitly being divided by population
Is a distribution provided (i.e. uniform distribution between a range of two values, or a gaussian distribution with a mean and variance provided)
Is provenance or reasonable assumptions for this distribution choice provided?
we hope this provides the necessary information for a posteriori adjustments to parameter choices of simulated forecasts are not satisfactory
Argument for evidence ontology?
Author statement
Calibration result
Transformation of existing evidence (aggregate parameters)
WAG
Prior predictive check against data to see if data is in support of parameters
Missing initial conditions:
Ensure information is provided to map from observed variables in data to initial conditions for model state variables
Check for the existence of a function that maps observed variables for a start time map to the initial conditions of model state variables
-Reasoning provided for this function choice
Observed variables should be conserved if fractions are used e.g. {infected} in observed data vs. {infected, sick, affected} model states
Check that all model state variables are provided an initial condition
Check that the initial state values are reasonable
Check that the solution mapping applied to the start state is the same as the observed variables for that date.
Observation models
Ensure information is provided to map from simulated state variable outputs from the model to observed variables
Check for the existence of a function that maps model state variables to values that agree with variables in observed data
Reasoning provided for this function choice
Check that values for every observed variables is calculated
Ensure mapping from IM_3b are not redundant combinations of variables
Intended idea: Flow from state to state is allowable; one state = scalar multiple of another shouldn’t be allowed
Example: “Hospitalization = 10% of Infected individuals” should not be allowed
Stiffness test
Check if stiffness arises from simple “patient zero” initial conditions (i.e. minimum infectious population and maximum susceptible)
-(b) Check that differences in parameter scales are reasonable to avoid introducing stiffness into the system (may not be a concern…)
Permanence of Death
Make sure dead individuals stay dead; i.e. no reaction that allow dead individuals to return to a non-dead state
Ensemble Model tests (EM tests)
Test scale of each model separately to test agreement
Ensure total populations are in agreement across different models (either scale-free or not)
Test that initial conditions after mapping are in agreement across different models
Test post-mapping variables are reasonable across different models (How do we do this?)
Model sanity checks:
Total population tests
“Aggregate_parameter” interpretation
Scale/units test for parameters
Parameter value tests
Missing initial conditions:
Observation models
Ensure information is provided to map from simulated state variable outputs from the model to observed variables
Stiffness test
Permanence of Death
Ensemble Model tests (EM tests)