Wrapper for counterfactual simulations

[adrian-lison]

We need some practical structure for running our intended scans over interventions and time to detections in parallel and for many different seeds.

[adrian-lison]

Okay, here is a potential draft, what do people think? @smodee @joyiichow @ConradKunadu

It would probably be good to have an additional class CounterfactualSim which has a similar constructor as the Sim class, but that allows to provide a list of intervention packages (for example a "weak", "medium" and "strong" package that we want to compare). Depending on what approach we take for implementing adaptive interventions, we might also supply events that trigger these interventions. Then, in the CounterfactualSim class, we probably want the following functionality:

• a constructor which stores all the supplied parameters, including a supplied RNG seed (or creates one)
• an initialization method which initializes and runs the baseline scenario (no interventions). This includes the following tasks
  • save the people object (e.g. from synthPops) created during initialization of the baseline scenario, store it in a temporary file and save the file location in an object variable
  • check whether there happens an epidemic at all in the simulated baseline scenario (depending on the initial starting conditions, the outbreak may die out by itself)
  • determine what the range of detection times is that we want to scan (lower and upper bound) --> this is a task for another issue, maybe have a simple placeholder here
• a method scan_detection_times which simulated for each intervention package a range of detection times (determined during initialization above) and stores all the results
  • To do these simulations, the method has to create many copies of the baseline scenario, but with specific interventions and detection times supplied. To me it seems that the easiest approach based on the current implementation is to create new Sim objects with the respective interventions etc., but to supply the peoples object via the popfile argument. The disadvantage is that this loads this file over and over again, creating some overhead. Advantage is that this guarantees all objects to be completely independent (deepcopy does not seem to work reliably for pathosim objects) and this also makes parallelization more realistic (though not sure if we even want to parallelize at this level).
  • For storing results, I would prefer storing by detection time at the innermost level, and then by intervention package, not the other way round - because the most intuitive post-hoc analysis I would do is to look at QALYs as a function of detection time for a given intervention set)
• further methods for computing results statistics (e.g. for differences between detection times or intervention packages), and plotting

---

Once we have this class, we can add another CounterfactualMultiSim class that simply creates a lot of CounterfactualSim objects with different seeds:

• if we want to condition on an outbreak actually occurring (see Conrads thoughts on slack), we can have a dynamic procedure which initializes new CounterfactualSim objects one at a time and checks whether the outbreak really takes off. If not, it throws away that object and creates a new one with a different seed until we have enough seeds (according to some prespecified number, e.g. 100) for which the outbreak actually takes off.
• some functionality for running the scan_detection_times method of the different CounterfactualSim objects with the different seeds in parallel
• some functionality for aggregating the results from each CounterfactualSim across the different seeds and reporting/plotting summary statistics and distributions.

[ConradKunadu]

Okay, here is a potential draft, what do people think? @smodee @joyiichow @ConradKunadu

It would probably be good to have an additional class CounterfactualSim which has a similar constructor as the Sim class, but that allows to provide a list of intervention packages (for example a "weak", "medium" and "strong" package that we want to compare). Depending on what approach we take for implementing adaptive interventions, we might also supply events that trigger these interventions. Then, in the CounterfactualSim class, we probably want the following functionality:

• a constructor which stores all the supplied parameters, including a supplied RNG seed (or creates one)

• an initialization method which initializes and runs the baseline scenario (no interventions). This includes the following tasks

  • save the people object (e.g. from synthPops) created during initialization of the baseline scenario, store it in a temporary file and save the file location in an object variable
  • check whether there happens an epidemic at all in the simulated baseline scenario (depending on the initial starting conditions, the outbreak may die out by itself)
  • determine what the range of detection times is that we want to scan (lower and upper bound) --> this is a task for another issue, maybe have a simple placeholder here

• a method scan_detection_times which simulated for each intervention package a range of detection times (determined during initialization above) and stores all the results

  • To do these simulations, the method has to create many copies of the baseline scenario, but with specific interventions and detection times supplied. To me it seems that the easiest approach based on the current implementation is to create new Sim objects with the respective interventions etc., but to supply the peoples object via the popfile argument. The disadvantage is that this loads this file over and over again, creating some overhead. Advantage is that this guarantees all objects to be completely independent (deepcopy does not seem to work reliably for pathosim objects) and this also makes parallelization more realistic (though not sure if we even want to parallelize at this level).
  • For storing results, I would prefer storing by detection time at the innermost level, and then by intervention package, not the other way round - because the most intuitive post-hoc analysis I would do is to look at QALYs as a function of detection time for a given intervention set)
• further methods for computing results statistics (e.g. for differences between detection times or intervention packages), and plotting

Once we have this class, we can add another CounterfactualMultiSim class that simply creates a lot of CounterfactualSim objects with different seeds:

• if we want to condition on an outbreak actually occurring (see Conrads thoughts on slack), we can have a dynamic procedure which initializes new CounterfactualSim objects one at a time and checks whether the outbreak really takes off. If not, it throws away that object and creates a new one with a different seed until we have enough seeds (according to some prespecified number, e.g. 100) for which the outbreak actually takes off.
• some functionality for running the scan_detection_times method of the different CounterfactualSim objects with the different seeds in parallel
• some functionality for aggregating the results from each CounterfactualSim across the different seeds and reporting/plotting summary statistics and distributions.

Hope this isn't too late, but this seems great - really like this suggestion! Really have no constructive feedback here