ScottishCovidResponse / SCRCIssueTracking

Central issue tracking repository for all repos in the consortium
6 stars 0 forks source link

Ascertain whether output is correct #547

Closed kzscisoft closed 4 years ago

kzscisoft commented 4 years ago

The produced totals.qs file contains a data frame of totals for each compartment. However:

UPDATE: Talked to Ed from Swansea, apparently this is the integral number of people in each compartment. If you divide by the 365 days you get within 35,000 of the actual population for Glasgow.

UPDATE: Bizarrely this only applies to UK | Glasgow City in the samples, all others I have tried including UK | Fife, UK | Bristol, City of and UK | Cheshire West and Chester show the same lockdown scenario sending number of deaths to 0. Worrying still is that this value of zero does not appear to change even when lockdown duration is set to 0.

Until the exact meaning for these totals is established we cannot know if the new model run is valid, or what the data represent. Unfortunately the plotting script included (which contains unpresent imports) is specific to the original running structure of having every region, every lockdown scenario etc.

github-actions[bot] commented 4 years ago

Heads up @kzscisoft @richardreeve - the "model reconstruction" label was applied to this issue.

Nardus commented 4 years ago

Based on the plotting code in UK-view.R, the "totals" files likely do not contain any usable/interesting information. In this script, the various "totals.qs" files are used only in:

plot_attackrate()

Plots numbers in the 'cases' and 'deaths' compartments by age, but this plot is not used in the paper or supplement.

plot_epi()

Uses the totals data only to restrict which runs from the corresponding dynamics file are summarised. E.g. in the caption of figure 2A:

"From 200 realisations of each projection, 11 representative simulations are shown: one for each decile of the total number of cases, with the bold curve showing the simulation resulting in the median projected number of cases."

In the code, these deciles are identified by summing across all compartments in the totals file, but this must be for convenience only, since this clearly does not represent the true number of cases (at least based on the totals file output by a local run of the adapted code, where the sum would include susceptibles as well as the different bed counts).

All other plots and summaries are derived from the dynamics files, which based on the plotting code is supposed to be tracking the counts in all compartments over time and is likely the output we want. The dynamics file produced by a local run of the adapted code currently only contains counts for compartments named trace_school and trace_intervention - it's unclear why the other compartments are not present.

Nardus commented 4 years ago

A missing line in the SCRC adapted version of add_dynamics() (in Utilities.R) explains the missing compartments in dynamics output.

When this is fixed, make_table() in in the plotting code creates a range of useful summaries from the dynamics output, including total cases and total deaths. These numbers look more realistic.

kzscisoft commented 4 years ago

Hi @Nardus this is great work thank you! I will review your changes and get them merged in as soon as possible.