search

mrc-ide / global-lmic-reports

Global LMIC COVID-19 reports. Updated to include HICs. Previous data is periodically backed up at https://mrcdata.dide.ic.ac.uk/global-lmic-reports/
https://mrc-ide.github.io/global-lmic-reports
9 stars 1 forks source link

Recent estimates looking unsmoothed—why? #4

Closed CGiattino closed 2 years ago

CGiattino commented 2 years ago

Hi,

I'm the researcher at Our World in Data who presents your model estimates of true infections. (See my article, which I update with new data weekly, here.)

The past few updates you've published have looked unsmoothed/noisy relative to previous updates. For example, on this chart showing Brazil. My plots of the data do seem to match what is shown on your reports.

I just wanted to check that you are aware of this, and to ask why the estimates are no longer smoothed/look so noisy.

Thanks, Charlie

lbaertlein1 commented 2 years ago

Hello, Just to confirm and help troubleshoot: I'm seeing this issue as well, but it seems limited to the infections estimates - where the incident hospitalizations, ICU admissions, and deaths appear to be similarly smoothed in old and new model runs. I compared v8 outputs from 2021-08-19 to outputs from 2021-12-05, looking at the daily change in y_mean within compartments, matched on country and date. I didn't see a difference between the two outputs for hospital_incidence, ICU_incidence, or deaths, but did see a difference in infections: the average day-to-day change in y_mean for infections increased from ~400 to ~800, and when plotting the infections estimates side-by-side from the two outputs, the more recent outputs appear noticeably less smooth for infections, but not hospitalizations, ICU, or deaths.

CGiattino commented 2 years ago

Yes, I was only referring to the infections estimates. That's the only data I use from your model output. I haven't looked at the other output like deaths, etc.

GBarnsley commented 2 years ago

Hi both, apologies I'd missed this issue. I'll have a look into it, there hasn't been any major changes to the way trends are modelling so this is most likely a bug in how the outputs are generated. Just to confirm @CGiattino are you taking the infections from the files in \data?

GBarnsley commented 2 years ago

I haven't found any reason to believe there is a problem with the code. What I believe is happening is because I have made a couple of changes to the likelihood over the last couple of weeks: integrating fitting to cases at the tail of the epidemic and adjustments to disease characteristics for the Delta Variant. Infections are closely affected by estimated Rt trends, which have become more noisy as the MCMC explores different balances of mobility and spline effects to fit to the new likelihood. You can see this from the estimated effective reproduction numbers on the webpages. Brazil in-particular is currently estimating a bi-model distribution for the effect of mobility so it looks particularly messy. The healthcare projections are only impacted by the changes in R through several compartments so are effectively smoothed by the model.

As the chains become better tuned, the trajectories should smooth out, given the priors are set to encourage minor changes, Though keep in mind the original outputs were the result of almost a years worth of tuning. I'll keep an eye on this however, and its likely that how we model trends may change in the future.

CGiattino commented 2 years ago

Yes, I take the files from /mrc-ide/global-lmic-reports/tree/master/data.

Okay, that all makes sense. Thanks for looking into this.