epiforecasts / covid

Temporal variation in transmission during the COVID-19 outbreak
https://epiforecasts.io/covid/
MIT License
109 stars 30 forks source link

The recent NJ new case data has test reporting artifacts probably relating to easter. The last four days should be averaged or windowed. Other data may also have this problem #38

Closed guqin closed 4 years ago

gunnar-kaestle commented 4 years ago

To get rid of the weekend artifact, a 7 day window seems more appropriate = new cases of the last week as average value.

seabbs commented 4 years ago

Thanks for highlighting this - do you suggestions for an alternative data source?

In general, this is why we use a conservative reporting delay distribution to mitigate some of these issues. Our Rt estimates are also optimally windowed to reduce noise as a second step. Happy for alternative suggestions?

gunnar-kaestle commented 4 years ago

In principle, the moving average has a weird transfer function resulting in a bumpy amplification factor, meaning that small variations of the input variations can result in large output variations. https://electronics.stackexchange.com/questions/154039/whats-the-laplace-transfer-function-of-a-moving-average

Nevertheless, the 7-day-average compensates quite good the usual weekly behaviour of us humans, but I combined this serially with a first order low pass (e.g. T=2d) meaning after 3T 95% of a step response has been reached.

If you talk about probability distributions, then R is the integral of the probability distribution for a secondary infection r(t). Basically, the n.th generation is the n.th convolution of r(t), if I understand correctly. Thus the development for an (undisturbed) development with R=constant, it would be F(r(t))^n with F as Laplace transform of the probability distribution. The reporting delay as probability distribution m(t) is an additional step, also with a (single) convolution: F(r(t))^n * F(m(t)). Did you consult with somebody from measurement engineering how to implement best an R observer with changing r(t) and m(t)? With intensified contract tracing, the shape of the probability distribution does change and gets smaller tails, and not only reduce the R evenly as the area below the curve.

seabbs commented 4 years ago

Thanks for this and sorry for the delay.

We have now added a weekly reporting model to our updated estimates. We have not added a rolling average as we feel this obscures information. You are correct that this does lead to artefacts when known events occur (like here) but as we are trying to act as a tracker we feel that the benefit of having maximal information outweighs this. Closing this for now but happy to hear other ideas for reporting models that can account for this without smoothing information.

Time-varying generations times are also something we are looking at. The issue with this is two-fold 1. is data and 2. is fitting the model with two smoothly varying parameters.