Discrepancy between incidence and cumulative death forecasts for JHU_IDD-CovidSP

[eycramer]

Hi @jlessler,

We are filing this issue because we noticed that your one-week ahead forecasts of incident and cumulative deaths didn't agree with each other this week. We would expect the one-week ahead predictive median for cumulative deaths to be approximately equal to the last reported cumulative deaths plus the one-week ahead predictive median for incident deaths, but this was not the case for your forecast submitted this week. The reported cumulative deaths for the US as of Saturday August 29 was 182761, according to the data reported in the JHU CSSE repository at the end of day on Sunday August 30. The median of your submitted one week ahead forecast for cumulative deaths was 187508, which suggests a forecast of 4747 incident deaths. However, this is not in agreement with the median of the submitted one week ahead forecast for incident deaths, which was 6710.

We can imagine that this might happen for any of several reasons: for example, you may be using two different models to forecast incident and cumulative deaths, or you may be calculating cumulative deaths using a different reference data set or a different procedure for aggregating to the national level than we are (for reference, we describe our approach to calculating aggregated cumulative deaths based on the JHU CSSE data here). Regardless, we wanted to bring this discrepancy to your attention in case you were not aware of it.

Our goal is to make the ensemble forecasts of incident and cumulative deaths consistent with each other. This is only possible if the submitted forecasts of incident and cumulative deaths from each model are consistent with each other. Can you please respond to this issue with a description of why there is this discrepancy and also whether you have a plan to fix it? Thanks!

[jlessler]

The "discrepancy" is because we base our forecasts on USA Facts data, not JHU CSSE. We have not been prioritizing switching to JHU CSSE because of problems with the JHU CSSE data set and how we do our modeling (which may have been recently fixed), but can move it up the priority list if it is an issue.

[nickreich]

@jlessler It is a priority for us and CDC to try to have the ensemble forecasts for incident and cumulative deaths be a bit more aligned than they are now. So I guess it might be useful to move it up the priority list a bit if possible, or alternatively to think about whether there is another more post-hoc approach that could be used to sync up the numbers between the incident and cumulative forecasts.

[nickreich]

Happy to discuss more if we can be of help in figuring out the best approach here.

[jlessler]

We will be converting to JHU-CSSE for at least the baseline value for the creation of the cumulative forecast next week. Just did not have time to get the change in for this weeks forecast.

[nickreich]

Great - thanks. We have been monitoring the discrepancies each week and will keep an eye to see if this solves the discrepancy that we see on our end.

[elray1]

Looks like this has been resolved; I'll close this issue.