Discrepancy between incidence and cumulative death forecasts for LANL-GrowthRate

[eycramer]

Hi @LaCastro,

We are filing this issue because we noticed that your one-week ahead forecasts of incident and cumulative deaths didn't agree with each other this week. We would expect the one-week ahead predictive median for cumulative deaths to be approximately equal to the last reported cumulative deaths plus the one-week ahead predictive median for incident deaths, but this was not the case for your forecast submitted this week. The reported cumulative deaths for the US as of Saturday December 26th was 331,909 according to the data reported in the JHU CSSE repository at the end of day on Sunday December 27th. The median of your submitted one week ahead forecast for cumulative deaths was 356,975.5, which suggests a forecast of 25,066.5 incident deaths. However, this is not in agreement with the median of the submitted one week ahead forecast for incident deaths, which was 21,098.

We can imagine that this might happen for any of several reasons and we wanted to bring this discrepancy to your attention in case you were not aware of it.

Can you please respond to this issue with a description of why there is this discrepancy and also whether you have a plan to fix it? Thanks!

[LaCastro]

Thank you for bringing this to my attention. It does sound like it could be a number of things. Could it be that our last submitted forecast was from Dec 24th?

Our team is out this week, returning back to work on Jan 4th. I will look into the discrepancy first thing when returning and also check to make sure it’s not included in the forecast we submit on the 4th. Since we will not be able to address the discrepancy for this week in a timely manner, we understand if the forecast is not eligible for inclusion in the ensemble this week.

On Tue, Dec 29, 2020 at 8:40 AM Estee Cramer notifications@github.com wrote:

Hi @LaCastro https://github.com/LaCastro,

We are filing this issue because we noticed that your one-week ahead forecasts of incident and cumulative deaths didn't agree with each other this week. We would expect the one-week ahead predictive median for cumulative deaths to be approximately equal to the last reported cumulative deaths plus the one-week ahead predictive median for incident deaths, but this was not the case for your forecast submitted this week. The reported cumulative deaths for the US as of Saturday December 26th was 331,909 according to the data reported in the JHU CSSE repository at the end of day on Sunday December 27th. The median of your submitted one week ahead forecast for cumulative deaths was 356,975.5, which suggests a forecast of 25,066.5 incident deaths. However, this is not in agreement with the median of the submitted one week ahead forecast for incident deaths, which was 21,098.

We can imagine that this might happen for any of several reasons and we wanted to bring this discrepancy to your attention in case you were not aware of it.

Can you please respond to this issue with a description of why there is this discrepancy and also whether you have a plan to fix it? Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/reichlab/covid19-forecast-hub/issues/2457, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACBNBNSEFCPVZOJPVXS27JDSXHZ5DANCNFSM4VNKTWSQ .

[eycramer]

@LaCastro

Thanks for looking into this! It definitely might be because you submitted on the 24th. Your model was still included in the ensemble this week.

[LaCastro]

@eycramer

I checked our submission and I believe the apparent discrepancy comes from the fact that we were submitting based on data available through December 23rd.

We internally produce daily forecasts which we use to aggregate to the weekly level. Our median cumulative deaths forecast for December 26th was 335,851.0. With our median one week ahead forecast for cumulative deaths at 356,975.5, this would suggest an incident deaths forecast of 21,124.5. This is in approximate agreement with our submitted one week ahead forecast for incident deaths of 21,098.

The apparent discrepancy comes from the fact that our cumulative deaths forecast for December 26th was off by about ~4,000 from what was eventually reported.

Since this issue only comes up if we don't use the last reported data available for a given forecast week, there is no fix at the time. We anticipate submitting regularly on Mondays from here on.

[eycramer]

@LaCastro

Thanks for verifying! I'll close this issue now.