Santa Cruz County, California Rt model

[keith-bontrager]

Is this forecast model current? The value is inconsistent with trends in local infection stats and with other Rt models. It is pulling the Calcat ensemble down.

Regards,

Keith

[keith-bontrager]

[joshuaschwab]

I just updated the Santa Cruz forecast with data through Nov 6. Median Rt is now 0.74, higher than in the screenshot but still below 1.0. The issue is that the core LEMMA model (master branch) used in CalCAT fits to hospitalizations, ICU and deaths, but not cases. While cases have been increasing, hospitalizations have not (there was a two day spike in ICU on Nov 4-5 that may be a data error - on Nov 6 ICU is back to its previous level). Fitting to cases has advantages (hospitalizations lag cases by about a week) but also drawbacks (can be influenced by availability of testing and who is tested). We have a FitToCases branch of LEMMA that does use cases. I will run Santa Cruz using the FitToCases branch and update this Issue with that forecast.

[joshuaschwab]

Using the FitToCases branch we get a median Re of 1.34. https://github.com/LocalEpi/LEMMA-Forecasts/blob/addCases/Forecasts/Santa%20Cruz.pdf This is a particularly tough case to forecast because the data are pointing in very different directions - hospitalizations are falling but cases are rising. Given that some CalCAT models are using cases, it doesn't seem like a terrible idea to include LEMMA without cases in the ensemble. But it would be important for users to look at more than just the point estimate of the ensemble. I would hope that users would see the wide range of forecasts from the models and take away that there is a lot of uncertainty right now. With that said, our team is meeting this week to considering changing the main LEMMA model to include cases, given that testing availability is more stable now than it was early in the epidemic.

[keith-bontrager]

Hi Joshua,

Thanks for getting back on this.

The county population is small so the stats are lumpy at best. I understand why the case rates are not very useful for modelling. The local public health folks tried to increase the testing volume but only got to the state recommended minimum. It is still far from useful from a surveillance POV.

The mortality rate was very low until about a month ago when there was an outbreak in an acute care facility and doubled in a few weeks. They'd managed to avoid that for the entire duration until then. ICU bed use spiked at the same time. Given the small numbers it's likely that big changes in hospitalizations and mortality numbers are going to make the forecast swing around a bit unless they can be weighted to accommodate that somehow. But that sort of complexity might make the simple model a lot less so.

I've tried to correlate the Rt forecasts from a few models to the time history of case rates in the county (conveniently ignoring the bias in the case numbers). There are some intervals where the forecast seems to be in synch, and others where they are definitely not. Doing the same for the state works out a lot better. That points to the problem with stats in a small population.

I'm trying to help some locals understand the forecasts, which is what motivated the original question. My background is in physics, so the math is familiar, but I am not fluent in the various epidemiological models so there are plenty of gray areas.

Now it's getting cold and the case rates are on a fairly steep rise. It'll take another week or two for the hospitalizations to show a trend if they are going to. We'll see how that goes.

Again, thanks for the reply and the effort you and yours are putting into this.

Regards,

Keith

---

From: Joshua Schwab notifications@github.com Sent: Monday, November 9, 2020 7:04 PM To: LocalEpi/LEMMA-Forecasts LEMMA-Forecasts@noreply.github.com Cc: keith-bontrager keith_bontrager@hotmail.com; Author author@noreply.github.com Subject: Re: [LocalEpi/LEMMA-Forecasts] Santa Cruz County, California Rt model (#4)

Using the FitToCases branch we get a median Re of 1.34. https://github.com/LocalEpi/LEMMA-Forecasts/blob/addCases/Forecasts/Santa%20Cruz.pdf This is a particularly tough case to forecast because the data are pointing in very different directions - hospitalizations are falling but cases are rising. Given that some CalCAT models are using cases, it doesn't seem like a terrible idea to include LEMMA without cases in the ensemble. But it would be important for users to look at more than just the point estimate of the ensemble. I would hope that users would see the wide range of forecasts from the models and take away that there is a lot of uncertainty right now. With that said, our team is meeting this week to considering changing the main LEMMA model to include cases, given that testing availability is more stable now than it was early in the epidemic.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/LocalEpi/LEMMA-Forecasts/issues/4#issuecomment-724419359, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARVOSWWX5IMFSCF56MC6ZCLSPCUVRANCNFSM4TN3BXQQ.

[joshuaschwab]

Hi Keith, Thanks for the additional context, that's helpful. You've probably seen the Santa Cruz model: https://www.santacruzhealth.org/HSAHome/HSADivisions/PublicHealth/CommunicableDiseaseControl/CoronavirusHome/LocalCOVID-19ForecastModels.aspx The Santa Cruz model includes cases and is pretty similar to LEMMA (the Santa Cruz team was very helpful to us in moving our model to Stan). I'll be curious to see their Rt forecast when it updates tomorrow. Josh

[keith-bontrager]

Hi Joshua,

I've seen the local forecasting page. It doesn't look like they've updated it yet but I'll keep an eye on it.

Some additional context for you: The local public health folks are attributing the rise in cases + decline in hospitalizations to infections in kids caused by halloween activities. Lots of new infections in the under 19 demographic. Infection rates in that group had been well below the average otherwise. There were very few kids on the streets so it's likely that parents organized indoor parties. Tha's probably happening in a lot of places though it may be masked by other things in areas that are hot spots now.

I tracked the effects protests and beach holiday weekends (thousands of out of town folks come here) had on the epidemic curve last summer. There was no obvious change in the rates.

I'll leave you to it now. Thanks for the conversation.

Regards,

Keith

---

From: Joshua Schwab notifications@github.com Sent: Tuesday, November 10, 2020 2:11 PM To: LocalEpi/LEMMA-Forecasts LEMMA-Forecasts@noreply.github.com Cc: keith-bontrager keith_bontrager@hotmail.com; Author author@noreply.github.com Subject: Re: [LocalEpi/LEMMA-Forecasts] Santa Cruz County, California Rt model (#4)

Hi Keith, Thanks for the additional context, that's helpful. You've probably seen the Santa Cruz model: https://www.santacruzhealth.org/HSAHome/HSADivisions/PublicHealth/CommunicableDiseaseControl/CoronavirusHome/LocalCOVID-19ForecastModels.aspx The Santa Cruz model includes cases and is pretty similar to LEMMA (the Santa Cruz team was very helpful to us in moving our model to Stan). I'll be curious to see their Rt forecast when it updates tomorrow. Josh

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/LocalEpi/LEMMA-Forecasts/issues/4#issuecomment-724997883, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARVOSWT7YIILDDHSPKHCHJDSPG275ANCNFSM4TN3BXQQ.

[keith-bontrager]

They just updated the local model. Rt is up a little over 1 now. The ensemble forecast is in pretty close agreement with the local forecast.

---

From: Joshua Schwab notifications@github.com Sent: Tuesday, November 10, 2020 2:11 PM To: LocalEpi/LEMMA-Forecasts LEMMA-Forecasts@noreply.github.com Cc: keith-bontrager keith_bontrager@hotmail.com; Author author@noreply.github.com Subject: Re: [LocalEpi/LEMMA-Forecasts] Santa Cruz County, California Rt model (#4)

Hi Keith, Thanks for the additional context, that's helpful. You've probably seen the Santa Cruz model: https://www.santacruzhealth.org/HSAHome/HSADivisions/PublicHealth/CommunicableDiseaseControl/CoronavirusHome/LocalCOVID-19ForecastModels.aspx The Santa Cruz model includes cases and is pretty similar to LEMMA (the Santa Cruz team was very helpful to us in moving our model to Stan). I'll be curious to see their Rt forecast when it updates tomorrow. Josh

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/LocalEpi/LEMMA-Forecasts/issues/4#issuecomment-724997883, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARVOSWT7YIILDDHSPKHCHJDSPG275ANCNFSM4TN3BXQQ.