neherlab / covid19_scenarios

Models of COVID-19 outbreak trajectories and hospital demand
https://covid19-scenarios.org
MIT License
1.36k stars 352 forks source link

Actual ICU case numbers exceed model's ICU bed capacity in some cases #621

Open ccpf opened 4 years ago

ccpf commented 4 years ago

πŸ› Bug Report

The number of actual cases in ICU sometimes exceeds the models ICU bed capacity (for instance in the ESP-Madrid scenario).

How to reproduce

Steps to reproduce the issue:

  1. Open the application in a browser
  2. For instance, run the ESP-Madrid scenario and compare the ICU cases from data to the model's ICU bed capacity. Actual data show about 1500 cases in ICU while the model uses 634.

😯 Current Behavior

The model ICU bed capacity is below the official number of cases in ICU

πŸ€” Expected Behavior

The model's ICU bed capacity should not be below the official number

πŸ’ Possible Solution

You could have a condition in the model whereby the models bed capacity can never be below the officially reported number of current ICU cases. I guess it would be an easy thing to do that when a scenario loads you perform a quick check of whether the maximum of ICU cases exceeds the model's preset ICU bed capacity and if so to simply adjust the model capacity to match the maximum in the data. Of course the user can always adjust the number manually but some may not see it and therefore get bad results.

πŸ”¦ Context

As you penalize ICU overflow with an increased mortality (severity parameter is set to 2 by default), this will overestimate your mortalities in some cases. You can of course adjust it by: (i) setting the severity parameter to 1 but this means that overflows down the road will not be penalized, or (ii) you can simply increase the bed capacity manually (if the user is aware of the issue).

πŸ’» Code Sample

🌍 Your Environment

no relevant

Related

-

noleti commented 4 years ago

A bit more context: current ICU capacity data is hard to get for countries, let alone states. We aggregate a number of manual and automatic sources for this data; when good data is not available we are estimating capacity (usually based on 2-3% of hospital beds). What we could do in addition would be to check if official ICU bed data or estimates are below the reported current ICU cases in the parser output. That data source usually cannot be trusted fully either, but it could give at least an indication of numbers we have to look at again.

nnoll commented 4 years ago

Another thing to keep in mind is what the definition of critical means for each reporting source. For example, it could (and does in some cases) refer to both "counted" ICU beds and makeshift ICU beds. The later would correspond to the overflow category for us. In this case, it is more appropriate to compare the empirical data to critical + overflow for us (which is the total number of critical cases).

noleti commented 4 years ago

If the empirical data should be below our simulation critical+overflow, then we cannot use it to set the ICU beds only (which need to be set before simulation, and critical+overflow is a result of simulation).

The pragmatic solution to this would be to just require the user to increase the ICU bed parameter if (subjective) discongruity is observed.