cmu-delphi / delphi-epidata

An open API for epidemiological data.
https://cmu-delphi.github.io/delphi-epidata/
MIT License
100 stars 68 forks source link

Update COVID-19 Hospitalization by State documentation #796

Closed ryantibs closed 2 years ago

ryantibs commented 2 years ago

The COVID-19 Hospitalization by State documentation says that updates are roughly weekly. See the last bullet point here.

My understanding is that we have been able to find a way to get updates daily. We should update the documentation.

krivard commented 2 years ago

We probably just need to drop this sentence: "New issues are expected to be released roughly weekly."

We already say in the header that we pull data from two healthdata.gov datasets to improve the frequency of updates:

ryantibs commented 2 years ago

Sounds good (about dropping that sentence).

I'm confused about the regularity of update (is it really that irregular?). In recent times it has "felt" like it was regular, from my own use of the dashboard.

ryantibs commented 2 years ago

This probably doesn't belong here (on this issue), but I'm answering my own question, and recording here for now. Will be important for us to discuss at team leads shortly.

This is what I get for lag over time (at the national level).

library(covidcast)
library(ggplot2)

x = seq(as.Date("2020-09-01"), as.Date("2022-01-06"), by = "day")
y = sapply(x, function(date) date - max(
  covidcast_signal(
    "hhs", 
    "confirmed_admissions_covid_1d", 
    geo_type = "nation", 
    start = date - 10, 
    as_of = date)$time_value))
df = data.frame(x, y)
ggplot2::ggplot(df, aes(x = x, y = y)) + 
  geom_point() + 
  scale_x_date(breaks = "1 month", labels = "%b")
Screen Shot 2022-01-06 at 1 47 30 PM

So it's been regular at around 2 days for a long stretch, apart from a few recent hiccups, and irregular before that.

krivard commented 2 years ago

I'd be fine with us saying that updates are "typically daily", however there seems to be an active problem with the dataset starting January 2. This is their visualization tool, showing "Days since update" somehow getting larger and larger even though there are daily (and sometimes multiple times daily) updates since then:

image