Add NSSP data as an endpoint

[nickreich]

https://covid.cdc.gov/covid-data-tracker/#ed-visits

[krivard]

Nice! This particular link looks like it would be compatible with the covidcast schema, but NSSP provides way, way more than that through ESSENCE. Are you just looking for the COVID-related ED visits, or something more detailed that would require its own endpoint?

[nickreich]

I'm not clear on the "endpoint" terminology, but yes, it would be nice to have multiple different signals (COVID-related ED visits, flu-related ED visits, etc...). This has come up recently as a possible data source to use for future FluSight challenges, in conversations with CDC, UMass, and CMU Delphi folks.

[krivard]

Ah sorry, by "endpoint" I mean the following: Each endpoint of the API returns data in a different format. For example, the covidcast endpoint is largely designed for sample-based estimates, and includes only the value, stdev, and sample size in each row returned; by contrast, the covid_hosp endpoint returns over 60 different fields in each row, most of them raw counts; the fluview endpoint returns 5 different fields plus an age-stratified count.

Making a new endpoint means we can support whatever output format we want, but you have to use the delphi_epidata client to access the data, which is rather bare-bones at the moment. Adding it into the covidcast endpoint restricts the output format, but you can use the covidcast client libraries to access the data, which do things like automatically format the results as a data frame, plot choropleth maps, and compute correlation analyses.

If you need a small number of time series (like <5?) or you don't know exactly which ones you want yet, then adding this into covidcast is the most expedient.

So far I have:

• COVID-related ED visits
• Influenza-related ED visits

Anything else?

[nickreich]

mentioning here that I've had an offline discussion with @ryantibs about this. I'm honestly not sure exactly how many time-series might be useful from the larger NSSP/ESSENCE systems. This may tie into future plans for using these data sources for future flu modeling efforts as well.

[brookslogan]

TL;DR: I can't find any regularly updated NSSP ILI or influenza time series data. That would be an issue for potential usage for flu forecasting.

There are multiple ways that NSSP data extracts are currently published; here are three:

• State&agegroup-level (linked above): diagnosed COVID-19, reference dates from 2020-06-16, regular version history from WebArchive from 2020-08-24, seems to be regularly updated
• National+HHS-level: CLI and ILI, reference weeks from 201940, spotty version history from WebArchive from 2021-04-11, maybe because the time series ends with 2020w14 (2020, not 2021) and isn't regularly updated
• National? via MMWR: COVID-19 and influenza, reference times from Jun 2018, maybe just a one-off image graphic rather than something regularly scrape-able

The state&agegroup-level "diagnosed COVID-19" series is the only one I have seen with ongoing updates. If this is to be used for ILI situational awareness or forecasting, there will need to be regularly published/shared ILI time series as well.

[nickreich]

I want to strongly bump this up as a request for hoovering these data into the EpiData API. It came up again today in a conversation with CDC as these data may serve as future modeling/forecasting targets. I see two related datasets:

• 2023 Respiratory Virus Response - NSSP Emergency Department Visit Trajectories by State- COVID-19, Flu, RSV, Combined
• 2023 Respiratory Virus Response - NSSP Emergency Department Visit Trajectories by State and Sub State Regions- COVID-19, Flu, RSV, Combined 

[melange396]

This may be achieved, at least partially, by https://github.com/cmu-delphi/covidcast-indicators/pull/1952