search

cmu-delphi / covidcast-indicators

Back end for producing indicators and loading them into the COVIDcast API.
https://cmu-delphi.github.io/delphi-epidata/api/covidcast.html
MIT License
12 stars 17 forks source link

Patch 07/31/2024 data for hospital admission #2011

Open aysim319 opened 4 months ago

aysim319 commented 4 months ago

Actual Behavior: Chronicle run for hopsital admission for 7/31 failed due to exceeding expected max files. The files themselves are correct and the expected max size needs to be increased

Expected behavior should run fine Context (https://delphi-org.slack.com/archives/C01LZ3A2UMU/p1722393650290459)

minhkhul commented 4 months ago

Let's wait for a few days and see if we keep geting 4+ files, which is max of what we expect. I wrote a script to check how many times have we received 3+ inpatient source files per day and there are a few from 2022. But only yesterday do we receive 5 files. Which means it could just be yesterday being a weird day.

aysim319 commented 2 months ago

Need to consider the parquet files in prod that is held off for now since more formal process needs to be taken going to take a while. But will patch the data and come back when things are more formally set up

aysim319 commented 2 months ago

finished with the patch, but not done with the parquet file until we figure out how to deal with them