search

WadhwaniAI / covid-modelling

Repo for modelling the spread of novel Covid-19
MIT License
5 stars 0 forks source link

Data pipeline #45

Closed nayana-wiai closed 3 years ago

nayana-wiai commented 4 years ago

Making a PR so that @sansiddh and @ramkamalwai can have visibility. This can be integrated when all the pipelines are stable.

sansiddh commented 3 years ago

What is the status of the data pipelines? Can we merge this PR? Or close it?

nayana-wiai commented 3 years ago

This PR standardises the data pipeline across regions.

I've updated mainly data/processing/processing.py and data/processing/granular.py. I've added the new config parameters to docs/data. I have a 'label' to identify a region since sometimes we use state to identify a region and sometimes we use district. This label is currently being used to identify the pickle file to use for cached data. Any suggestions on a better way to do this? Should I just change this to district?

nayana-wiai commented 3 years ago

@siddhant-shingi I have updated data/processing/granular.py, I missed it earlier.

sansiddh commented 3 years ago

I would prefer if we don't merge this yet. If we need it for oncall work urgently, we can merge it