Make .xls more parsable

ccodwg / Covid19Canada

Epidemiological Data from the COVID-19 Epidemic in Canada

https://opencovid.ca/

Creative Commons Attribution 4.0 International

325 stars 139 forks source link

Make .xls more parsable #5

Closed noleti closed 4 years ago

noleti commented 4 years ago

I'm writing a parser for the .xls for https://github.com/neherlab/covid19_scenarios, and ran into some minor issues. In particular

The data rows are not starting on the same row among worksheets (4,3,3 for cases, deaths,recovered)
province is used as label for two columns on the recovered sheet
there is no 'health_region' column on the recovered sheet
dates colums are labelled 'date_report' or 'date_death_report' and 'date_recovered'. It would be easier if they would all be called 'date_report' if possible It would be great if these could be unified to make parsing easier.

ishaberry commented 4 years ago

Hi there-- I have updated this so the data rows are all 4, and so that province is not used as a label for two columns. We are not collecting data for recoveries at the health_region level (because this is not being reported in a consistent manner) so there is no column for that. We have kept separate names for the dates because these do not line up with each other, and reflect different things (one is the date of a case report, one is date of reported death, and one is a timeseries of dates for recoveries.). Hope that helps!

noleti commented 4 years ago

Thanks a lot for the quick fix, this simplified the parser already. For your reference, my parser looks like this: https://github.com/noleti/covid19_scenarios_data/blob/canadaParser/parsers/canada.py

ishaberry commented 4 years ago

Amazing!!