search

chrism0dwk / covid19uk

MIT License
11 stars 10 forks source link

Problem with loading commute data #9

Closed GBarnsley closed 4 years ago

GBarnsley commented 4 years ago

Function load_commute_volume in pydata.py doesn't work correctly. The sort_index sorts by day not date so it ends up in the wrong order and commute.merge doesn't correctly match several dates. Adding "commute_raw.index = pd.to_datetime(commute_raw.index, format = '%d/%m/%Y')" just after "commute_raw = pd.read_csv(filename, index_col='date')", changes the index to a datetime format, allowing the sorting and merging to work correctly.

chrism0dwk commented 4 years ago

@GBarnsley good spot, thanks! Actually, this was caused by a data update having a different date format. Format now fixed as ISO %Y-%m-%d. Will Pandas barf if another date format is offered, I wonder?

GBarnsley commented 4 years ago

Yeah it will break if the day, month, year order changes

chrism0dwk commented 4 years ago

Good! We'll then know the format has changed.