search

djay / covidthailand

Thailand Covid testing and case data gathered and combined from various sources for others to download or view
126 stars 15 forks source link

Dashboard province data incorrect sometimes #58

Closed djay closed 3 years ago

djay commented 3 years ago

Seems to happen when it needs to get more than one date is one session for example 2021-09-01 is actually 85 not 35 but 156 is correct. https://github.com/djay/covidthailand/runs/3492265169?check_suite_focus=true

2021-09-02 00:00:00 MOPH Dashboard Retry Missing data at (datetime.datetime(2021, 9, 2, 0, 0), 'Buriram') for ['Vac Given 1 Cum', 'Vac Given 2 Cum', 'Vac Given 3 Cum']. Retry
2021-09-02 MOPH Dashboard 156.0 0.0 0.0 0.0 156.0 0.0 NaN NaN NaN NaN Buriram
...
2021-09-01 00:00:00 MOPH Dashboard Retry Missing data at (datetime.datetime(2021, 9, 1, 0, 0), 'Buriram') for ['Vac Given 1 Cum', 'Vac Given 2 Cum', 'Vac Given 3 Cum']. Retry
2021-09-01 MOPH Dashboard 34.0 0.0 0.0 0.0 34.0 1.0 NaN NaN NaN NaN Buriram

The existing code tries to reset the connection if there is a failure. somehow it must be going wrong. https://github.com/djay/covidthailand/blob/main/utils_scraping.py#L694

djay commented 3 years ago

This is fixed now. Still need to go back and reset some of the wrong data which has been remembered