Data have not been updated for 4 Days

RuijingZ commented 2 years ago

Hi there,

Data have not been updated for 4 days, but the official dashboard has data until 08/16/2022. Could you check if files can resume updating?

Thanks, Ruijing

dancarmoz commented 2 years ago

Thanks for writing! I updated it now and it should resume updating. This isn't the first time this happened, unfortunately, where the thread running the script just stops for no clear reason. Unfortunately this leaves serious gaps in the history of the files. I might be able to fill these gaps in the future from other repositories, but it would be quite a bit of work.

If you ever notice in the future that the script isn't updating, please do let me know. I do not check on it as often as I should, these days. Normally any update should be synced to the repo in at most 30 minutes after first becoming available, so even if you see an update has not been uploaded for a few hours after being available in the dashboard, that's cause enough for concern.

RuijingZ commented 2 years ago

Thanks for writing! I updated it now and it should resume updating. This isn't the first time this happened, unfortunately, where the thread running the script just stops for no clear reason. Unfortunately this leaves serious gaps in the history of the files. I might be able to fill these gaps in the future from other repositories, but it would be quite a bit of work.

If you ever notice in the future that the script isn't updating, please do let me know. I do not check on it as often as I should, these days. Normally any update should be synced to the repo in at most 30 minutes after first becoming available, so even if you see an update has not been uploaded for a few hours after being available in the dashboard, that's cause enough for concern.

Hi, I observed that these data haven't been updated for 5 days now. Could you help check? Thank you!

dancarmoz commented 2 years ago

Oh no :( Thanks for letting me know. Unfortunately I am abroad and don't have easy access to a computer that can run the updating script. I'll be able to restart updates in a couple of days.

I hope to be able to cover some past gaps in the future with others' backups. And perhaps I should also look for more robust solutions than running the script locally.

RuijingZ commented 1 year ago

Oh no :( Thanks for letting me know. Unfortunately I am abroad and don't have easy access to a computer that can run the updating script. I'll be able to restart updates in a couple of days.

I hope to be able to cover some past gaps in the future with others' backups. And perhaps I should also look for more robust solutions than running the script locally.

Hi, thanks for maintaining these datasets! They haven't been updated for several days again. Is there anything that need to be fixed?

dancarmoz commented 1 year ago

Thanks again for letting me know, I've been abroad so it's taken me a few days until I managed to take a look. Unfortunately, it seems something quite substantial has changed in the API this time, and the url that so far used to work has stopped accepting SSL connection from my script (and it also appears to no longer be used internally, or at least mostly so...). So this fix might be quite non-trivial and time consuming, if possible.

dancarmoz commented 1 year ago

Well, turns out the solution was easier than expected: the problem was that the middle certificate in the server's certificate chain was not provided to the user, and not present in the local CA store that python's certifi package uses. The solution was just to install it manually in that certificate manager. This would probably be necessary for anyone running this script on their own machines should they run into the same issue.

Here is a good description of the problem and how to fix it: https://stackoverflow.com/questions/27068163/python-requests-not-handling-missing-intermediate-certificate-only-from-one-mach/66111417#66111417

And this is the missing certificate that needs to be appended to certifi's cacert.pem file (NOTE: it is much more sensible to get this directly from the API's site, as I did, then to copy and paste some random cerificate someone put up on github :D )

dancarmoz / israel_moh_covid_dashboard_data

Data have not been updated for 4 Days #23