CSVs with indicators of all Cantons combined haven't been generated after 2022-04-12

openZH / covid_19

COVID19 case numbers of Cantons of Switzerland and Principality of Liechtenstein (FL). The data is updated at best once a day (times of collection and update may vary). Start with the README.

https://www.zh.ch/de/gesundheit/coronavirus/zahlen-fakten-covid-19.zhweb-noredirect.zhweb-cache.html?keywords=covid19&keyword=covid19#/

Creative Commons Attribution 4.0 International

425 stars 177 forks source link

CSVs with indicators of all Cantons combined haven't been generated after 2022-04-12 #1961

Closed andreasamsler closed 2 years ago

andreasamsler commented 2 years ago

Affected:

Measures taken:

manually run Validate CSV #8120
analysing issues

andreasamsler commented 2 years ago

Issue:

SO: https://github.com/openZH/covid_19/blob/master/fallzahlen_kanton_total_csv_v2/COVID19_Fallzahlen_Kanton_SO_total.csv

Analysis:

Error in Row 531: cumulative field ncumul_conf: 108915 < 109485

Measures taken:

Invalid value in Row 531 manually deleted
manually run Validate CSV #8123

andreasamsler commented 2 years ago

Measures taken:

manually run Run scrapers #44213

Results:

7 errors: AI, FL, AG, GE, UR, TI, VD

Next GitHub Action

merge_csvs

Results:

Blocked: Commit and push to repo

Analysis:

global GitHub issue?

andreasamsler commented 2 years ago

According to

there were issues with GitHub Actions on April 12 and 13, which - according to the info above - should be resolved by now.

Measures:

assess with tech squad, how to proceed.

andreasamsler commented 2 years ago

While the reason for this issue is not yet found, the ressource for ZH only was manually updated:

https://github.com/openZH/covid_19/blob/master/fallzahlen_kanton_zh/COVID19_Fallzahlen_Kanton_ZH_total.csv

Commit details: https://github.com/openZH/covid_19/commit/b0088074cc4158bff748e3ed54b7ccb9b58e1e35

metaodi commented 2 years ago

Will have a look soon-ish

metaodi commented 2 years ago

@andreasamsler okay the most urgent problem is fixed now. The scraper can finally commit again and the combined CSV is generated and commited as well.

As you noticed, there are still some scrapers with errors, I will check them another time and see which of those can be fixed.

amslera commented 2 years ago

Many thanks @metaodi !

Further note: Our initiative to adapt individual cantonal scrapers to ongoing changes to cantonal websites remains limited. Especially in cases where cantons do not provide data in a structured form.