owid / etl

A compute graph for loading and transforming OWID's data
https://docs.owid.io/projects/etl
MIT License
78 stars 21 forks source link

📊 h5n1: automate update #3163

Closed lucasrodes closed 1 month ago

lucasrodes commented 1 month ago

Fixes https://github.com/owid/owid-issues/issues/1623

I've manually updated the data.

The data is provided, temporary, as a blob file, and can't be accessed from python.

owidbot commented 1 month ago
Quick links (staging server): Site Admin Wizard

Login: ssh owid@staging-site-h5n1-automate-update

chart-diff: ✅ No charts for review.
data-diff: ❌ Found differences ```diff = Dataset garden/who/latest/avian_influenza_ah5n1 = Table avian_influenza_ah5n1_year ~ Column avian_cases_year (changed metadata, changed data) - - url_main: https://www.cdc.gov/flu/avianflu/chart-epi-curve-ah5n1.html + + url_main: https://www.cdc.gov/bird-flu/php/avian-flu-summary/chart-epi-curve-ah5n1.html ? +++++ ++++ + ++++++++ ~ Changed values: 6 / 868 (0.69%) country year avian_cases_year - avian_cases_year + Asia 2024 6 10 Cambodia 2024 5 7 China 2024 0 2 North America 2024 1 13 World 2024 8 24 = Table avian_influenza_ah5n1_month ~ Dim country + + New values: 62 / 10261 (0.60%) date country 2024-06-01 Iraq 2024-06-01 Laos 2024-07-01 Laos 2024-06-01 Spain 2024-07-01 Thailand ~ Dim date + + New values: 62 / 10261 (0.60%) country date Iraq 2024-06-01 Laos 2024-06-01 Laos 2024-07-01 Spain 2024-06-01 Thailand 2024-07-01 ~ Column avian_cases_month (changed metadata, new data, changed data) - - url_main: https://www.cdc.gov/flu/avianflu/chart-epi-curve-ah5n1.html + + url_main: https://www.cdc.gov/bird-flu/php/avian-flu-summary/chart-epi-curve-ah5n1.html ? +++++ ++++ + ++++++++ + + New values: 62 / 10261 (0.60%) country date avian_cases_month Iraq 2024-06-01 0 Laos 2024-06-01 0 Laos 2024-07-01 0 Spain 2024-06-01 0 Thailand 2024-07-01 0 ~ Changed values: 3 / 10261 (0.03%) country date avian_cases_month - avian_cases_month + North America 2024-05-01 1.0 2 United States 2024-05-01 1.0 2 World 2024-05-01 2.0 3 Legend: +New ~Modified -Removed =Identical Details Hint: Run this locally with etl diff REMOTE data/ --include yourdataset --verbose --snippet ``` Automatically updated datasets matching _weekly_wildfires|excess_mortality|covid|fluid|flunet|country_profile|garden/ihme_gbd/2019/gbd_risk_ are not included

Edited: 2024-08-20 23:40:32 UTC Execution time: 13.23 seconds