owid / etl

A compute graph for loading and transforming OWID's data
https://docs.owid.io/projects/etl
MIT License
86 stars 23 forks source link

🐛 education: EdStats fixing data provider's bug in formal education data #3600

Closed veronikasamborska1994 closed 2 days ago

owidbot commented 2 days ago
Quick links (staging server): Site Dev Site Preview Admin Wizard Docs

Login: ssh owid@staging-site-formal-ed-bug

chart-diff: ✅ No charts for review.
data-diff: ❌ Found differences ```diff = Dataset garden/education/2023-08-14/oecd_education = Table oecd_education ~ Column no_formal_education (changed data) ~ Changed values: 1688 / 7000 (24.11%) country year no_formal_education - no_formal_education + Aruba 2035 0.03 3.0 Guinea 2040 0.39 39.0 India 2100 0.02 2.0 Luxembourg 2045 0.02 2.0 North Macedonia 2020 0.02 2.0 ~ Column population_with_basic_education (changed data) ~ Changed values: 1688 / 7000 (24.11%) country year population_with_basic_education - population_with_basic_education + Aruba 2035 99.970001 97.0 Guinea 2040 99.610001 61.0 India 2100 99.980003 98.0 Luxembourg 2045 99.980003 98.0 North Macedonia 2020 99.980003 98.0 = Dataset garden/education/2023-12-15/wittgenstein_center = Table wittgenstein_center_data ~ Column no_formal_education (changed data) ~ Changed values: 281 / 7022 (4.00%) country year no_formal_education - no_formal_education + Azerbaijan 2010 0.01 1.0 Cote d'Ivoire 2010 0.46 46.0 Ethiopia 2015 0.43 43.0 Iran 2015 0.15 15.000001 Sao Tome and Principe 2010 0.09 9.0 ~ Column population_with_basic_education (changed data) ~ Changed values: 281 / 7022 (4.00%) country year population_with_basic_education - population_with_basic_education + Azerbaijan 2010 99.989998 99.0 Cote d'Ivoire 2010 99.540001 54.0 Ethiopia 2015 99.57 57.0 Iran 2015 99.849998 85.0 Sao Tome and Principe 2010 99.910004 91.0 = Dataset garden/wb/2024-11-04/edstats = Table edstats ~ Column wittgenstein_projection__percentage_of_the_population_age_15plus_by_highest_level_of_educational_attainment__no_education__total (changed data) ~ Changed values: 1688 / 13095 (12.89%) country year wittgenstein_projection__percentage_of_the_population_age_15plus_by_highest_level_of_educational_attainment__no_education__total - wittgenstein_projection__percentage_of_the_population_age_15plus_by_highest_level_of_educational_attainment__no_education__total + Aruba 2035 0.03 3.0 Guinea 2040 0.39 39.0 India 2100 0.02 2.0 Luxembourg 2045 0.02 2.0 North Macedonia 2020 0.02 2.0 Legend: +New ~Modified -Removed =Identical Details Hint: Run this locally with etl diff REMOTE data/ --include yourdataset --verbose --snippet ``` Automatically updated datasets matching _weekly_wildfires|excess_mortality|covid|fluid|flunet|country_profile|garden/ihme_gbd/2019/gbd_risk_ are not included

Edited: 2024-11-23 21:27:10 UTC Execution time: 15.90 seconds