owid / etl

A compute graph for loading and transforming OWID's data
https://docs.owid.io/projects/etl
MIT License
85 stars 23 forks source link

📊 hfd #3589

Closed lucasrodes closed 20 hours ago

lucasrodes commented 22 hours ago

possible alternative to https://github.com/owid/etl/pull/3567 in case passing CI/CD checks is a problem.

owidbot commented 22 hours ago
Quick links (staging server): Site Dev Site Preview Admin Wizard Docs

Login: ssh owid@staging-site-data-hfd

chart-diff: ✅ No charts for review.
data-diff: ❌ Found differences ```diff + Dataset garden/hmd/2024-11-19/hfd + + Table cohort + + Column cmab + + Column cmab40 + + Column sdcmab + + Column sdcmab40 + + Column ccf + + Column ccf40 + + Table period + + Column adjtfr + + Column cbr + + Column mab + + Column mab40 + + Column patfr + + Column tmab + + Column sdmab + + Column sdmab40 + + Column tfr + + Column tfr40 + + Column b = Dataset garden/homicide/2024-10-30/unodc = Table total ~ Dim country + + New values: 774 / 44526 (1.74%) year category sex age unit_of_measurement country 1993 Total Female Total Counts United Kingdom 2014 Total Female 20-24 Rate per 100,000 population United Kingdom 2014 Total Male Total Rate per 100,000 population United Kingdom 2019 Total Female Total Counts United Kingdom 2021 Total Male 60 and older Rate per 100,000 population United Kingdom ~ Dim year + + New values: 774 / 44526 (1.74%) country category sex age unit_of_measurement year United Kingdom Total Female Total Counts 1993 United Kingdom Total Female 20-24 Rate per 100,000 population 2014 United Kingdom Total Male Total Rate per 100,000 population 2014 United Kingdom Total Female Total Counts 2019 United Kingdom Total Male 60 and older Rate per 100,000 population 2021 ~ Dim category + + New values: 774 / 44526 (1.74%) country year sex age unit_of_measurement category United Kingdom 1993 Female Total Counts Total United Kingdom 2014 Female 20-24 Rate per 100,000 population Total United Kingdom 2014 Male Total Rate per 100,000 population Total United Kingdom 2019 Female Total Counts Total United Kingdom 2021 Male 60 and older Rate per 100,000 population Total ~ Dim sex + + New values: 774 / 44526 (1.74%) country year category age unit_of_measurement sex United Kingdom 1993 Total Total Counts Female United Kingdom 2014 Total 20-24 Rate per 100,000 population Female United Kingdom 2014 Total Total Rate per 100,000 population Male United Kingdom 2019 Total Total Counts Female United Kingdom 2021 Total 60 and older Rate per 100,000 population Male ~ Dim age + + New values: 774 / 44526 (1.74%) country year category sex unit_of_measurement age United Kingdom 1993 Total Female Counts Total United Kingdom 2014 Total Female Rate per 100,000 population 20-24 United Kingdom 2014 Total Male Rate per 100,000 population Total United Kingdom 2019 Total Female Counts Total United Kingdom 2021 Total Male Rate per 100,000 population 60 and older ~ Dim unit_of_measurement + + New values: 774 / 44526 (1.74%) country year category sex age unit_of_measurement United Kingdom 1993 Total Female Total Counts United Kingdom 2014 Total Female 20-24 Rate per 100,000 population United Kingdom 2014 Total Male Total Rate per 100,000 population United Kingdom 2019 Total Female Total Counts United Kingdom 2021 Total Male 60 and older Rate per 100,000 population ~ Column value (changed metadata, new data) + + - producer: Various sources + + title: Population + + description: |- + + Our World in Data builds and maintains a long-run dataset on population by country, region, and for the world, based on various sources. + + + + You can find more information on these sources and how our time series is constructed on this page: https://ourworldindata.org/population-sources + + citation_full: |- + + The long-run data on population is based on various sources, described on this page: https://ourworldindata.org/population-sources + + attribution: Population based on various sources (2023) + + attribution_short: Population + + url_main: https://ourworldindata.org/population-sources + + date_accessed: '2023-03-31' + + date_published: '2023-03-31' + + license: + + name: CC BY 4.0 + + licenses: + + - name: Creative Commons BY 4.0 + + url: https://docs.google.com/document/d/1-RmthhS2EPMK_HIpnPctcXpB0n7ADSWnXa5Hb3PxNq4/edit?usp=sharing + + - name: CC BY 3.0 + + url: https://dataportaal.pbl.nl/downloads/HYDE/HYDE3.2/readme_release_HYDE3.2.1.txt + + - name: CC BY 3.0 IGO + + url: http://creativecommons.org/licenses/by/3.0/igo/ + + processing_level: major + + description_processing: |- + + Values for the United Kingdom are calcuated by Our World in Data from UNODC data for England, Wales, Scotland and Northern Ireland. + + New values: 774 / 44526 (1.74%) country year category sex age unit_of_measurement value United Kingdom 1993 Total Female Total Counts 24.896551 United Kingdom 2014 Total Female 20-24 Rate per 100,000 population 0.001544 United Kingdom 2014 Total Male Total Rate per 100,000 population 0.089543 United Kingdom 2019 Total Female Total Counts 26.0 United Kingdom 2021 Total Male 60 and older Rate per 100,000 population 0.010404 = Table by_mechanisms ~ Dim country + + New values: 182 / 9100 (2.00%) year category sex age unit_of_measurement country 2008 unspecified means Total Total Counts United Kingdom 2010 firearms or explosives Total Total Rate per 100,000 population United Kingdom 2015 sharp or blunt object, including motor vehicles Total Total Counts United Kingdom 2016 without a weapon or by another mechanism Total Total Counts United Kingdom 2022 unspecified means Total Total Rate per 100,000 population United Kingdom ~ Dim year + + New values: 182 / 9100 (2.00%) country category sex age unit_of_measurement year United Kingdom unspecified means Total Total Counts 2008 United Kingdom firearms or explosives Total Total Rate per 100,000 population 2010 United Kingdom sharp or blunt object, including motor vehicles Total Total Counts 2015 United Kingdom without a weapon or by another mechanism Total Total Counts 2016 United Kingdom unspecified means Total Total Rate per 100,000 population 2022 ~ Dim category + + New values: 182 / 9100 (2.00%) country year sex age unit_of_measurement category United Kingdom 2008 Total Total Counts unspecified means United Kingdom 2010 Total Total Rate per 100,000 population firearms or explosives United Kingdom 2015 Total Total Counts sharp or blunt object, including motor vehicles United Kingdom 2016 Total Total Counts without a weapon or by another mechanism United Kingdom 2022 Total Total Rate per 100,000 population unspecified means ~ Dim sex + + New values: 182 / 9100 (2.00%) country year category age unit_of_measurement sex United Kingdom 2008 unspecified means Total Counts Total United Kingdom 2010 firearms or explosives Total Rate per 100,000 population Total United Kingdom 2015 sharp or blunt object, including motor vehicles Total Counts Total United Kingdom 2016 without a weapon or by another mechanism Total Counts Total United Kingdom 2022 unspecified means Total Rate per 100,000 population Total ~ Dim age + + New values: 182 / 9100 (2.00%) country year category sex unit_of_measurement age United Kingdom 2008 unspecified means Total Counts Total United Kingdom 2010 firearms or explosives Total Rate per 100,000 population Total United Kingdom 2015 sharp or blunt object, including motor vehicles Total Counts Total United Kingdom 2016 without a weapon or by another mechanism Total Counts Total United Kingdom 2022 unspecified means Total Rate per 100,000 population Total ~ Dim unit_of_measurement + + New values: 182 / 9100 (2.00%) country year category sex age unit_of_measurement United Kingdom 2008 unspecified means Total Total Counts United Kingdom 2010 firearms or explosives Total Total Rate per 100,000 population United Kingdom 2015 sharp or blunt object, including motor vehicles Total Total Counts United Kingdom 2016 without a weapon or by another mechanism Total Total Counts United Kingdom 2022 unspecified means Total Total Rate per 100,000 population ~ Column value (new data) + + New values: 182 / 9100 (2.00%) country year category sex age unit_of_measurement value United Kingdom 2008 unspecified means Total Total Counts 4.0 United Kingdom 2010 firearms or explosives Total Total Rate per 100,000 population 0.00478 United Kingdom 2015 sharp or blunt object, including motor vehicles Total Total Counts 47.0 United Kingdom 2016 without a weapon or by another mechanism Total Total Counts 26.0 United Kingdom 2022 unspecified means Total Total Rate per 100,000 population 0.004444 = Table by_situational_context = Table by_relationship_to_perpetrator ~ Dim country + + New values: 438 / 21798 (2.01%) year category sex age unit_of_measurement country 2010 Perpetrator is an intimate partner Female Total Counts United Kingdom 2016 Another known perpetrator Total Total Counts United Kingdom 2017 Perpetrator is a family member Male Total Rate per 100,000 population United Kingdom 2019 Perpetrator is a family member Female Total Counts United Kingdom 2020 Another known perpetrator Female Total Counts United Kingdom ~ Dim year + + New values: 438 / 21798 (2.01%) country category sex age unit_of_measurement year United Kingdom Perpetrator is an intimate partner Female Total Counts 2010 United Kingdom Another known perpetrator Total Total Counts 2016 United Kingdom Perpetrator is a family member Male Total Rate per 100,000 population 2017 United Kingdom Perpetrator is a family member Female Total Counts 2019 United Kingdom Another known perpetrator Female Total Counts 2020 ~ Dim category + + New values: 438 / 21798 (2.01%) country year sex age unit_of_measurement category United Kingdom 2010 Female Total Counts Perpetrator is an intimate partner United Kingdom 2016 Total Total Counts Another known perpetrator United Kingdom 2017 Male Total Rate per 100,000 population Perpetrator is a family member United Kingdom 2019 Female Total Counts Perpetrator is a family member United Kingdom 2020 Female Total Counts Another known perpetrator ~ Dim sex + + New values: 438 / 21798 (2.01%) country year category age unit_of_measurement sex United Kingdom 2010 Perpetrator is an intimate partner Total Counts Female United Kingdom 2016 Another known perpetrator Total Counts Total United Kingdom 2017 Perpetrator is a family member Total Rate per 100,000 population Male United Kingdom 2019 Perpetrator is a family member Total Counts Female United Kingdom 2020 Another known perpetrator Total Counts Female ~ Dim age + + New values: 438 / 21798 (2.01%) country year category sex unit_of_measurement age United Kingdom 2010 Perpetrator is an intimate partner Female Counts Total United Kingdom 2016 Another known perpetrator Total Counts Total United Kingdom 2017 Perpetrator is a family member Male Rate per 100,000 population Total United Kingdom 2019 Perpetrator is a family member Female Counts Total United Kingdom 2020 Another known perpetrator Female Counts Total ~ Dim unit_of_measurement + + New values: 438 / 21798 (2.01%) country year category sex age unit_of_measurement United Kingdom 2010 Perpetrator is an intimate partner Female Total Counts United Kingdom 2016 Another known perpetrator Total Total Counts United Kingdom 2017 Perpetrator is a family member Male Total Rate per 100,000 population United Kingdom 2019 Perpetrator is a family member Female Total Counts United Kingdom 2020 Another known perpetrator Female Total Counts ~ Column value (changed metadata, new data) + + - producer: Various sources + + title: Population + + description: |- + + Our World in Data builds and maintains a long-run dataset on population by country, region, and for the world, based on various sources. + + + + You can find more information on these sources and how our time series is constructed on this page: https://ourworldindata.org/population-sources + + citation_full: |- + + The long-run data on population is based on various sources, described on this page: https://ourworldindata.org/population-sources + + attribution: Population based on various sources (2023) + + attribution_short: Population + + url_main: https://ourworldindata.org/population-sources + + date_accessed: '2023-03-31' + + date_published: '2023-03-31' + + license: + + name: CC BY 4.0 + + licenses: + + - name: Creative Commons BY 4.0 + + url: https://docs.google.com/document/d/1-RmthhS2EPMK_HIpnPctcXpB0n7ADSWnXa5Hb3PxNq4/edit?usp=sharing + + - name: CC BY 3.0 + + url: https://dataportaal.pbl.nl/downloads/HYDE/HYDE3.2/readme_release_HYDE3.2.1.txt + + - name: CC BY 3.0 IGO + + url: http://creativecommons.org/licenses/by/3.0/igo/ + + processing_level: major + + description_processing: |- + + Values for the United Kingdom are calcuated by Our World in Data from UNODC data for England, Wales, Scotland and Northern Ireland. + + New values: 438 / 21798 (2.01%) country year category sex age unit_of_measurement value United Kingdom 2010 Perpetrator is an intimate partner Female Total Counts 17.0 United Kingdom 2016 Another known perpetrator Total Total Counts 28.0 United Kingdom 2017 Perpetrator is a family member Male Total Rate per 100,000 population 0.004541 United Kingdom 2019 Perpetrator is a family member Female Total Counts 5.0 United Kingdom 2020 Another known perpetrator Female Total Counts 5.0 Legend: +New ~Modified -Removed =Identical Details Hint: Run this locally with etl diff REMOTE data/ --include yourdataset --verbose --snippet ``` Automatically updated datasets matching _weekly_wildfires|excess_mortality|covid|fluid|flunet|country_profile|garden/ihme_gbd/2019/gbd_risk_ are not included

Edited: 2024-11-21 18:07:30 UTC Execution time: 320.11 seconds

lucasrodes commented 20 hours ago

closing because https://github.com/owid/etl/pull/3567 seems to be operational