project-seldon / intake

Ideas, bug reports, questions and any other type of feedback.
https://projectseldon.org
1 stars 0 forks source link

Explore / 2022 & 2023 Population Data #46

Closed skipjack closed 12 months ago

skipjack commented 1 year ago

Potential sources...

This would help stabilize any other charts (e.g. worldwide politics) that use population data as well.

skipjack commented 1 year ago

Made some good progress starting to hit the UN API but unfortunately it started 500-ing tonight right as I was about to wrap things up. Hoping it will be back up in the morning and more stable from here on out, but if not we can explore explore a "fall back to previous scrape approach" as we discussed briefly in the past. Even if their API is a bit unstable it seems like it may contain some much better (and broader) data points than what we're getting from other sources like world bank.

skipjack commented 1 year ago

You can see their own site is down likely due to the same underlying issue...

Image

Image

skipjack commented 1 year ago

I'm finding it difficult to grok this documentation but the sample links they provide there do seem to work (they return XML but I think we can change that via a request header)...

https://data.un.org/ws/rest/data/DF_UNData_UNFCC/A.EN_ATM_CO2E_XLULUCF.USA+CAN.Gg_CO2?startPeriod=1995&endPeriod=2023

That being said, I'm not seeing a listing of indicators to their corresponding ids anywhere and at least for the one above the data stops at 2023. So it's a little hard to tell which API contains their most recent data, certainly it would seem like we should be querying the same place as their live portal.

skipjack commented 1 year ago

Still down, but maybe these excel files are another option...

https://population.un.org/wpp/Download/Standard/MostUsed/