search

EAPD-DRB / OG-ZAF

Overlapping-generations macroeconomic model for evaluating fiscal policy in South Africa
https://eapd-drb.github.io/OG-ZAF/
Creative Commons Zero v1.0 Universal
9 stars 10 forks source link

Labor share data from UN Statistics is returning a server error #46

Closed SeaCelo closed 11 months ago

SeaCelo commented 11 months ago

Labor share data (SDG indicator 10.4.1) is sourced from UN Statistics portal. Variable 'un_data_a' in 'macro_params.py'

This API is returning an error: 404 error: https://data.un.org/ws/rest/data/IAEG-SDGs,DF_SDG_GLH,1.10/..SL_EMP_GTOTL.710...........?startPeriod=2021&endPeriod=2021&format=csv

500 error using this method: https://unstats.un.org/sdgs/UNSDGAPIV5/swagger/index.html https://unstats.un.org/sdgs/UNSDGAPIV5/v1/sdg/Indicator/Data?indicator=SL_EMP_GTOTL&areaCode=710&timePeriod=2020

SeaCelo commented 11 months ago

@jdebacker:
It turns out that the "data flow" changes almost every quarter when there are updated to the DSD. The current DSD is version 1.15 (from 1.10).

Currently working link: https://data.un.org/ws/rest/data/IAEG-SDGs,DF_SDG_GLH,1.15/..SL_EMP_GTOTL.710...........?startPeriod=2020&endPeriod=2020&format=csv

I will work on a method to get or compute the correct value for the future.

edit: link to the documentation: https://data.un.org/Host.aspx?Content=API

SeaCelo commented 11 months ago

The fix is to skip the dataflow version in the url. This will always get the latest available.

https://data.un.org/ws/rest/data/IAEG-SDGs,DF_SDG_GLH,/..SL_EMP_GTOTL.710...........?startPeriod=2020&endPeriod=2020&format=csv

I will make the changes in a future PR

@jdebacker

jdebacker commented 11 months ago

Great find - thanks @SeaCelo!

SeaCelo commented 11 months ago

@jdebacker I think we need another check. The current data is only available for 2020. I think you are using 2021, which returns "no data" for this call. Do we need some logic to test or throw a message to the user?

jdebacker commented 11 months ago

The current data is only available for 2020. I think you are using 2021, which returns "no data" for this call. Do we need some logic to test or throw a message to the user?

Thanks for noticing this Marcelo. In my actual run of run_og_zaf.py the target URL is

https://data.un.org/ws/rest/data/IAEG-SDGs,DF_SDG_GLH/..SL_EMP_GTOTL.710...........?startPeriod=1947&endPeriod=2023&format=csv

which does work. It's pulling the start and end year from those set above used for the FRED data. But it seems to work at long as the start year is prior to 2021.

SeaCelo commented 11 months ago

@jdebacker You are correct. The start is 1947 so it will take the latest available before today. No more issues and this can close with #45