search

openclimatedata / unfccc-detailed-data-by-party

UNFCCC Emissions data from the Detailed Data By Party interface
7 stars 4 forks source link

Last year not shown #9

Closed cslovell closed 5 years ago

cslovell commented 5 years ago

The last year of the data (2016, in this case) is not shown in the final csv file.

The API returns 2016 and the year is present in the json files but not in the final csv file.

https://github.com/openclimatedata/unfccc-detailed-data-by-party/blob/8927160de5917e2660e371c62a72cbd1b33edb08/scripts/process.py#L40

I guess at one point the API call returned "Last Inventory Year (2016)" and index[indexed_keys[-1]][-5:-1] transformed this to simply "2016". Now it seems the API returns "2016" and this operation transforms it to "201". In my local version I've commented out this line and everything works fine. Not sure if this behavior can be verified and changed in the source code here?

Also later in the file there is this line: 

ordered = [
            "Party",
            "Parent Category",
            "Category",
            "Gas",
            "Unit",
            "Base year",
        ] + list(filtered.columns[5:-1])

removing -1 from the last line seems to fix this.

rgieseke commented 5 years ago

Thanks for the report! I'll look into it.

rgieseke commented 5 years ago

I can confirm the "Last Inventory" problem, thanks again!

Not sure about the second point, I think this is required for not duplicating "Base year" - i'll check that tomorrow, it's getting late here.

rgieseke commented 5 years ago

I think the actual issue might be that the "Last inventory" year is now 2017 and that the script wasn't updated to include this change. I've commited a fix (https://github.com/openclimatedata/unfccc-detailed-data-by-party/commit/7335c1564f93b4dfbb545b0139d4cf7f67390e1f) to always use up-to-date year mappings but this needs more testing.

rgieseke commented 5 years ago

@cslovell Can you give this a try? This should now work again (and hopefully with future updates as well) and the latest commits on master should include 2017 as well.

@JGuetschow Can you try reading this into your workflow as well if you get the chance?

rgieseke commented 5 years ago

I tagged a new release with this download, please open an issue when encountering any further issues!

https://github.com/openclimatedata/unfccc-detailed-data-by-party/releases/tag/2019-08-01