search

openclimatedata / national-inventory-submisions

Downloader for UNFCCC National Inventory Submissions
7 stars 3 forks source link

Download BUR submissions #4

Closed rgieseke closed 5 years ago

rgieseke commented 5 years ago

Automate BUR download should work in similar way as CRF data

Two tasks

make data-bur
make download-bur

URL: https://unfccc.int/BURs

rgieseke commented 5 years ago

@JGuetschow pushed a first attempt, the kind colum is either BUR{1,2,3} or NIR or NC if it's not easy to parse otherwise. Let me know what you think or need ...

JGuetschow commented 5 years ago

Collecting the links (make-bur) works fine and the resulting csv looks good. The actual download is not implemented yet, right? I can't remember if you do that or if I said I would do that to sort downloads into the folders I have for the BUR reading.

rgieseke commented 5 years ago

Not yet implemented ... wanted to check with you if you have special ideas for doing that ...

JGuetschow commented 5 years ago

Currently I have a folder for each BUR round ("first_BUR_files" for BUR1, "second_BUR_files" for BUR2, "third_BUR_files" for BUR3). In each of the folders there is a folder for each country that has submitted a BUR in that round (so e.g. BUR1/Argentina) which contains all downloaded files (BUR pdf, NIR pdf, etc) and also all data read from the BUR (csv and xls output from tabula and similar tools). But the structure could also be changed in the BUR repository, or we use a script that copies the downloaded data into the right folders (or fork the BUR code from this repo into the BUR repo).

JGuetschow commented 5 years ago

But as not too many BURs are submitted, I think copying into the right folder for analysis by hand is also completely feasible. In that case it would just be important, that the additional files (NIR etc) can be associated to their respective BUR.

rgieseke commented 5 years ago

For the time being a simple download is probably ok then, i'll add that later ... the diff of the CSV will highlight added files which can be copied into your structure.

JGuetschow commented 5 years ago

Sounds good to me

rgieseke commented 5 years ago

I've added a basic download script.

Could be improved by creating subdirs by country or making the filenames nicer e.g. by prefixing them.

One can of course use the CSV data to copy from filenames into another structure as well.