search

iati-data-access / data-backend

GNU Affero General Public License v3.0
1 stars 0 forks source link

Script to create the bulk download ZIP files #19

Closed simon-20 closed 10 months ago

odscjames commented 10 months ago

If someone deletes or reclassifies data such that a country or code previously had data but now doesn't, will the XLSX file be deleted from the disk? Or maybe a blank XLSX file be written?

If not there is a risk that we continue serving old data we should have deleted.

This isn't a comment on the script, this is a pre existing problem, but if it is an issue then unfortunately the script makes it worse and we need to catch that.

simon-20 commented 10 months ago

From what I can work out, this may be possible if all of the items for the region or country in question are deleted. It seems that the code doesn't write out empty Excel files where there is no data. Realistically, this means it is a possibility for those country and regions which only contain data from one activity; if that activity was then deleted, then the file wouldn't be deleted, or emptied.

I'll create an issue for this; it belongs on the repo of the iati-flattener library, because that's the relevant code.

odscjames commented 10 months ago

https://github.com/iati-data-access/iati-flattener/issues/18