search

aiddata / geo-datasets

Scripts for preparing datasets in GeoQuery
http://geoquery.org
MIT License
20 stars 11 forks source link

worldPop_age_sex_structures #147

Open sgoodm opened 1 year ago

sgoodm commented 1 year ago
sgoodm commented 1 year ago

This is ready for a full run, but expecting it will be 1-1.5TB total and take 24-48 hours using current job specs.

Prior to running, should also confirm final format for displaying these in GeoQuery in case it impacts file structure. (As it is, every age/sex combination would be treated as its own GeoQuery dataset. This would add ~40 datasets to GeoQuery which only has 60 something total. Concern is flooding the interface given our filtering / search isn't amazing)

We are also running low on space on aiddata10 and in the middle of transferring data to the new nova filesystem. Probably will just hold off on running this for a bit.