search

worldbank / DECAT_Space2Stats

https://worldbank.github.io/DECAT_Space2Stats/
Other
1 stars 2 forks source link

Update combined_parquet data for ingestion #17

Open zacharyDez opened 3 months ago

zacharyDez commented 3 months ago

@bpstewar planned on modifying the data pipeline used to generate the combined dataset ingested into our database.

The two scripts I added for creating the combined_population.parquet are:

  1. https://github.com/worldbank/DECAT_Space2Stats/blob/review/population-parquet/src/combine_population_data.py: to pivot the data and combine into a single file
  2. https://github.com/worldbank/DECAT_Space2Stats/blob/review/population-parquet/src/add_total_pop.py: to add total counts

We should also close https://github.com/worldbank/DECAT_Space2Stats/pull/7 as part of this effort.