rdatools / rdabase

Redistricting analytics data & shared code
MIT License
0 stars 0 forks source link

Reduce the size of ensembles on disk #113

Open alecramsay opened 1 month ago

alecramsay commented 1 month ago

... especially but not only if we need to increase the number of plans collected.

There are 3 levels to this:

Then reverse the process when reading an ensemble from disk.

This would allow us to store much larger ensembles in GitHub w/o having to resort to LFS (which can be very tricky, in my experience) and reduce transfer times (back from the cluster).

alecramsay commented 1 month ago

The maximum number of precincts is CA with 25,607. The maximum number of districts is the PA state house with 203.