quantifyearth / yirgacheffe

A set of GDAL wrappers around raster and vector datasets to make computation easier
ISC License
6 stars 3 forks source link

Parallel save #4

Closed mdales closed 3 months ago

mdales commented 3 months ago

This adds a new experimental feature, which is parallel save. Compared to regular save, which breaks work into chunks to prevent memory overloading, parallel save will throw those chunks out to different CPU cores. In practices this doesn't give you as big a win as you might expect, due to collating and saving the work is still a serialised operation, and with large GeoTIFFs with LZW compression that becomes the bottleneck.