search

versatiles-org / versatiles-rs

VersaTiles - A toolbox for converting, checking and serving map tiles in various formats.
https://versatiles.org
MIT License
57 stars 2 forks source link

Use a worker pool for (re)compression (especially for Brotli compression) #41

Closed MichaelKreil closed 1 year ago

MichaelKreil commented 1 year ago

When we want to recompress 360 mio. tiles we need to use all cores and an efficient worker pool. Compression is done here: https://github.com/versatiles-org/versatiles-rs/blob/main/src/shared/compress.rs

MichaelKreil commented 1 year ago

Worker and memory pools are not the right way to do it in Rust: https://kerkour.com/rust-worker-pool Instead: Use Iterators