search

b-r-u / osmpbf

A Rust library for reading the OpenStreetMap PBF file format (*.osm.pbf).
Apache License 2.0
122 stars 19 forks source link

Add lower level API to more control over parallel execution #37

Open bjornharrtell opened 2 years ago

bjornharrtell commented 2 years ago

Would be good to be able to get a ParallelIterator directly for cases where more control is needed.

b-r-u commented 2 years ago

I agree and I will try to see if it is feasible.

ThomasCartier commented 6 months ago

Hi, just used the code and I concur, when reading the nodes of a huge pbf file, it would be nice to have a number of parallel reads spread by the number of cores so that we can accumulate data in a per-thread container, then at the end use a mutex to merge them. Instead of locking a common container for every node insertion ( kills the performance).

Do you think it's achievable?

Thanks, great lib btw