Previously, each worker thread tried to eagerly insert their object into the index, blocking if another thread had the lock.
The index update operation can be expensive -- it potentially has to rebalance the map, and potentially has to grow + copy the value's vector.
But I think no one reads the index until the output stage, so the workers could just add their updates to a queue and move on with their other work.
That's what they now do. If, after adding the object, the queue is looking large, one worker will self-nominate to flush the queue into the map. Two queues are maintained, so that other workers aren't blocked while this flush is happening.
I promise I'm running out of PRs. :)
Previously, each worker thread tried to eagerly insert their object into the index, blocking if another thread had the lock.
The index update operation can be expensive -- it potentially has to rebalance the
map
, and potentially has to grow + copy the value'svector
.But I think no one reads the index until the output stage, so the workers could just add their updates to a queue and move on with their other work.
That's what they now do. If, after adding the object, the queue is looking large, one worker will self-nominate to flush the queue into the map. Two queues are maintained, so that other workers aren't blocked while this flush is happening.
refactor_geometries
branch for reading GB:This branch: