efficient / libcuckoo

A high-performance, concurrent hash table
Other
1.62k stars 275 forks source link

Why not providing the concurrent iterating without blocking other threads? #121

Closed fesun closed 5 years ago

fesun commented 5 years ago

Libcuckoo only provides the iterators for locked table which blocks concurrent access. Is it possible to add the non-100% accurate iterating like others like tbb concurrent_hash_map?

One possible implementation: Take locks one by one and iterates all buckets protected by this lock, just like rehash.

manugoyal commented 5 years ago

Hi @fesun, I suppose it's possible, though it was a deliberate design decision to avoid such iterators, precisely because they are invalidated upon any concurrent operation. TBB itself has the following warning about its iterators:

Do not call concurrent operations, including count and find while iterating the table. Use concurrent_unordered_map if concurrent traversal and insertion are required.

I would be weary of using these iterators as even a non-100% accurate iterator, because TBB doesn't seem to make any guarantees about the validity of the memory referenced by the iterators while other concurrent operations are occurring.

So we decided to avoid the possibility of such invalidation scenarios by only allowing iterators over a locked table.