search

greg7mdp / parallel-hashmap

A family of header-only, very fast and memory-friendly hashmap and btree containers.
https://greg7mdp.github.io/parallel-hashmap/
Apache License 2.0
2.53k stars 239 forks source link

error with node_hash_map<uint32_t, std::vector<uint32_t>> #203

Closed HatakeKakaxi closed 1 year ago

HatakeKakaxi commented 1 year ago

I'm trying to use phmap::node_hash_map<uint32_t, std::vector> for concurrency. In my case, multi threads are trying to write the hash map, each thread performs like this: seed_index_pos[index].emplace_back(1); However, I got an error like this: Assertion failed: (seq.getindex() < capacity_ && "full table!"), function find_first_non_full, file phmap.h, line 2176. Can anyone help me?

HatakeKakaxi commented 1 year ago

When I try to use phmap::parallel_node_hash_map<uint32_t, std::vector>, I got nearly the same error.

greg7mdp commented 1 year ago

Hi @HatakeKakaxi , when using a parallel_node_hash_map from multiple threads, it is not safe to use iterators, or references returned by operator[](). If you provide a full example of what you are doing, I can suggest how to use extended APIs to solve your issue.

HatakeKakaxi commented 1 year ago

Hi @HatakeKakaxi , when using a parallel_node_hash_map from multiple threads, it is not safe to use iterators, or references returned by operator[](). If you provide a full example of what you are doing, I can suggest how to use extended APIs to solve your issue.

Thank you, @greg7mdp. Here is my example. I defined a phmap::parallel_node_hash_map<uint32_t, std::vector<uint32_t>> seed_index_pos in the main thread, and modify this parallel_node_hash_map in other threads. I use a boost::thread_pool with 8 threads to modify this hash map. The modication is just like this: seed_index_pos[index].emplace_back(1); Did you mean that it's not safe to use the operator[]() ? Which APIs should I use ?

greg7mdp commented 1 year ago

Did you mean that it's not safe to use the operator[]() ?

This is correct, it is not safe in a multithreaded context.

Which APIs should I use ?

You should most likely use lazy_emplace_l. Please see mt_word_counter.cc example in the examples directory.