greg7mdp
commented
10 months ago Hi @kanonka , I already mentioned the right way to implement this in issue #119, Here it is (BTW -1 will never be inserted so you don't need the loop at the end):
class Test
{
StrIntParallelMap m_stringsMap;
volatile long m_curIdx;
public:
int add(const SimpleString& str)
{
int newIndex = -1;
m_stringsMap.lazy_emplace_l(
str,
[&](Map::value_type& p) {
newIndex = p.second;
}, // called only when key was already present
[&](const Map::constructor& ctor) // construct value_type in place when key not present
{
newIndex = InterlockedIncrement(&m_curIdx);
ctor(str, newIndex);
});
return newIndex;
}
};Please let me know how this performs.
kanonka
Hi, I have a specific case of data when parallel hash map is performing quite slowly.
Type declaration:
typedef std::string SimpleString;
class srwlock { SRWLOCK _lock; public: srwlock() { InitializeSRWLock(&_lock); } void lock() { AcquireSRWLockExclusive(&_lock); } void unlock() { ReleaseSRWLockExclusive(&_lock); }
};
typedef phmap::parallel_flat_hash_map<SimpleString, int, phmap::priv::hash_default_hash,
phmap::priv::hash_default_eq,
std::allocator<std::pair<const SimpleString, int>>, 8, srwlock> StrIntParallelMap;
Class declaration: class Test { StrIntParallelMap m_stringsMap; volatile long m_curIdx; public: int add(const SimpleString& str) { int newIndex = -1;
}
This works fine on most datasets. But one is weird: There are about 240 million string values that I'm parsing, and I call "add" for each value. About 239 millions of them is the very same string of about ~150 chars. Others are pretty much random, but shorter. So, when calling "add(value)" in parallel_for loop, my CPU load drops to ~19%, and the process takes about 233 seconds. If I replace StrIntParallelMap with concurrency::concurrent_unordered_map<SimpleString, int>, no CPU load drop, and process is done much faster in ~17 seconds.
Any idea?