dib-lab / kProcessor

kProcessor: kmers processing framework.
https://kprocessor.readthedocs.io
BSD 3-Clause "New" or "Revised" License
11 stars 1 forks source link

Problem in updating parallel_hashmap in v2 #98

Closed mr-eyes closed 2 years ago

mr-eyes commented 2 years ago

@shokrof The latest version of kmerDecoder has a recent version of the parallel_hashmap library. When I try to update it in the v2 branch, I got errors from this line

https://github.com/dib-lab/kProcessor/blob/871d32087367f8a885746e22af0bc0868c5c4492/src/algorithms.cpp#L640-L641

Snippet from the error log:

/home/mabuelanin/dib-dev/try_kp2/lib/kProcessor/src/algorithms.cpp:1143:82: warning: unused parameter ‘output’ [-Wunused-parameter]
 1143 |     void mergeIndexes(vector<kDataFrame *> &input, string tmpFolder, kDataFrame *output) {
      |                                                                      ~~~~~~~~~~~~^~~~~~
In file included from /home/mabuelanin/dib-dev/try_kp2/lib/kProcessor/ThirdParty/kmerDecoder/include/kmerDecoder.hpp:5,
                 from /home/mabuelanin/dib-dev/try_kp2/lib/kProcessor/include/kProcessor/kDataFrame.hpp:7,
                 from /home/mabuelanin/dib-dev/try_kp2/lib/kProcessor/include/kProcessor/algorithms.hpp:14,
                 from /home/mabuelanin/dib-dev/try_kp2/lib/kProcessor/src/algorithms.cpp:1:
/home/mabuelanin/dib-dev/try_kp2/lib/kProcessor/ThirdParty/kmerDecoder/lib/parallel-hashmap/parallel_hashmap/phmap.h: In instantiation of ‘bool phmap::priv::parallel_hash_map<N, RefSet, Mtx_, Policy, Hash, Eq, Alloc>::try_emplace_l(K&&, F&&, Args&& ...) [with K = long unsigned int&; F = kProcessor::parallelJoin(std::vector<std::__cxx11::basic_string<char> >&, std::vector<unsigned int>, uint64_t)::<lambda(uint32_t&)>; Args = {unsigned int&}; long unsigned int N = 4; RefSet = phmap::priv::raw_hash_set; Mtx_ = std::mutex; Policy = phmap::priv::FlatHashMapPolicy<long unsigned int, unsigned int>; Hash = std::hash<long unsigned int>; Eq = std::equal_to<long unsigned int>; Alloc = std::allocator<std::pair<const long unsigned int, long unsigned int> >]’:
/home/mabuelanin/dib-dev/try_kp2/lib/kProcessor/src/algorithms.cpp:644:36:   required from here
/home/mabuelanin/dib-dev/try_kp2/lib/kProcessor/ThirdParty/kmerDecoder/lib/parallel-hashmap/parallel_hashmap/phmap.h:3910:31: error: no match for call to ‘(kProcessor::parallelJoin(std::vector<std::__cxx11::basic_string<char> >&, std::vector<unsigned int>, uint64_t)::<lambda(uint32_t&)>) (phmap::priv::parallel_hash_map<4, phmap::priv::raw_hash_set, std::mutex, phmap::priv::FlatHashMapPolicy<long unsigned int, unsigned int>, std::hash<long unsigned int>, std::equal_to<long unsigned int>, std::allocator<std::pair<const long unsigned int, long unsigned int> > >::value_type&)’
 3910 |             std::forward<F>(f)(const_cast<value_type &>(*it)); // in case of the set, non "key" part of value_type can be changed
      |             ~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/mabuelanin/dib-dev/try_kp2/lib/kProcessor/src/algorithms.cpp:641:40: note: candidate: ‘kProcessor::parallelJoin(std::vector<std::__cxx11::basic_string<char> >&, std::vector<unsigned int>, uint64_t)::<lambda(uint32_t&)>’
  641 |                                        [&order](uint32_t& v) {
      |                                        ^
/home/mabuelanin/dib-dev/try_kp2/lib/kProcessor/src/algorithms.cpp:641:40: note:   no known conversion for argument 1 from ‘phmap::priv::parallel_hash_map<4, phmap::priv::raw_hash_set, std::mutex, phmap::priv::FlatHashMapPolicy<long unsigned int, unsigned int>, std::hash<long unsigned int>, std::equal_to<long unsigned int>, std::allocator<std::pair<const long unsigned int, long unsigned int> > >::value_type’ {aka ‘std::pair<const long unsigned int, unsigned int>’} to ‘uint32_t&’ {aka ‘unsigned int&’}
In file included from /home/mabuelanin/dib-dev/try_kp2/lib/kProcessor/ThirdParty/kmerDecoder/include/kmerDecoder.hpp:4,
                 from /home/mabuelanin/dib-dev/try_kp2/lib/kProcessor/include/kProcessor/kDataFrame.hpp:7,
                 from /home/mabuelanin/dib-dev/try_kp2/lib/kProcessor/include/kProcessor/algorithms.hpp:14,
                 from /home/mabuelanin/dib-dev/try_kp2/lib/kProcessor/src/algorithms.cpp:1:
/home/mabuelanin/dib-dev/try_kp2/lib/kProcessor/ThirdParty/kmerDecoder/include/kmerDecoder.hpp: At global scope:
mr-eyes commented 2 years ago

In the parallel_hashmap recent release, they significantly increased the phmap loading/saving time.

mr-eyes commented 2 years ago

Fixed in #99