greg7mdp / parallel-hashmap

A family of header-only, very fast and memory-friendly hashmap and btree containers.
https://greg7mdp.github.io/parallel-hashmap/
Apache License 2.0
2.56k stars 240 forks source link

gcc 12.1 btree compiler issue #156

Closed jmbnyc closed 2 years ago

jmbnyc commented 2 years ago

Wondering if this makes any sense to you: gcc --version gcc (GCC) 12.1.0

gcc -B/usr/lib/x86_64-linux-gnu -fpermissive -std=c++11 -fopenmp -g -Wno-unused-local-typedefs -Wno-deprecated-declarations -faligned-new -O3 -DNDEBUG -Wall -msse4.2 -mpclmul -I/home/jmb/gfs/oss/parallel_hashmap/1.35/include -c btree-compile.cpp

In file included from btree-compile.cpp:1: In member function ‘void phmap::priv::btree_node::set_child(size_type, phmap::priv::btree_node) [with Params = phmap::priv::map_params<long unsigned int, map_data, phmap::Less, std::allocator<std::pair<const long unsigned int, map_data> >, 256, false>]’, inlined from ‘void phmap::priv::btree_node::init_child(int, phmap::priv::btree_node) [with Params = phmap::priv::map_params<long unsigned int, map_data, phmap::Less, std::allocator<std::pair<const long unsigned int, map_data> >, 256, false>]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/btree.h:1225:22, inlined from ‘void phmap::priv::btree_node::swap(phmap::priv::btree_node, allocator_type) [with Params = phmap::priv::map_params<long unsigned int, map_data, phmap::Less, std::allocator<std::pair<const long unsigned int, map_data> >, 256, false>]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/btree.h:2408:36, inlined from ‘phmap::priv::btree::iterator phmap::priv::btree::internal_emplace(iterator, Args&& ...) [with Args = {std::pair<const long unsigned int, map_data>}; Params = phmap::priv::map_params<long unsigned int, map_data, phmap::Less, std::allocator<std::pair<const long unsigned int, map_data> >, 256, false>]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/btree.h:3118:32, inlined from ‘std::pair<phmap::priv::btree_iterator<phmap::priv::btree_node, typename Params::reference, typename Params::pointer>, bool> phmap::priv::btree::insert_unique(const key_type&, Args&& ...) [with Args = {std::pair<const long unsigned int, map_data>}; Params = phmap::priv::map_params<long unsigned int, map_data, phmap::Less, std::allocator<std::pair<const long unsigned int, map_data> >, 256, false>]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/btree.h:2551:33, inlined from ‘std::pair<typename Tree::iterator, bool> phmap::priv::btree_set_container::insert(value_type&&) [with Tree = phmap::priv::btree<phmap::priv::map_params<long unsigned int, map_data, phmap::Less, std::allocator<std::pair<const long unsigned int, map_data> >, 256, false> >]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/btree.h:3485:45, inlined from ‘int main(int, char)’ at btree-compile.cpp:22:17: /home/jmb/gfs/oss/parallel_hashmap/1.35/include/btree.h:1221:30: warning: array subscript [33, 287] is outside array bounds of ‘phmap::priv::Allocate<8, std::allocator<std::pair<const long unsigned int, map_data> > >(std::allocator<std::pair<const long unsigned int, map_data> >, size_t)::M [32]’ [-Warray-bounds] 1221 | mutable_child(i) = c; | ~~~^ In file included from /home/jmb/gfs/oss/gcc/gcc-12.1.0/usr/local/include/c++/12.1.0/x86_64-pc-linux-gnu/bits/c++allocator.h:33, from /home/jmb/gfs/oss/gcc/gcc-12.1.0/usr/local/include/c++/12.1.0/bits/allocator.h:46, from /home/jmb/gfs/oss/gcc/gcc-12.1.0/usr/local/include/c++/12.1.0/memory:64, from /home/jmb/gfs/oss/parallel_hashmap/1.35/include/phmap_fwd_decl.h:21, from /home/jmb/gfs/oss/parallel_hashmap/1.35/include/btree.h:64: In member function ‘_Tp std::__new_allocator<_Tp>::allocate(size_type, const void) [with _Tp = phmap::priv::Allocate<8, std::allocator<std::pair<const long unsigned int, map_data> > >(std::allocator<std::pair<const long unsigned int, map_data> >, size_t)::M]’, inlined from ‘static phmap::allocator_traits::pointer phmap::allocator_traits::allocate(Alloc&, size_type) [with Alloc = std::allocator<phmap::priv::Allocate<8, std::allocator<std::pair<const long unsigned int, map_data> > >(std::allocator<std::pair<const long unsigned int, map_data> >, size_t)::M>]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/phmap_base.h:1468:26, inlined from ‘void phmap::priv::Allocate(Alloc, size_t) [with long unsigned int Alignment = 8; Alloc = std::allocator<std::pair<const long unsigned int, map_data> >]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/phmap_base.h:4356:25, inlined from ‘phmap::priv::btree::node_type phmap::priv::btree::allocate(size_type) [with Params = phmap::priv::map_params<long unsigned int, map_data, phmap::Less, std::allocator<std::pair<const long unsigned int, map_data> >, 256, false>]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/btree.h:1978:62, inlined from ‘phmap::priv::btree::node_type* phmap::priv::btree::new_leaf_root_node(int) [with Params = phmap::priv::map_params<long unsigned int, map_data, phmap::Less, std::allocator<std::pair<const long unsigned int, map_data> >, 256, false>]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/btree.h:1992:36, inlined from ‘phmap::priv::btree::iterator phmap::priv::btree::internal_emplace(iterator, Args&& ...) [with Args = {std::pair<const long unsigned int, map_data>}; Params = phmap::priv::map_params<long unsigned int, map_data, phmap::Less, std::allocator<std::pair<const long unsigned int, map_data> >, 256, false>]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/btree.h:3117:39, inlined from ‘std::pair<phmap::priv::btree_iterator<phmap::priv::btree_node, typename Params::reference, typename Params::pointer>, bool> phmap::priv::btree::insert_unique(const key_type&, Args&& ...) [with Args = {std::pair<const long unsigned int, map_data>}; Params = phmap::priv::map_params<long unsigned int, map_data, phmap::Less, std::allocator<std::pair<const long unsigned int, map_data> >, 256, false>]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/btree.h:2551:33, inlined from ‘std::pair<typename Tree::iterator, bool> phmap::priv::btree_set_container::insert(value_type&&) [with Tree = phmap::priv::btree<phmap::priv::map_params<long unsigned int, map_data, phmap::Less, std::allocator<std::pair<const long unsigned int, map_data> >, 256, false> >]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/btree.h:3485:45, inlined from ‘int main(int, char*)’ at btree-compile.cpp:22:17: /home/jmb/gfs/oss/gcc/gcc-12.1.0/usr/local/include/c++/12.1.0/bits/new_allocator.h:137:55: note: at offset [264, 2296] into object of size [64, 256] allocated by ‘operator new’ 137 | return static_cast<_Tp>(_GLIBCXX_OPERATOR_NEW(__n sizeof(_Tp))); | ^ In member function ‘phmap::priv::btree_node phmap::priv::btree_node::child(size_type) const [with Params = phmap::priv::map_params<long unsigned int, map_data, phmap::Less, std::allocator<std::pair<const long unsigned int, map_data> >, 256, false>]’, inlined from ‘void phmap::priv::btree_node::swap(phmap::priv::btree_node, allocator_type) [with Params = phmap::priv::map_params<long unsigned int, map_data, phmap::Less, std::allocator<std::pair<const long unsigned int, map_data> >, 256, false>]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/btree.h:2403:46, inlined from ‘phmap::priv::btree::iterator phmap::priv::btree::internal_emplace(iterator, Args&& ...) [with Args = {std::pair<const long unsigned int, map_data>}; Params = phmap::priv::map_params<long unsigned int, map_data, phmap::Less, std::allocator<std::pair<const long unsigned int, map_data> >, 256, false>]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/btree.h:3118:32, inlined from ‘std::pair<phmap::priv::btree_iterator<phmap::priv::btree_node, typename Params::reference, typename Params::pointer>, bool> phmap::priv::btree::insert_unique(const key_type&, Args&& ...) [with Args = {std::pair<const long unsigned int, map_data>}; Params = phmap::priv::map_params<long unsigned int, map_data, phmap::Less, std::allocator<std::pair<const long unsigned int, map_data> >, 256, false>]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/btree.h:2551:33, inlined from ‘std::pair<typename Tree::iterator, bool> phmap::priv::btree_set_container::insert(value_type&&) [with Tree = phmap::priv::btree<phmap::priv::map_params<long unsigned int, map_data, phmap::Less, std::allocator<std::pair<const long unsigned int, map_data> >, 256, false> >]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/btree.h:3485:45, inlined from ‘int main(int, char)’ at btree-compile.cpp:22:17: /home/jmb/gfs/oss/parallel_hashmap/1.35/include/btree.h:1214:70: warning: array subscript 32 is outside array bounds of ‘phmap::priv::Allocate<8, std::allocator<std::pair<const long unsigned int, map_data> > >(std::allocator<std::pair<const long unsigned int, map_data> >, size_t)::M [32]’ [-Warray-bounds] 1214 | btree_node child(size_type i) const { return GetField<3>()[i]; } | ^ In member function ‘_Tp std::__new_allocator<_Tp>::allocate(size_type, const void) [with _Tp = phmap::priv::Allocate<8, std::allocator<std::pair<const long unsigned int, map_data> > >(std::allocator<std::pair<const long unsigned int, map_data> >, size_t)::M]’, inlined from ‘static phmap::allocator_traits::pointer phmap::allocator_traits::allocate(Alloc&, size_type) [with Alloc = std::allocator<phmap::priv::Allocate<8, std::allocator<std::pair<const long unsigned int, map_data> > >(std::allocator<std::pair<const long unsigned int, map_data> >, size_t)::M>]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/phmap_base.h:1468:26, inlined from ‘void phmap::priv::Allocate(Alloc, size_t) [with long unsigned int Alignment = 8; Alloc = std::allocator<std::pair<const long unsigned int, map_data> >]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/phmap_base.h:4356:25, inlined from ‘phmap::priv::btree::node_type phmap::priv::btree::allocate(size_type) [with Params = phmap::priv::map_params<long unsigned int, map_data, phmap::Less, std::allocator<std::pair<const long unsigned int, map_data> >, 256, false>]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/btree.h:1978:62, inlined from ‘phmap::priv::btree::node_type phmap::priv::btree::new_leaf_root_node(int) [with Params = phmap::priv::map_params<long unsigned int, map_data, phmap::Less, std::allocator<std::pair<const long unsigned int, map_data> >, 256, false>]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/btree.h:1992:36, inlined from ‘phmap::priv::btree::iterator phmap::priv::btree::internal_emplace(iterator, Args&& ...) [with Args = {std::pair<const long unsigned int, map_data>}; Params = phmap::priv::map_params<long unsigned int, map_data, phmap::Less, std::allocator<std::pair<const long unsigned int, map_data> >, 256, false>]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/btree.h:3117:39, inlined from ‘std::pair<phmap::priv::btree_iterator<phmap::priv::btree_node, typename Params::reference, typename Params::pointer>, bool> phmap::priv::btree::insert_unique(const key_type&, Args&& ...) [with Args = {std::pair<const long unsigned int, map_data>}; Params = phmap::priv::map_params<long unsigned int, map_data, phmap::Less, std::allocator<std::pair<const long unsigned int, map_data> >, 256, false>]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/btree.h:2551:33, inlined from ‘std::pair<typename Tree::iterator, bool> phmap::priv::btree_set_container::insert(value_type&&) [with Tree = phmap::priv::btree<phmap::priv::map_params<long unsigned int, map_data, phmap::Less, std::allocator<std::pair<const long unsigned int, map_data> >, 256, false> >]’ at /home/jmb/gfs/oss/parallel_hashmap/1.35/include/btree.h:3485:45, inlined from ‘int main(int, char*)’ at btree-compile.cpp:22:17: /home/jmb/gfs/oss/gcc/gcc-12.1.0/usr/local/include/c++/12.1.0/bits/new_allocator.h:137:55: note: at offset 256 into object of size [64, 256] allocated by ‘operator new’ 137 | return static_cast<_Tp>(_GLIBCXX_OPERATOR_NEW(__n * sizeof(_Tp))); | ^

greg7mdp commented 2 years ago

I'll have a look, give me a couple days, I think it will need some research to address this. Thanks for the report!

jmbnyc commented 2 years ago

Thanks Greg. We are trying to get our code base uplifted to gcc 12.1 and right now this is not a show stopper because it is just a warning but it seems a bit ominous and something that at a min should be understood. I realize now that I did not attach the simple program I wrote to shot you the issue (we are getting the same compilation warnings in every spot where we use btree_map so I created a small stand alone program to show you).

Well, attach does not appear to work so I'll just inline the code here:

include

include

class map_data { public: map_data(uint64t x) : x(x), y(2*x) {;}

uint64_t _x; uint64_t _y; };

using my_map = phmap::btree_map<uint64_t, map_data>;

int main(int, char**) { my_map myMap; for (size_t i = 0; i < 128; ++i) { myMap.insert(my_map::value_type(i, map_data(i))); } auto it = myMap.find(256); if (it != myMap.end()) { fprintf(stderr, "FOUND\n"); } else { fprintf(stderr, "NOT FOUND\n"); } return 0; }

jmbnyc commented 2 years ago

hmm, looks like the code was mangled but I think you can see that it is just a simple instantiation of the btree_map that will produce the warnings.

greg7mdp commented 2 years ago

Hey @jmbnyc , please give the latest version a try, the warning should be fixed.

jmbnyc commented 2 years ago

Greg, Sweet. It is the only remaining 'warning' in our code base with the uplift to gcc 12.1 so it would be great to get it resolved and be able to move forward with gcc 12.1.

greg7mdp commented 2 years ago

Hey Jeffrey, I'm closing the issue. Please reopen if you still see warnings. Thanks again for using phmap (ps: if you are using C++20, you may want to switch to my gtl library which has all the same classes and more).

jmbnyc commented 2 years ago

Greg, Sorry, I integrated this afternoon and confirm the warnings are no longer present. I meant to update earlier but got busy with other work. Thanks again.

greg7mdp commented 2 years ago

Perfect, than you for letting me know!