martinus / unordered_dense

A fast & densely stored hashmap and hashset based on robin-hood backward shift deletion
MIT License
898 stars 72 forks source link

array of segmented_map #111

Open spencer-lunarg opened 7 months ago

spencer-lunarg commented 7 months ago

(I don't have a good way to reproduce this, so please feel free to close this issue if you find it not possible to fix easily or something not supported... but wanted to at least report it)

Trying to use this library, we found that an array of segmented_map causes failure for both MinGW and Ubuntu 22.04/clang-14/fsanitize=thread

// error
ankerl::unordered_dense::segmented_map<int, int> maps[2];

// no error
ankerl::unordered_dense::segmented_map<int, int> maps[1];

// no error
std::array<ankerl::unordered_dense::segmented_map<int, int>, 2> maps;
/usr/include/x86_64-linux-gnu/bits/string_fortified.h:59:33: error: ‘void* __builtin_memset(void*, int, long unsigned int)’ writing 34359738368 bytes into a region of size 32 overflows the destination [-Werror=stringop-overflow=]
   59 |   return __builtin___memset_chk (__dest, __ch, __len,
      |          ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
   60 |                                  __glibc_objsize0 (__dest));
      |                                  ~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/include/x86_64-linux-gnu/c++/11/bits/c++allocator.h:33,
                 from /usr/include/c++/11/bits/allocator.h:46,
                 from /usr/include/c++/11/unordered_map:40,
                 from /usr/include/c++/11/functional:61,
                 from /usr/include/c++/11/pstl/glue_algorithm_defs.h:13,
                 from /usr/include/c++/11/algorithm:74,

The issue is seems to be the fact bucket_count() returns a large value

In this case, the Bucket was 8 and that with max uint32 is how we got 8 * 4294967296 = 34359738368

     void clear_buckets() {
        if (m_buckets != nullptr) {
            std::memset(&*m_buckets, 0, sizeof(Bucket) * bucket_count());
        }
    }

Spent some time trying to understand, but seems the

wmorgan6796 commented 7 months ago

To chime in here as well:

The latest version of unordered_dense fires off compiler warnings as noted here:

_deps/unordered_dense-src/include/ankerl/unordered_dense.h:984:24: error: 'void* memset(void*, int, size_t)' forming offset [32, 34359738367] is out of the bounds [0, 32] [-Werror=array-bounds=]
  984 |             std::memset(&*m_buckets, 0, sizeof(Bucket) * bucket_count());
      |             ~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1plus: all warnings being treated as errors

I'm having trouble integrating the new version as its a requirement that we treat warnings as errors.

Information: GCC 13.2, Ubuntu 20.04, CMake 3.27

The relevant usage in my code is this:

struct Entry {
  std::string a;
  uint64_t b;
};

ankerl::unordered_dense::map<boost::uuids::uuid, Entry, boost::hash<boost::uuids::uuid>> a_;

I tried including the library as a system include, but this didn't not solve the problem. Any guidance would be helpful