search

LLNL / zfp

Compressed numerical arrays that support high-speed random access
http://zfp.llnl.gov
BSD 3-Clause "New" or "Revised" License
754 stars 152 forks source link

race condition when using private_const_view from inside oneapi::tbb:parallel_for lambda #199

Closed aavbsouza closed 1 year ago

aavbsouza commented 1 year ago

Hello everyone I am trying to use private_const_view with the oneapi tbb library. I am using for now the tbb algorithm parallel_for. This simple example below demonstrates the issue:

#include <numeric>
#include <vector>
#include <oneapi/tbb/parallel_for.h>
#include <zfp/constarray2.hpp>

void test() {
    const std::size_t n1 = 2000;
    const std::size_t n2 = 60000;
    const std::size_t nindex = 10000;
    std::vector<float> buf(n1 * n2);
    zfp::const_array2<float> comp_mat;
    comp_mat.set_config(zfp_config_precision(8));
    comp_mat.resize(n1, n2);
    comp_mat.set(buf.data());
    std::vector<std::size_t> indexes(nindex);
    std::vector<double> results(nindex);
    std::iota(indexes.begin(), indexes.end(), 0);
    oneapi::tbb::parallel_for(oneapi::tbb::blocked_range<std::size_t>(0, indexes.size()),
                              [&](const oneapi::tbb::blocked_range<std::size_t> &r) {
                                  for (std::size_t i = r.begin(); i < r.end(); ++i) {
                                      zfp::const_array2<float>::private_const_view pv(&comp_mat);
                                      double sum = 0.0;
                                      for (std::size_t i1 = 0; i1 < n1; ++i1) {
                                          sum += pv(i1, i);
                                      }
                                      results[i] = sum;
                                  }
                              });
}
int main() {
    test();
    return 0;
}

When running with thread sanitize (rockylinux docker image, gcc 11.3.1-2) I got this error:

==================
WARNING: ThreadSanitizer: data race (pid=109698)
  Write of size 8 at 0x7b0c00000040 by thread T1:
    #0 stream_read_word /host/build-gcc/_deps/zfp-src/include/zfp/bitstream.inl:151 (zfp_tbb+0x458abc)
    #1 stream_rseek /host/build-gcc/_deps/zfp-src/include/zfp/bitstream.inl:345 (zfp_tbb+0x458abc)
    #2 zfp::codec::zfp_base<2u, float>::decode_block(zfp_stream*, unsigned long, float*) /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:186 (zfp_tbb+0x409e15)
    #3 zfp::codec::zfp_base<2u, float>::decode_block(unsigned long, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:171 (zfp_tbb+0x409e15)
    #4 zfp::codec::zfp2<float>::decode_block(unsigned long, unsigned int, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:299 (zfp_tbb+0x409e15)
    #5 zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::decode(unsigned long, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store2.hpp:107 (zfp_tbb+0x409e15)
    #6 zfp::internal::BlockCache2<float, zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::line(unsigned long, unsigned long, bool) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/cache2.hpp:176 (zfp_tbb+0x409e15)
    #7 zfp::internal::BlockCache2<float, zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::get(unsigned long, unsigned long) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/cache2.hpp:59 (zfp_tbb+0x409e15)
    #8 zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::get(unsigned long, unsigned long) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/view2.hpp:405 (zfp_tbb+0x409e15)
    #9 zfp::internal::dim2::const_handle<zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> > >::get() const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/handle2.hpp:28 (zfp_tbb+0x409e15)
    #10 zfp::internal::dim2::const_reference<zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> > >::operator float() const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/reference2.hpp:19 (zfp_tbb+0x409e15)
    #11 operator() /workspaces/cpp-zfp/tests/zfp_tbb.cpp:24 (zfp_tbb+0x409e15)
    #12 run_body /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:119 (zfp_tbb+0x40aa4b)
    #13 work_balance<tbb::detail::d1::start_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)>, const tbb::detail::d1::auto_partitioner>, tbb::detail::d1::blocked_range<long unsigned int> > /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/partitioner.h:462 (zfp_tbb+0x40aa4b)
    #14 execute<tbb::detail::d1::start_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)>, const tbb::detail::d1::auto_partitioner>, tbb::detail::d1::blocked_range<long unsigned int> > /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/partitioner.h:288 (zfp_tbb+0x40aa4b)
    #15 execute /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:172 (zfp_tbb+0x40aa4b)
    #16 tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::outermost_worker_waiter>(tbb::detail::d1::task*, tbb::detail::r1::outermost_worker_waiter&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.h:322 (zfp_tbb+0x415fe0)
    #17 tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<tbb::detail::r1::outermost_worker_waiter>(tbb::detail::d1::task*, tbb::detail::r1::outermost_worker_waiter&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.h:458 (zfp_tbb+0x415fe0)
    #18 tbb::detail::r1::arena::process(tbb::detail::r1::thread_data&) /host/build-gcc/_deps/tbb-src/src/tbb/arena.cpp:137 (zfp_tbb+0x415fe0)
    #19 tbb::detail::r1::market::process(rml::job&) /host/build-gcc/_deps/tbb-src/src/tbb/market.cpp:599 (zfp_tbb+0x42cfea)
    #20 tbb::detail::r1::rml::private_worker::run() /host/build-gcc/_deps/tbb-src/src/tbb/private_server.cpp:271 (zfp_tbb+0x431dce)
    #21 tbb::detail::r1::rml::private_worker::thread_routine(void*) /host/build-gcc/_deps/tbb-src/src/tbb/private_server.cpp:221 (zfp_tbb+0x43213d)

  Previous write of size 8 at 0x7b0c00000040 by main thread:
    #0 stream_rseek /host/build-gcc/_deps/zfp-src/include/zfp/bitstream.inl:343 (zfp_tbb+0x458a85)
    #1 zfp::codec::zfp_base<2u, float>::decode_block(zfp_stream*, unsigned long, float*) /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:186 (zfp_tbb+0x409e15)
    #2 zfp::codec::zfp_base<2u, float>::decode_block(unsigned long, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:171 (zfp_tbb+0x409e15)
    #3 zfp::codec::zfp2<float>::decode_block(unsigned long, unsigned int, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:299 (zfp_tbb+0x409e15)
    #4 zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::decode(unsigned long, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store2.hpp:107 (zfp_tbb+0x409e15)
    #5 zfp::internal::BlockCache2<float, zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::line(unsigned long, unsigned long, bool) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/cache2.hpp:176 (zfp_tbb+0x409e15)
    #6 zfp::internal::BlockCache2<float, zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::get(unsigned long, unsigned long) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/cache2.hpp:59 (zfp_tbb+0x409e15)
    #7 zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::get(unsigned long, unsigned long) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/view2.hpp:405 (zfp_tbb+0x409e15)
    #8 zfp::internal::dim2::const_handle<zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> > >::get() const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/handle2.hpp:28 (zfp_tbb+0x409e15)
    #9 zfp::internal::dim2::const_reference<zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> > >::operator float() const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/reference2.hpp:19 (zfp_tbb+0x409e15)
    #10 operator() /workspaces/cpp-zfp/tests/zfp_tbb.cpp:24 (zfp_tbb+0x409e15)
    #11 run_body /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:119 (zfp_tbb+0x40aa4b)
    #12 work_balance<tbb::detail::d1::start_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)>, const tbb::detail::d1::auto_partitioner>, tbb::detail::d1::blocked_range<long unsigned int> > /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/partitioner.h:462 (zfp_tbb+0x40aa4b)
    #13 execute<tbb::detail::d1::start_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)>, const tbb::detail::d1::auto_partitioner>, tbb::detail::d1::blocked_range<long unsigned int> > /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/partitioner.h:288 (zfp_tbb+0x40aa4b)
    #14 execute /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:172 (zfp_tbb+0x40aa4b)
    #15 tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::external_waiter>(tbb::detail::d1::task*, tbb::detail::r1::external_waiter&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.h:322 (zfp_tbb+0x437bf0)
    #16 tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<tbb::detail::r1::external_waiter>(tbb::detail::d1::task*, tbb::detail::r1::external_waiter&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.h:458 (zfp_tbb+0x437bf0)
    #17 tbb::detail::r1::task_dispatcher::execute_and_wait(tbb::detail::d1::task*, tbb::detail::d1::wait_context&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.cpp:168 (zfp_tbb+0x437bf0)
    #18 tbb::detail::r1::execute_and_wait(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&, tbb::detail::d1::wait_context&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.cpp:121 (zfp_tbb+0x43800c)
    #19 tbb::detail::d1::execute_and_wait(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&, tbb::detail::d1::wait_context&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/detail/_task.h:191 (zfp_tbb+0x40cd17)
    #20 run /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:114 (zfp_tbb+0x40cd17)
    #21 run /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:103 (zfp_tbb+0x40cd17)
    #22 parallel_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)> > /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:231 (zfp_tbb+0x40cd17)
    #23 test() /workspaces/cpp-zfp/tests/zfp_tbb.cpp:18 (zfp_tbb+0x40cd17)
    #24 main /workspaces/cpp-zfp/tests/zfp_tbb.cpp:31 (zfp_tbb+0x408be7)

  Location is heap block of size 40 at 0x7b0c00000030 allocated by main thread:
    #0 malloc <null> (libtsan.so.0+0x32dd7)
    #1 stream_open /host/build-gcc/_deps/zfp-src/include/zfp/bitstream.inl:444 (zfp_tbb+0x458e75)
    #2 zfp::codec::zfp_base<2u, float>::open(void*, unsigned long) /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:60 (zfp_tbb+0x40cf97)
    #3 zfp::internal::BlockStore<zfp::codec::zfp2<float>, zfp::index::hybrid4>::compact() /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store.hpp:122 (zfp_tbb+0x40cf97)
    #4 zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::set(float const*, bool) /host/build-gcc/_deps/zfp-src/include/zfp/constarray2.hpp:224 (zfp_tbb+0x40cf97)
    #5 test() /workspaces/cpp-zfp/tests/zfp_tbb.cpp:14 (zfp_tbb+0x40cf97)
    #6 main /workspaces/cpp-zfp/tests/zfp_tbb.cpp:31 (zfp_tbb+0x408be7)

  Thread T1 (tid=110125, running) created by main thread at:
    #0 pthread_create <null> (libtsan.so.0+0x61748)
    #1 tbb::detail::r1::rml::internal::thread_monitor::launch(void* (*)(void*), void*, unsigned long) /host/build-gcc/_deps/tbb-src/src/tbb/rml_thread_monitor.h:195 (zfp_tbb+0x431a44)
    #2 tbb::detail::r1::rml::private_worker::wake_or_launch() /host/build-gcc/_deps/tbb-src/src/tbb/private_server.cpp:305 (zfp_tbb+0x431a44)
    #3 tbb::detail::r1::rml::private_server::wake_some(int) /host/build-gcc/_deps/tbb-src/src/tbb/private_server.cpp:412 (zfp_tbb+0x431a44)
    #4 tbb::detail::r1::rml::private_server::adjust_job_count_estimate(int) /host/build-gcc/_deps/tbb-src/src/tbb/private_server.cpp:423 (zfp_tbb+0x431cba)
    #5 tbb::detail::r1::market::adjust_demand(tbb::detail::r1::arena&, int, bool) /host/build-gcc/_deps/tbb-src/src/tbb/market.cpp:588 (zfp_tbb+0x42cbad)
    #6 void tbb::detail::r1::arena::advertise_new_work<(tbb::detail::r1::arena::new_work_type)0>() /host/build-gcc/_deps/tbb-src/src/tbb/arena.h:547 (zfp_tbb+0x4395e3)
    #7 spawn_and_notify /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.cpp:26 (zfp_tbb+0x43604b)
    #8 tbb::detail::r1::spawn(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.cpp:39 (zfp_tbb+0x43604b)
    #9 tbb::detail::d1::spawn(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/detail/_task.h:182 (zfp_tbb+0x40a4ea)
    #10 tbb::detail::d1::auto_partition_type::spawn_task(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/partitioner.h:511 (zfp_tbb+0x40a4ea)
    #11 spawn_self /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:146 (zfp_tbb+0x40a4ea)
    #12 offer_work_impl<tbb::detail::d1::start_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)>, const tbb::detail::d1::auto_partitioner>&, tbb::detail::d0::split&> /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:142 (zfp_tbb+0x40a4ea)
    #13 offer_work /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:124 (zfp_tbb+0x40a4ea)
    #14 execute<tbb::detail::d1::start_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)>, const tbb::detail::d1::auto_partitioner>, tbb::detail::d1::blocked_range<long unsigned int> > /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/partitioner.h:284 (zfp_tbb+0x40a4ea)
    #15 execute /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:172 (zfp_tbb+0x40a4ea)
    #16 tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::external_waiter>(tbb::detail::d1::task*, tbb::detail::r1::external_waiter&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.h:322 (zfp_tbb+0x437bf0)
    #17 tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<tbb::detail::r1::external_waiter>(tbb::detail::d1::task*, tbb::detail::r1::external_waiter&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.h:458 (zfp_tbb+0x437bf0)
    #18 tbb::detail::r1::task_dispatcher::execute_and_wait(tbb::detail::d1::task*, tbb::detail::d1::wait_context&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.cpp:168 (zfp_tbb+0x437bf0)
    #19 tbb::detail::r1::execute_and_wait(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&, tbb::detail::d1::wait_context&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.cpp:121 (zfp_tbb+0x43800c)
    #20 tbb::detail::d1::execute_and_wait(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&, tbb::detail::d1::wait_context&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/detail/_task.h:191 (zfp_tbb+0x40cd17)
    #21 run /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:114 (zfp_tbb+0x40cd17)
    #22 run /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:103 (zfp_tbb+0x40cd17)
    #23 parallel_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)> > /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:231 (zfp_tbb+0x40cd17)
    #24 test() /workspaces/cpp-zfp/tests/zfp_tbb.cpp:18 (zfp_tbb+0x40cd17)
    #25 main /workspaces/cpp-zfp/tests/zfp_tbb.cpp:31 (zfp_tbb+0x408be7)

SUMMARY: ThreadSanitizer: data race /host/build-gcc/_deps/zfp-src/include/zfp/bitstream.inl:151 in stream_read_word
==================
==================
WARNING: ThreadSanitizer: data race (pid=109698)
  Write of size 8 at 0x7b0c00000030 by thread T1:
    #0 stream_rseek /host/build-gcc/_deps/zfp-src/include/zfp/bitstream.inl:350 (zfp_tbb+0x458a94)
    #1 zfp::codec::zfp_base<2u, float>::decode_block(zfp_stream*, unsigned long, float*) /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:186 (zfp_tbb+0x409e15)
    #2 zfp::codec::zfp_base<2u, float>::decode_block(unsigned long, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:171 (zfp_tbb+0x409e15)
    #3 zfp::codec::zfp2<float>::decode_block(unsigned long, unsigned int, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:299 (zfp_tbb+0x409e15)
    #4 zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::decode(unsigned long, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store2.hpp:107 (zfp_tbb+0x409e15)
    #5 zfp::internal::BlockCache2<float, zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::line(unsigned long, unsigned long, bool) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/cache2.hpp:176 (zfp_tbb+0x409e15)
    #6 zfp::internal::BlockCache2<float, zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::get(unsigned long, unsigned long) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/cache2.hpp:59 (zfp_tbb+0x409e15)
    #7 zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::get(unsigned long, unsigned long) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/view2.hpp:405 (zfp_tbb+0x409e15)
    #8 zfp::internal::dim2::const_handle<zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> > >::get() const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/handle2.hpp:28 (zfp_tbb+0x409e15)
    #9 zfp::internal::dim2::const_reference<zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> > >::operator float() const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/reference2.hpp:19 (zfp_tbb+0x409e15)
    #10 operator() /workspaces/cpp-zfp/tests/zfp_tbb.cpp:24 (zfp_tbb+0x409e15)
    #11 run_body /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:119 (zfp_tbb+0x40aa4b)
    #12 work_balance<tbb::detail::d1::start_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)>, const tbb::detail::d1::auto_partitioner>, tbb::detail::d1::blocked_range<long unsigned int> > /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/partitioner.h:462 (zfp_tbb+0x40aa4b)
    #13 execute<tbb::detail::d1::start_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)>, const tbb::detail::d1::auto_partitioner>, tbb::detail::d1::blocked_range<long unsigned int> > /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/partitioner.h:288 (zfp_tbb+0x40aa4b)
    #14 execute /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:172 (zfp_tbb+0x40aa4b)
    #15 tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::outermost_worker_waiter>(tbb::detail::d1::task*, tbb::detail::r1::outermost_worker_waiter&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.h:322 (zfp_tbb+0x415fe0)
    #16 tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<tbb::detail::r1::outermost_worker_waiter>(tbb::detail::d1::task*, tbb::detail::r1::outermost_worker_waiter&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.h:458 (zfp_tbb+0x415fe0)
    #17 tbb::detail::r1::arena::process(tbb::detail::r1::thread_data&) /host/build-gcc/_deps/tbb-src/src/tbb/arena.cpp:137 (zfp_tbb+0x415fe0)
    #18 tbb::detail::r1::market::process(rml::job&) /host/build-gcc/_deps/tbb-src/src/tbb/market.cpp:599 (zfp_tbb+0x42cfea)
    #19 tbb::detail::r1::rml::private_worker::run() /host/build-gcc/_deps/tbb-src/src/tbb/private_server.cpp:271 (zfp_tbb+0x431dce)
    #20 tbb::detail::r1::rml::private_worker::thread_routine(void*) /host/build-gcc/_deps/tbb-src/src/tbb/private_server.cpp:221 (zfp_tbb+0x43213d)

  Previous write of size 8 at 0x7b0c00000030 by main thread:
    #0 stream_rseek /host/build-gcc/_deps/zfp-src/include/zfp/bitstream.inl:350 (zfp_tbb+0x458a94)
    #1 zfp::codec::zfp_base<2u, float>::decode_block(zfp_stream*, unsigned long, float*) /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:186 (zfp_tbb+0x409e15)
    #2 zfp::codec::zfp_base<2u, float>::decode_block(unsigned long, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:171 (zfp_tbb+0x409e15)
    #3 zfp::codec::zfp2<float>::decode_block(unsigned long, unsigned int, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:299 (zfp_tbb+0x409e15)
    #4 zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::decode(unsigned long, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store2.hpp:107 (zfp_tbb+0x409e15)
    #5 zfp::internal::BlockCache2<float, zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::line(unsigned long, unsigned long, bool) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/cache2.hpp:176 (zfp_tbb+0x409e15)
    #6 zfp::internal::BlockCache2<float, zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::get(unsigned long, unsigned long) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/cache2.hpp:59 (zfp_tbb+0x409e15)
    #7 zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::get(unsigned long, unsigned long) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/view2.hpp:405 (zfp_tbb+0x409e15)
    #8 zfp::internal::dim2::const_handle<zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> > >::get() const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/handle2.hpp:28 (zfp_tbb+0x409e15)
    #9 zfp::internal::dim2::const_reference<zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> > >::operator float() const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/reference2.hpp:19 (zfp_tbb+0x409e15)
    #10 operator() /workspaces/cpp-zfp/tests/zfp_tbb.cpp:24 (zfp_tbb+0x409e15)
    #11 run_body /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:119 (zfp_tbb+0x40aa4b)
    #12 work_balance<tbb::detail::d1::start_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)>, const tbb::detail::d1::auto_partitioner>, tbb::detail::d1::blocked_range<long unsigned int> > /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/partitioner.h:462 (zfp_tbb+0x40aa4b)
    #13 execute<tbb::detail::d1::start_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)>, const tbb::detail::d1::auto_partitioner>, tbb::detail::d1::blocked_range<long unsigned int> > /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/partitioner.h:288 (zfp_tbb+0x40aa4b)
    #14 execute /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:172 (zfp_tbb+0x40aa4b)
    #15 tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::external_waiter>(tbb::detail::d1::task*, tbb::detail::r1::external_waiter&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.h:322 (zfp_tbb+0x437bf0)
    #16 tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<tbb::detail::r1::external_waiter>(tbb::detail::d1::task*, tbb::detail::r1::external_waiter&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.h:458 (zfp_tbb+0x437bf0)
    #17 tbb::detail::r1::task_dispatcher::execute_and_wait(tbb::detail::d1::task*, tbb::detail::d1::wait_context&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.cpp:168 (zfp_tbb+0x437bf0)
    #18 tbb::detail::r1::execute_and_wait(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&, tbb::detail::d1::wait_context&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.cpp:121 (zfp_tbb+0x43800c)
    #19 tbb::detail::d1::execute_and_wait(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&, tbb::detail::d1::wait_context&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/detail/_task.h:191 (zfp_tbb+0x40cd17)
    #20 run /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:114 (zfp_tbb+0x40cd17)
    #21 run /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:103 (zfp_tbb+0x40cd17)
    #22 parallel_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)> > /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:231 (zfp_tbb+0x40cd17)
    #23 test() /workspaces/cpp-zfp/tests/zfp_tbb.cpp:18 (zfp_tbb+0x40cd17)
    #24 main /workspaces/cpp-zfp/tests/zfp_tbb.cpp:31 (zfp_tbb+0x408be7)

  Location is heap block of size 40 at 0x7b0c00000030 allocated by main thread:
    #0 malloc <null> (libtsan.so.0+0x32dd7)
    #1 stream_open /host/build-gcc/_deps/zfp-src/include/zfp/bitstream.inl:444 (zfp_tbb+0x458e75)
    #2 zfp::codec::zfp_base<2u, float>::open(void*, unsigned long) /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:60 (zfp_tbb+0x40cf97)
    #3 zfp::internal::BlockStore<zfp::codec::zfp2<float>, zfp::index::hybrid4>::compact() /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store.hpp:122 (zfp_tbb+0x40cf97)
    #4 zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::set(float const*, bool) /host/build-gcc/_deps/zfp-src/include/zfp/constarray2.hpp:224 (zfp_tbb+0x40cf97)
    #5 test() /workspaces/cpp-zfp/tests/zfp_tbb.cpp:14 (zfp_tbb+0x40cf97)
    #6 main /workspaces/cpp-zfp/tests/zfp_tbb.cpp:31 (zfp_tbb+0x408be7)

  Thread T1 (tid=110125, running) created by main thread at:
    #0 pthread_create <null> (libtsan.so.0+0x61748)
    #1 tbb::detail::r1::rml::internal::thread_monitor::launch(void* (*)(void*), void*, unsigned long) /host/build-gcc/_deps/tbb-src/src/tbb/rml_thread_monitor.h:195 (zfp_tbb+0x431a44)
    #2 tbb::detail::r1::rml::private_worker::wake_or_launch() /host/build-gcc/_deps/tbb-src/src/tbb/private_server.cpp:305 (zfp_tbb+0x431a44)
    #3 tbb::detail::r1::rml::private_server::wake_some(int) /host/build-gcc/_deps/tbb-src/src/tbb/private_server.cpp:412 (zfp_tbb+0x431a44)
    #4 tbb::detail::r1::rml::private_server::adjust_job_count_estimate(int) /host/build-gcc/_deps/tbb-src/src/tbb/private_server.cpp:423 (zfp_tbb+0x431cba)
    #5 tbb::detail::r1::market::adjust_demand(tbb::detail::r1::arena&, int, bool) /host/build-gcc/_deps/tbb-src/src/tbb/market.cpp:588 (zfp_tbb+0x42cbad)
    #6 void tbb::detail::r1::arena::advertise_new_work<(tbb::detail::r1::arena::new_work_type)0>() /host/build-gcc/_deps/tbb-src/src/tbb/arena.h:547 (zfp_tbb+0x4395e3)
    #7 spawn_and_notify /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.cpp:26 (zfp_tbb+0x43604b)
    #8 tbb::detail::r1::spawn(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.cpp:39 (zfp_tbb+0x43604b)
    #9 tbb::detail::d1::spawn(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/detail/_task.h:182 (zfp_tbb+0x40a4ea)
    #10 tbb::detail::d1::auto_partition_type::spawn_task(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/partitioner.h:511 (zfp_tbb+0x40a4ea)
    #11 spawn_self /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:146 (zfp_tbb+0x40a4ea)
    #12 offer_work_impl<tbb::detail::d1::start_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)>, const tbb::detail::d1::auto_partitioner>&, tbb::detail::d0::split&> /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:142 (zfp_tbb+0x40a4ea)
    #13 offer_work /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:124 (zfp_tbb+0x40a4ea)
    #14 execute<tbb::detail::d1::start_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)>, const tbb::detail::d1::auto_partitioner>, tbb::detail::d1::blocked_range<long unsigned int> > /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/partitioner.h:284 (zfp_tbb+0x40a4ea)
    #15 execute /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:172 (zfp_tbb+0x40a4ea)
    #16 tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::external_waiter>(tbb::detail::d1::task*, tbb::detail::r1::external_waiter&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.h:322 (zfp_tbb+0x437bf0)
    #17 tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<tbb::detail::r1::external_waiter>(tbb::detail::d1::task*, tbb::detail::r1::external_waiter&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.h:458 (zfp_tbb+0x437bf0)
    #18 tbb::detail::r1::task_dispatcher::execute_and_wait(tbb::detail::d1::task*, tbb::detail::d1::wait_context&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.cpp:168 (zfp_tbb+0x437bf0)
    #19 tbb::detail::r1::execute_and_wait(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&, tbb::detail::d1::wait_context&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.cpp:121 (zfp_tbb+0x43800c)
    #20 tbb::detail::d1::execute_and_wait(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&, tbb::detail::d1::wait_context&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/detail/_task.h:191 (zfp_tbb+0x40cd17)
    #21 run /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:114 (zfp_tbb+0x40cd17)
    #22 run /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:103 (zfp_tbb+0x40cd17)
    #23 parallel_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)> > /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:231 (zfp_tbb+0x40cd17)
    #24 test() /workspaces/cpp-zfp/tests/zfp_tbb.cpp:18 (zfp_tbb+0x40cd17)
    #25 main /workspaces/cpp-zfp/tests/zfp_tbb.cpp:31 (zfp_tbb+0x408be7)

SUMMARY: ThreadSanitizer: data race /host/build-gcc/_deps/zfp-src/include/zfp/bitstream.inl:350 in stream_rseek
==================
==================
WARNING: ThreadSanitizer: data race (pid=109698)
  Write of size 8 at 0x7b0c00000038 by thread T2:
    #0 stream_rseek /host/build-gcc/_deps/zfp-src/include/zfp/bitstream.inl:350 (zfp_tbb+0x458aa0)
    #1 zfp::codec::zfp_base<2u, float>::decode_block(zfp_stream*, unsigned long, float*) /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:186 (zfp_tbb+0x409e15)
    #2 zfp::codec::zfp_base<2u, float>::decode_block(unsigned long, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:171 (zfp_tbb+0x409e15)
    #3 zfp::codec::zfp2<float>::decode_block(unsigned long, unsigned int, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:299 (zfp_tbb+0x409e15)
    #4 zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::decode(unsigned long, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store2.hpp:107 (zfp_tbb+0x409e15)
    #5 zfp::internal::BlockCache2<float, zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::line(unsigned long, unsigned long, bool) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/cache2.hpp:176 (zfp_tbb+0x409e15)
    #6 zfp::internal::BlockCache2<float, zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::get(unsigned long, unsigned long) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/cache2.hpp:59 (zfp_tbb+0x409e15)
    #7 zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::get(unsigned long, unsigned long) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/view2.hpp:405 (zfp_tbb+0x409e15)
    #8 zfp::internal::dim2::const_handle<zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> > >::get() const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/handle2.hpp:28 (zfp_tbb+0x409e15)
    #9 zfp::internal::dim2::const_reference<zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> > >::operator float() const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/reference2.hpp:19 (zfp_tbb+0x409e15)
    #10 operator() /workspaces/cpp-zfp/tests/zfp_tbb.cpp:24 (zfp_tbb+0x409e15)
    #11 run_body /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:119 (zfp_tbb+0x40aa4b)
    #12 work_balance<tbb::detail::d1::start_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)>, const tbb::detail::d1::auto_partitioner>, tbb::detail::d1::blocked_range<long unsigned int> > /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/partitioner.h:462 (zfp_tbb+0x40aa4b)
    #13 execute<tbb::detail::d1::start_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)>, const tbb::detail::d1::auto_partitioner>, tbb::detail::d1::blocked_range<long unsigned int> > /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/partitioner.h:288 (zfp_tbb+0x40aa4b)
    #14 execute /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:172 (zfp_tbb+0x40aa4b)
    #15 tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::outermost_worker_waiter>(tbb::detail::d1::task*, tbb::detail::r1::outermost_worker_waiter&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.h:322 (zfp_tbb+0x415fe0)
    #16 tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<tbb::detail::r1::outermost_worker_waiter>(tbb::detail::d1::task*, tbb::detail::r1::outermost_worker_waiter&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.h:458 (zfp_tbb+0x415fe0)
    #17 tbb::detail::r1::arena::process(tbb::detail::r1::thread_data&) /host/build-gcc/_deps/tbb-src/src/tbb/arena.cpp:137 (zfp_tbb+0x415fe0)
    #18 tbb::detail::r1::market::process(rml::job&) /host/build-gcc/_deps/tbb-src/src/tbb/market.cpp:599 (zfp_tbb+0x42cfea)
    #19 tbb::detail::r1::rml::private_worker::run() /host/build-gcc/_deps/tbb-src/src/tbb/private_server.cpp:271 (zfp_tbb+0x431dce)
    #20 tbb::detail::r1::rml::private_worker::thread_routine(void*) /host/build-gcc/_deps/tbb-src/src/tbb/private_server.cpp:221 (zfp_tbb+0x43213d)

  Previous write of size 8 at 0x7b0c00000038 by main thread:
    #0 stream_rseek /host/build-gcc/_deps/zfp-src/include/zfp/bitstream.inl:350 (zfp_tbb+0x458aa0)
    #1 zfp::codec::zfp_base<2u, float>::decode_block(zfp_stream*, unsigned long, float*) /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:186 (zfp_tbb+0x409e15)
    #2 zfp::codec::zfp_base<2u, float>::decode_block(unsigned long, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:171 (zfp_tbb+0x409e15)
    #3 zfp::codec::zfp2<float>::decode_block(unsigned long, unsigned int, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:299 (zfp_tbb+0x409e15)
    #4 zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::decode(unsigned long, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store2.hpp:107 (zfp_tbb+0x409e15)
    #5 zfp::internal::BlockCache2<float, zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::line(unsigned long, unsigned long, bool) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/cache2.hpp:176 (zfp_tbb+0x409e15)
    #6 zfp::internal::BlockCache2<float, zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::get(unsigned long, unsigned long) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/cache2.hpp:59 (zfp_tbb+0x409e15)
    #7 zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::get(unsigned long, unsigned long) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/view2.hpp:405 (zfp_tbb+0x409e15)
    #8 zfp::internal::dim2::const_handle<zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> > >::get() const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/handle2.hpp:28 (zfp_tbb+0x409e15)
    #9 zfp::internal::dim2::const_reference<zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> > >::operator float() const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/reference2.hpp:19 (zfp_tbb+0x409e15)
    #10 operator() /workspaces/cpp-zfp/tests/zfp_tbb.cpp:24 (zfp_tbb+0x409e15)
    #11 run_body /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:119 (zfp_tbb+0x40aa4b)
    #12 work_balance<tbb::detail::d1::start_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)>, const tbb::detail::d1::auto_partitioner>, tbb::detail::d1::blocked_range<long unsigned int> > /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/partitioner.h:462 (zfp_tbb+0x40aa4b)
    #13 execute<tbb::detail::d1::start_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)>, const tbb::detail::d1::auto_partitioner>, tbb::detail::d1::blocked_range<long unsigned int> > /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/partitioner.h:288 (zfp_tbb+0x40aa4b)
    #14 execute /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:172 (zfp_tbb+0x40aa4b)
    #15 tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::external_waiter>(tbb::detail::d1::task*, tbb::detail::r1::external_waiter&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.h:322 (zfp_tbb+0x437bf0)
    #16 tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<tbb::detail::r1::external_waiter>(tbb::detail::d1::task*, tbb::detail::r1::external_waiter&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.h:458 (zfp_tbb+0x437bf0)
    #17 tbb::detail::r1::task_dispatcher::execute_and_wait(tbb::detail::d1::task*, tbb::detail::d1::wait_context&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.cpp:168 (zfp_tbb+0x437bf0)
    #18 tbb::detail::r1::execute_and_wait(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&, tbb::detail::d1::wait_context&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.cpp:121 (zfp_tbb+0x43800c)
    #19 tbb::detail::d1::execute_and_wait(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&, tbb::detail::d1::wait_context&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/detail/_task.h:191 (zfp_tbb+0x40cd17)
    #20 run /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:114 (zfp_tbb+0x40cd17)
    #21 run /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:103 (zfp_tbb+0x40cd17)
    #22 parallel_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)> > /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:231 (zfp_tbb+0x40cd17)
    #23 test() /workspaces/cpp-zfp/tests/zfp_tbb.cpp:18 (zfp_tbb+0x40cd17)
    #24 main /workspaces/cpp-zfp/tests/zfp_tbb.cpp:31 (zfp_tbb+0x408be7)

  Location is heap block of size 40 at 0x7b0c00000030 allocated by main thread:
    #0 malloc <null> (libtsan.so.0+0x32dd7)
    #1 stream_open /host/build-gcc/_deps/zfp-src/include/zfp/bitstream.inl:444 (zfp_tbb+0x458e75)
    #2 zfp::codec::zfp_base<2u, float>::open(void*, unsigned long) /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:60 (zfp_tbb+0x40cf97)
    #3 zfp::internal::BlockStore<zfp::codec::zfp2<float>, zfp::index::hybrid4>::compact() /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store.hpp:122 (zfp_tbb+0x40cf97)
    #4 zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::set(float const*, bool) /host/build-gcc/_deps/zfp-src/include/zfp/constarray2.hpp:224 (zfp_tbb+0x40cf97)
    #5 test() /workspaces/cpp-zfp/tests/zfp_tbb.cpp:14 (zfp_tbb+0x40cf97)
    #6 main /workspaces/cpp-zfp/tests/zfp_tbb.cpp:31 (zfp_tbb+0x408be7)

  Thread T2 (tid=110126, running) created by main thread at:
    #0 pthread_create <null> (libtsan.so.0+0x61748)
    #1 tbb::detail::r1::rml::internal::thread_monitor::launch(void* (*)(void*), void*, unsigned long) /host/build-gcc/_deps/tbb-src/src/tbb/rml_thread_monitor.h:195 (zfp_tbb+0x431a44)
    #2 tbb::detail::r1::rml::private_worker::wake_or_launch() /host/build-gcc/_deps/tbb-src/src/tbb/private_server.cpp:305 (zfp_tbb+0x431a44)
    #3 tbb::detail::r1::rml::private_server::wake_some(int) /host/build-gcc/_deps/tbb-src/src/tbb/private_server.cpp:412 (zfp_tbb+0x431a44)
    #4 tbb::detail::r1::rml::private_server::adjust_job_count_estimate(int) /host/build-gcc/_deps/tbb-src/src/tbb/private_server.cpp:423 (zfp_tbb+0x431cba)
    #5 tbb::detail::r1::market::adjust_demand(tbb::detail::r1::arena&, int, bool) /host/build-gcc/_deps/tbb-src/src/tbb/market.cpp:588 (zfp_tbb+0x42cbad)
    #6 void tbb::detail::r1::arena::advertise_new_work<(tbb::detail::r1::arena::new_work_type)0>() /host/build-gcc/_deps/tbb-src/src/tbb/arena.h:547 (zfp_tbb+0x4395e3)
    #7 spawn_and_notify /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.cpp:26 (zfp_tbb+0x43604b)
    #8 tbb::detail::r1::spawn(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.cpp:39 (zfp_tbb+0x43604b)
    #9 tbb::detail::d1::spawn(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/detail/_task.h:182 (zfp_tbb+0x40a4ea)
    #10 tbb::detail::d1::auto_partition_type::spawn_task(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/partitioner.h:511 (zfp_tbb+0x40a4ea)
    #11 spawn_self /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:146 (zfp_tbb+0x40a4ea)
    #12 offer_work_impl<tbb::detail::d1::start_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)>, const tbb::detail::d1::auto_partitioner>&, tbb::detail::d0::split&> /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:142 (zfp_tbb+0x40a4ea)
    #13 offer_work /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:124 (zfp_tbb+0x40a4ea)
    #14 execute<tbb::detail::d1::start_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)>, const tbb::detail::d1::auto_partitioner>, tbb::detail::d1::blocked_range<long unsigned int> > /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/partitioner.h:284 (zfp_tbb+0x40a4ea)
    #15 execute /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:172 (zfp_tbb+0x40a4ea)
    #16 tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::external_waiter>(tbb::detail::d1::task*, tbb::detail::r1::external_waiter&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.h:322 (zfp_tbb+0x437bf0)
    #17 tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<tbb::detail::r1::external_waiter>(tbb::detail::d1::task*, tbb::detail::r1::external_waiter&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.h:458 (zfp_tbb+0x437bf0)
    #18 tbb::detail::r1::task_dispatcher::execute_and_wait(tbb::detail::d1::task*, tbb::detail::d1::wait_context&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.cpp:168 (zfp_tbb+0x437bf0)
    #19 tbb::detail::r1::execute_and_wait(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&, tbb::detail::d1::wait_context&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/task_dispatcher.cpp:121 (zfp_tbb+0x43800c)
    #20 tbb::detail::d1::execute_and_wait(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&, tbb::detail::d1::wait_context&, tbb::detail::d1::task_group_context&) /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/detail/_task.h:191 (zfp_tbb+0x40cd17)
    #21 run /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:114 (zfp_tbb+0x40cd17)
    #22 run /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:103 (zfp_tbb+0x40cd17)
    #23 parallel_for<tbb::detail::d1::blocked_range<long unsigned int>, test()::<lambda(const tbb::detail::d1::blocked_range<long unsigned int>&)> > /host/build-gcc/_deps/tbb-src/src/tbb/../../include/oneapi/tbb/parallel_for.h:231 (zfp_tbb+0x40cd17)
    #24 test() /workspaces/cpp-zfp/tests/zfp_tbb.cpp:18 (zfp_tbb+0x40cd17)
    #25 main /workspaces/cpp-zfp/tests/zfp_tbb.cpp:31 (zfp_tbb+0x408be7)

SUMMARY: ThreadSanitizer: data race /host/build-gcc/_deps/zfp-src/include/zfp/bitstream.inl:350 in stream_rseek
==================
lindstro commented 1 year ago

I'm afraid I'm unfamiliar with oneAPI and TBB, but I can make some general recommendations.

Before using a private_const_view you should flush the cache of the underlying array to ensure cache consistency. See the documentation and this code example: https://github.com/LLNL/zfp/blob/c4f175667568b70462de8d806a63773b3eb1fff7/examples/diffusion.cpp#L95-L103

It might also be instructive to capture the indices through which the private_const_view is accessed.

Finally, zfp's private views were written for use with OpenMP. I suspect that view reference counting, as needed to enforce thread safety, is being circumvented: https://github.com/LLNL/zfp/blob/c4f175667568b70462de8d806a63773b3eb1fff7/include/zfp/internal/array/store.hpp#L126-L136 We'd have to do something similar to support TBB.

aavbsouza commented 1 year ago

Hello @lindstro I have added the flush_cache without success. The error happens even if I only try to access a fixed index, for instance (0,0). I thought that compiling the zfp library with OpenMP enabled and also linking the program with OpenMP would be sufficient to use the OpenMP part of the code. I also have tried to create the views using the enumerable_thread_specific from the tbb library. I will try a similar arrangement using only OpenMP.

lindstro commented 1 year ago

I thought that compiling the zfp library with OpenMP enabled and also linking the program with OpenMP would be sufficient to use the OpenMP part of the code.

I'm afraid not as the zfp arrays are implemented in headers, and hence the OpenMP pragmas needs to be compiled together with the application code (i.e., your example above). Is it possible for TBB and OpenMP to coexist in the same application, e.g., by adding -fopenmp when compiling your code? If not, do you know the TBB equivalent to an OpenMP critical section?

aavbsouza commented 1 year ago

Hello @lindstro the oneapi tbb documentation states that is possible to mix the use of the library with other threading packages. The library supports many kinds of mutexes. I also believe that would be possible to use standard c++ mutexes.

Regarding the compilation I am linking with the openMP runtime

[1/2] /usr/bin/g++  -I/host/build-gcc/_deps/tbb-src/src/tbb/../../include -I/host/build-gcc/_deps/zfp-src/include -O2 -g -DNDEBUG -fPIE -fsanitize=thread -Wextra -Wall -pedantic -rdynamic -fno-omit-frame-pointer -g -ggdb -fdiagnostics-color=always -mtune=native -O3 -fopenmp -std=c++20 -MD -MT tests/CMakeFiles/zfp_tbb.dir/zfp_tbb.cpp.o -MF tests/CMakeFiles/zfp_tbb.dir/zfp_tbb.cpp.o.d -o tests/CMakeFiles/zfp_tbb.dir/zfp_tbb.cpp.o -c /workspaces/cpp-zfp/tests/zfp_tbb.cpp
[2/2] : && /usr/bin/g++ -O2 -g -DNDEBUG -fsanitize=thread tests/CMakeFiles/zfp_tbb.dir/zfp_tbb.cpp.o -o tests/zfp_tbb  -Wl,-rpath,/host/build-gcc/gnu_11.3_cxx20_64_relwithdebinfo:/host/build-gcc/_deps/zfp-build/lib64  gnu_11.3_cxx20_64_relwithdebinfo/libtbb.so.12.7  _deps/zfp-build/lib64/libzfp.so.1.0.0  /usr/lib/gcc/x86_64-redhat-linux/11/libgomp.so  /usr/lib64/libpthread.a && :
lindstro commented 1 year ago

Yes, but linking is not sufficient--you also need to compile your code with -fopenmp so that _OPENMP is defined and the pragmas in the zfp headers are processed. Even then, I'm not entirely convinced that non-OpenMP threads entering the OpenMP critical region will do the right thing, but it's worth a shot.

aavbsouza commented 1 year ago

Hello @lindstro I am compiling with -fopenmp is on the far right of the command. I also have tried to implement using openMP directly:

#include <numeric>
#include <vector>
#include <zfp/constarray2.hpp>

void test() {
    const std::size_t n1 = 2000;
    const std::size_t n2 = 60000;
    const std::size_t nindex = 10000;
    std::vector<float> buf(n1 * n2);
    zfp::const_array2<float> comp_mat(n1, n2, zfp_config_precision(10), buf.data());
    std::vector<std::size_t> indexes(nindex);
    std::vector<double> results(nindex);
    std::iota(indexes.begin(), indexes.end(), 0);

#pragma omp parallel default(none) shared(results, indexes, comp_mat)
    {
        zfp::const_array2<float>::private_const_view pv(&comp_mat);
#pragma omp for schedule(dynamic)
        for (std::size_t i = 0; i < indexes.size(); ++i) {
            double sum = 0.0;
            for (std::size_t i1 = 0; i1 < n1; ++i1) {
                sum += pv(i1, i);
            }
            results[i] = sum ;
        }
    }
}
int main() {
    test();
    return 0;
}

However I still got a data race:

WARNING: ThreadSanitizer: data race (pid=40760)
  Read of size 8 at 0x7ffdedbdd8e8 by thread T78:
    #0 zfp::internal::BlockStore<zfp::codec::zfp2<float>, zfp::index::hybrid4>::reference() /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store.hpp:132 (zfp_omp+0x40374f)
    #1 zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::private_const_view(zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>*, unsigned long) /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/view2.hpp:352 (zfp_omp+0x40374f)
    #2 test() [clone ._omp_fn.0] /workspaces/cpp-zfp/tests/zfp_omp.cpp:19 (zfp_omp+0x40374f)
    #3 gomp_thread_start <null> (libgomp.so.1+0x1d245)

  Previous write of size 8 at 0x7ffdedbdd8e8 by main thread:
    #0 zfp::internal::BlockStore<zfp::codec::zfp2<float>, zfp::index::hybrid4>::reference() /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store.hpp:132 (zfp_omp+0x40375f)
    #1 zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::private_const_view(zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>*, unsigned long) /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/view2.hpp:352 (zfp_omp+0x40375f)
    #2 test() [clone ._omp_fn.0] /workspaces/cpp-zfp/tests/zfp_omp.cpp:19 (zfp_omp+0x40375f)
    #3 GOMP_parallel <null> (libgomp.so.1+0x14575)
    #4 main /workspaces/cpp-zfp/tests/zfp_omp.cpp:31 (zfp_omp+0x4034c7)

  Location is stack of main thread.

  Location is global '<null>' at 0x000000000000 ([stack]+0x00000001f8e8)

  Thread T78 (tid=41286, running) created by main thread at:
    #0 pthread_create <null> (libtsan.so.0+0x61748)
    #1 gomp_team_start <null> (libgomp.so.1+0x1d8a3)
    #2 main /workspaces/cpp-zfp/tests/zfp_omp.cpp:31 (zfp_omp+0x4034c7)

SUMMARY: ThreadSanitizer: data race /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store.hpp:132 in zfp::internal::BlockStore<zfp::codec::zfp2<float>, zfp::index::hybrid4>::reference()
==================
==================
WARNING: ThreadSanitizer: data race (pid=40760)
  Write of size 1 at 0x7ffdedbdd940 by thread T78:
    #0 zfp::codec::zfp_base<2u, float>::set_thread_safety(bool) /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:102 (zfp_omp+0x403779)
    #1 zfp::internal::BlockStore<zfp::codec::zfp2<float>, zfp::index::hybrid4>::reference() /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store.hpp:133 (zfp_omp+0x403779)
    #2 zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::private_const_view(zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>*, unsigned long) /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/view2.hpp:352 (zfp_omp+0x403779)
    #3 test() [clone ._omp_fn.0] /workspaces/cpp-zfp/tests/zfp_omp.cpp:19 (zfp_omp+0x403779)
    #4 gomp_thread_start <null> (libgomp.so.1+0x1d245)

  Previous write of size 1 at 0x7ffdedbdd940 by main thread:
    [failed to restore the stack]

  Location is stack of main thread.

  Location is global '<null>' at 0x000000000000 ([stack]+0x00000001f940)

  Thread T78 (tid=41286, running) created by main thread at:
    #0 pthread_create <null> (libtsan.so.0+0x61748)
    #1 gomp_team_start <null> (libgomp.so.1+0x1d8a3)
    #2 main /workspaces/cpp-zfp/tests/zfp_omp.cpp:31 (zfp_omp+0x4034c7)

SUMMARY: ThreadSanitizer: data race /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:102 in zfp::codec::zfp_base<2u, float>::set_thread_safety(bool)
==================
==================
WARNING: ThreadSanitizer: data race (pid=40760)
  Read of size 8 at 0x7b0c00000030 by thread T78:
    #0 stream_clone /host/build-gcc/_deps/zfp-src/include/zfp/bitstream.inl:469 (libzfp.so.1+0x17e9e)
    #1 zfp::codec::zfp_base<2u, float>::clone_stream() const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:142 (zfp_omp+0x403f80)
    #2 zfp::codec::zfp_base<2u, float>::decode_block(unsigned long, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:165 (zfp_omp+0x403f80)
    #3 zfp::codec::zfp2<float>::decode_block(unsigned long, unsigned int, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:299 (zfp_omp+0x403f80)
    #4 zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::decode(unsigned long, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store2.hpp:107 (zfp_omp+0x403f80)
    #5 zfp::internal::BlockCache2<float, zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::line(unsigned long, unsigned long, bool) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/cache2.hpp:176 (zfp_omp+0x403f80)
    #6 zfp::internal::BlockCache2<float, zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::get(unsigned long, unsigned long) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/cache2.hpp:59 (zfp_omp+0x403f80)
    #7 zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::get(unsigned long, unsigned long) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/view2.hpp:405 (zfp_omp+0x403f80)
    #8 zfp::internal::dim2::const_handle<zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> > >::get() const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/handle2.hpp:28 (zfp_omp+0x403f80)
    #9 zfp::internal::dim2::const_reference<zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> > >::operator float() const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/reference2.hpp:19 (zfp_omp+0x403f80)
    #10 test() [clone ._omp_fn.0] /workspaces/cpp-zfp/tests/zfp_omp.cpp:24 (zfp_omp+0x403f80)
    #11 gomp_thread_start <null> (libgomp.so.1+0x1d245)

  Previous write of size 8 at 0x7b0c00000030 by main thread:
    [failed to restore the stack]

  Location is heap block of size 40 at 0x7b0c00000030 allocated by main thread:
    #0 malloc <null> (libtsan.so.0+0x32dd7)
    #1 stream_open /host/build-gcc/_deps/zfp-src/include/zfp/bitstream.inl:444 (libzfp.so.1+0x17dd5)
    #2 zfp::codec::zfp_base<2u, float>::open(void*, unsigned long) /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:60 (zfp_omp+0x4061db)
    #3 zfp::internal::BlockStore<zfp::codec::zfp2<float>, zfp::index::hybrid4>::compact() /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store.hpp:122 (zfp_omp+0x4061db)
    #4 zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::set(float const*, bool) /host/build-gcc/_deps/zfp-src/include/zfp/constarray2.hpp:224 (zfp_omp+0x4061db)
    #5 zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::const_array2(unsigned long, unsigned long, zfp_config const&, float const*, unsigned long) /host/build-gcc/_deps/zfp-src/include/zfp/constarray2.hpp:59 (zfp_omp+0x4061db)
    #6 test() /workspaces/cpp-zfp/tests/zfp_omp.cpp:12 (zfp_omp+0x4061db)
    #7 main /workspaces/cpp-zfp/tests/zfp_omp.cpp:31 (zfp_omp+0x4034c7)

  Thread T78 (tid=41286, running) created by main thread at:
    #0 pthread_create <null> (libtsan.so.0+0x61748)
    #1 gomp_team_start <null> (libgomp.so.1+0x1d8a3)
    #2 main /workspaces/cpp-zfp/tests/zfp_omp.cpp:31 (zfp_omp+0x4034c7)

SUMMARY: ThreadSanitizer: data race /host/build-gcc/_deps/zfp-src/include/zfp/bitstream.inl:469 in stream_clone
==================
==================
WARNING: ThreadSanitizer: data race on vptr (ctor/dtor vs virtual call) (pid=40760)
  Write of size 8 at 0x7ffdedbdd8d0 by main thread:
    #0 zfp::internal::BlockStore<zfp::codec::zfp2<float>, zfp::index::hybrid4>::~BlockStore() /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store.hpp:179 (zfp_omp+0x405fa9)
    #1 zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::~BlockStore2() /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store2.hpp:11 (zfp_omp+0x405fa9)
    #2 zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::~const_array2() /host/build-gcc/_deps/zfp-src/include/zfp/constarray2.hpp:70 (zfp_omp+0x405fa9)
    #3 test() /workspaces/cpp-zfp/tests/zfp_omp.cpp:29 (zfp_omp+0x405fa9)
    #4 main /workspaces/cpp-zfp/tests/zfp_omp.cpp:31 (zfp_omp+0x4034c7)

  Previous read of size 8 at 0x7ffdedbdd8d0 by thread T70:
    [failed to restore the stack]

  Location is stack of main thread.

  Location is global '<null>' at 0x000000000000 ([stack]+0x00000001f8d0)

  Thread T70 (tid=41278, running) created by main thread at:
    #0 pthread_create <null> (libtsan.so.0+0x61748)
    #1 gomp_team_start <null> (libgomp.so.1+0x1d8a3)
    #2 main /workspaces/cpp-zfp/tests/zfp_omp.cpp:31 (zfp_omp+0x4034c7)

SUMMARY: ThreadSanitizer: data race on vptr (ctor/dtor vs virtual call) /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store.hpp:179 in zfp::internal::BlockStore<zfp::codec::zfp2<float>, zfp::index::hybrid4>::~BlockStore()
==================
==================
WARNING: ThreadSanitizer: data race (pid=40760)
  Write of size 8 at 0x7b0c00000048 by main thread:
    #0 free <null> (libtsan.so.0+0x38c38)
    #1 stream_close /host/build-gcc/_deps/zfp-src/include/zfp/bitstream.inl:460 (libzfp.so.1+0x17e4d)
    #2 zfp::codec::zfp_base<2u, float>::close() /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:66 (zfp_omp+0x406024)
    #3 zfp::internal::BlockStore<zfp::codec::zfp2<float>, zfp::index::hybrid4>::free() /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store.hpp:227 (zfp_omp+0x406024)
    #4 zfp::internal::BlockStore<zfp::codec::zfp2<float>, zfp::index::hybrid4>::~BlockStore() /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store.hpp:179 (zfp_omp+0x406024)
    #5 zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::~BlockStore2() /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store2.hpp:11 (zfp_omp+0x406024)
    #6 zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::~const_array2() /host/build-gcc/_deps/zfp-src/include/zfp/constarray2.hpp:70 (zfp_omp+0x406024)
    #7 test() /workspaces/cpp-zfp/tests/zfp_omp.cpp:29 (zfp_omp+0x406024)
    #8 main /workspaces/cpp-zfp/tests/zfp_omp.cpp:31 (zfp_omp+0x4034c7)

  Previous read of size 8 at 0x7b0c00000048 by thread T43:
    #0 stream_clone /host/build-gcc/_deps/zfp-src/include/zfp/bitstream.inl:469 (libzfp.so.1+0x17e9e)
    #1 zfp::codec::zfp_base<2u, float>::clone_stream() const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:142 (zfp_omp+0x403f80)
    #2 zfp::codec::zfp_base<2u, float>::decode_block(unsigned long, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:165 (zfp_omp+0x403f80)
    #3 zfp::codec::zfp2<float>::decode_block(unsigned long, unsigned int, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:299 (zfp_omp+0x403f80)
    #4 zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::decode(unsigned long, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store2.hpp:107 (zfp_omp+0x403f80)
    #5 zfp::internal::BlockCache2<float, zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::line(unsigned long, unsigned long, bool) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/cache2.hpp:176 (zfp_omp+0x403f80)
    #6 zfp::internal::BlockCache2<float, zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::get(unsigned long, unsigned long) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/cache2.hpp:59 (zfp_omp+0x403f80)
    #7 zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::get(unsigned long, unsigned long) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/view2.hpp:405 (zfp_omp+0x403f80)
    #8 zfp::internal::dim2::const_handle<zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> > >::get() const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/handle2.hpp:28 (zfp_omp+0x403f80)
    #9 zfp::internal::dim2::const_reference<zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> > >::operator float() const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/reference2.hpp:19 (zfp_omp+0x403f80)
    #10 test() [clone ._omp_fn.0] /workspaces/cpp-zfp/tests/zfp_omp.cpp:24 (zfp_omp+0x403f80)
    #11 gomp_thread_start <null> (libgomp.so.1+0x1d245)

  Thread T43 (tid=41251, running) created by main thread at:
    #0 pthread_create <null> (libtsan.so.0+0x61748)
    #1 gomp_team_start <null> (libgomp.so.1+0x1d8a3)
    #2 main /workspaces/cpp-zfp/tests/zfp_omp.cpp:31 (zfp_omp+0x4034c7)

SUMMARY: ThreadSanitizer: data race (/lib64/libtsan.so.0+0x38c38) in free
==================
==================
WARNING: ThreadSanitizer: data race (pid=40760)
  Write of size 8 at 0x7b0c00000010 by main thread:
    #0 zfp_stream_set_bit_stream /host/build-gcc/_deps/zfp-src/src/zfp.c:747 (libzfp.so.1+0xf3de)
    #1 zfp::codec::zfp_base<2u, float>::close() /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:67 (zfp_omp+0x40603a)
    #2 zfp::internal::BlockStore<zfp::codec::zfp2<float>, zfp::index::hybrid4>::free() /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store.hpp:227 (zfp_omp+0x40603a)
    #3 zfp::internal::BlockStore<zfp::codec::zfp2<float>, zfp::index::hybrid4>::~BlockStore() /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store.hpp:179 (zfp_omp+0x40603a)
    #4 zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::~BlockStore2() /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store2.hpp:11 (zfp_omp+0x40603a)
    #5 zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::~const_array2() /host/build-gcc/_deps/zfp-src/include/zfp/constarray2.hpp:70 (zfp_omp+0x40603a)
    #6 test() /workspaces/cpp-zfp/tests/zfp_omp.cpp:29 (zfp_omp+0x40603a)
    #7 main /workspaces/cpp-zfp/tests/zfp_omp.cpp:31 (zfp_omp+0x4034c7)

  Previous read of size 8 at 0x7b0c00000010 by thread T29:
    #0 zfp::codec::zfp_base<2u, float>::clone_stream() const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:141 (zfp_omp+0x403f4d)
    #1 zfp::codec::zfp_base<2u, float>::decode_block(unsigned long, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:165 (zfp_omp+0x403f4d)
    #2 zfp::codec::zfp2<float>::decode_block(unsigned long, unsigned int, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:299 (zfp_omp+0x403f4d)
    #3 zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::decode(unsigned long, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store2.hpp:107 (zfp_omp+0x403f4d)
    #4 zfp::internal::BlockCache2<float, zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::line(unsigned long, unsigned long, bool) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/cache2.hpp:176 (zfp_omp+0x403f4d)
    #5 zfp::internal::BlockCache2<float, zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::get(unsigned long, unsigned long) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/cache2.hpp:59 (zfp_omp+0x403f4d)
    #6 zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::get(unsigned long, unsigned long) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/view2.hpp:405 (zfp_omp+0x403f4d)
    #7 zfp::internal::dim2::const_handle<zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> > >::get() const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/handle2.hpp:28 (zfp_omp+0x403f4d)
    #8 zfp::internal::dim2::const_reference<zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> > >::operator float() const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/reference2.hpp:19 (zfp_omp+0x403f4d)
    #9 test() [clone ._omp_fn.0] /workspaces/cpp-zfp/tests/zfp_omp.cpp:24 (zfp_omp+0x403f4d)
    #10 gomp_thread_start <null> (libgomp.so.1+0x1d245)

  Location is heap block of size 40 at 0x7b0c00000000 allocated by main thread:
    #0 malloc <null> (libtsan.so.0+0x32dd7)
    #1 zfp_stream_open /host/build-gcc/_deps/zfp-src/src/zfp.c:539 (libzfp.so.1+0xe58c)
    #2 zfp::codec::zfp_base<2u, float>::zfp_base() /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:21 (zfp_omp+0x40466a)
    #3 zfp::codec::zfp2<float>::zfp2() /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:286 (zfp_omp+0x40466a)
    #4 zfp::internal::BlockStore<zfp::codec::zfp2<float>, zfp::index::hybrid4>::BlockStore() /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store.hpp:175 (zfp_omp+0x40466a)
    #5 zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::BlockStore2(unsigned long, unsigned long, zfp_config const&) /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store2.hpp:21 (zfp_omp+0x40466a)
    #6 zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::const_array2(unsigned long, unsigned long, zfp_config const&, float const*, unsigned long) /host/build-gcc/_deps/zfp-src/include/zfp/constarray2.hpp:54 (zfp_omp+0x40466a)
    #7 test() /workspaces/cpp-zfp/tests/zfp_omp.cpp:12 (zfp_omp+0x40466a)
    #8 main /workspaces/cpp-zfp/tests/zfp_omp.cpp:31 (zfp_omp+0x4034c7)

  Thread T29 (tid=41237, running) created by main thread at:
    #0 pthread_create <null> (libtsan.so.0+0x61748)
    #1 gomp_team_start <null> (libgomp.so.1+0x1d8a3)
    #2 main /workspaces/cpp-zfp/tests/zfp_omp.cpp:31 (zfp_omp+0x4034c7)

SUMMARY: ThreadSanitizer: data race /host/build-gcc/_deps/zfp-src/src/zfp.c:747 in zfp_stream_set_bit_stream
==================
==================
WARNING: ThreadSanitizer: data race (pid=40760)
  Write of size 8 at 0x7b0c00000000 by main thread:
    #0 free <null> (libtsan.so.0+0x38c38)
    #1 zfp_stream_close /host/build-gcc/_deps/zfp-src/src/zfp.c:557 (libzfp.so.1+0xe656)
    #2 zfp::codec::zfp_base<2u, float>::~zfp_base() /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:31 (zfp_omp+0x406087)
    #3 zfp::codec::zfp2<float>::~zfp2() /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:286 (zfp_omp+0x406087)
    #4 zfp::internal::BlockStore<zfp::codec::zfp2<float>, zfp::index::hybrid4>::~BlockStore() /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store.hpp:179 (zfp_omp+0x406087)
    #5 zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::~BlockStore2() /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store2.hpp:11 (zfp_omp+0x406087)
    #6 zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::~const_array2() /host/build-gcc/_deps/zfp-src/include/zfp/constarray2.hpp:70 (zfp_omp+0x406087)
    #7 test() /workspaces/cpp-zfp/tests/zfp_omp.cpp:29 (zfp_omp+0x406087)
    #8 main /workspaces/cpp-zfp/tests/zfp_omp.cpp:31 (zfp_omp+0x4034c7)

  Previous read of size 8 at 0x7b0c00000000 by thread T29:
    #0 zfp::codec::zfp_base<2u, float>::clone_stream() const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:141 (zfp_omp+0x403f4d)
    #1 zfp::codec::zfp_base<2u, float>::decode_block(unsigned long, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:165 (zfp_omp+0x403f4d)
    #2 zfp::codec::zfp2<float>::decode_block(unsigned long, unsigned int, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/codec/zfpcodec.hpp:299 (zfp_omp+0x403f4d)
    #3 zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4>::decode(unsigned long, float*) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/store2.hpp:107 (zfp_omp+0x403f4d)
    #4 zfp::internal::BlockCache2<float, zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::line(unsigned long, unsigned long, bool) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/cache2.hpp:176 (zfp_omp+0x403f4d)
    #5 zfp::internal::BlockCache2<float, zfp::internal::BlockStore2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::get(unsigned long, unsigned long) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/cache2.hpp:59 (zfp_omp+0x403f4d)
    #6 zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> >::get(unsigned long, unsigned long) const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/view2.hpp:405 (zfp_omp+0x403f4d)
    #7 zfp::internal::dim2::const_handle<zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> > >::get() const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/handle2.hpp:28 (zfp_omp+0x403f4d)
    #8 zfp::internal::dim2::const_reference<zfp::internal::dim2::private_const_view<zfp::const_array2<float, zfp::codec::zfp2<float>, zfp::index::hybrid4> > >::operator float() const /host/build-gcc/_deps/zfp-src/include/zfp/internal/array/reference2.hpp:19 (zfp_omp+0x403f4d)
    #9 test() [clone ._omp_fn.0] /workspaces/cpp-zfp/tests/zfp_omp.cpp:24 (zfp_omp+0x403f4d)
    #10 gomp_thread_start <null> (libgomp.so.1+0x1d245)

  Thread T29 (tid=41237, running) created by main thread at:
    #0 pthread_create <null> (libtsan.so.0+0x61748)
    #1 gomp_team_start <null> (libgomp.so.1+0x1d8a3)
    #2 main /workspaces/cpp-zfp/tests/zfp_omp.cpp:31 (zfp_omp+0x4034c7)

SUMMARY: ThreadSanitizer: data race (/lib64/libtsan.so.0+0x38c38) in free
==================
ThreadSanitizer: reported 7 warnings
lindstro commented 1 year ago

Your OpenMP example is expected to fail as you're spawning multiple threads in the inner foor loop that are accessing the same private view. zfp's private views allow you to access the same array on N threads by creating N private views of the array, with one view per thread. See the example I provided in #198.

Returning to your TBB example, I missed the -c at the end of the line and thought this was the link line when I saw the references to .o files. Now, I guess we don't know to what extent TBB and OpenMP threads can be mixed; maybe what you're trying to do simply isn't supported. One guarantee we need is that each TBB thread gets assigned a thread-local private view, and I don't know enough about TBB to tell if that's the case (though I'd be surprised if that wasn't the case). From a performance standpoint, I see no reason to create a new private view in the outer loop; I would just pull that outside the i loop. Still, that should have no impact on the data race.

I think for now, you're stuck with OpenMP until we can research more whether and how to mix thread libraries. But it would surely be cleaner to avoid any such mixing and to use TBB primitives for any critical regions in zfp entered by TBB threads.

lindstro commented 1 year ago

Your OpenMP example is expected to fail as you're spawning multiple threads in the inner foor loop

My bad--I missed that the inner loop does not have a parallel directive, so your code is fine. I can confirm a reported data race with -fsanitize=thread as well as with my example from #198, so I believe more investigation is needed on our side. I will mark this as a bug for now.

lindstro commented 1 year ago

Doing some more research, it appears that ThreadSanitizer is to blame. It throws false positives on OpenMP code that clearly is thread safe. For instance, on the example from #198, ThreadSanitizer flags a line a code within the critical region as a data race, which obviously is nonsensical.

See also these links for even simpler examples of false positives: https://stackoverflow.com/questions/33004809/can-i-use-thread-sanitizer-for-openmp-programs https://stackoverflow.com/questions/59913536/race-condition-in-openmp-outside-parallel-block-threadsanitizer-false-positiv

Until it can be shown that there truly are some issues with zfp thread safety (e.g., segmentation faults, memory corruption, unexpected/inconsistent values when accessing arrays through private views, etc.), I will assume that the issue is with ThreadSanitizer and not with zfp.

aavbsouza commented 1 year ago

Hello @lindstro the original motivation for the issue was a segmentation fault on a code when running in parallel (with oneapi::tbb), the snippet used on this thread is based on a pattern that appear on this code. Others sections of this same code I was able to run with thread sanitize enabled without generating false positives. However the section in that code that is similar to the above snippet when executed with the thread sanitize enabled presents the same kind of errors show on this thread. And the problem only manifests if the parallelism is enabled.

lindstro commented 1 year ago

From the discussion above, we do expect issues when using zfp with TBB if the code is not compiled with OpenMP enabled. Do the segmentation faults persist if you compile with -fopenmp? Given that ThreadSanitizer gives false positives for the most basic OpenMP example, I think we cannot rely on it to diagnose data races. To make further progress, I would need a complete--but minimal--code example that invokes provably erroneous behavior.

lindstro commented 1 year ago

@aavbsouza Are you still having issues with zfp's private views and OpenMP? From our previous discussion, I think TBB is off the table, while the views appear to be working correctly when compiled with OpenMP support. If this is no longer an issue, then I'd like to go ahead and close it.

lindstro commented 1 year ago

I'm being assured by colleagues at Intel that it's safe to mix TBB and OpenMP, so all should work as long as the application is built with -fopenmp. I'm closing this issue for now, but feel free to reopen in case this issue is not resolved.