search

eclipse-iceoryx / iceoryx

Eclipse iceoryx™ - true zero-copy inter-process-communication
https://iceoryx.io
Apache License 2.0
1.57k stars 373 forks source link

Many tsan warnings during the message delivery , do we have a plan to eliminate it? #2046

Open tang-qh opened 8 months ago

tang-qh commented 8 months ago

Required information

Operating system: Ubuntu 20.04 LTS

Compiler version: GCC 7.5.0

Eclipse iceoryx version: v2.90.0

Observed result or behaviour: There are many tsan warnings during the message delivery process: `================== WARNING: ThreadSanitizer: data race (pid=174325) Read of size 8 at 0x7f05c4085a28 by thread T7 (mutexes: write M34623):

0 iox::rp::BaseRelativePointer::computeRawPtr() const external/iceoryx/iceoryx_hoofs/source/relocatable_pointer/base_relative_pointer.cpp:193 (libioxposh.so+0x00000019aaf8)

#1 iox::rp::RelativePointer<iox::mepoo::ChunkHeader>::get() const bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_hoofs/_virtual_includes/iceoryx_hoofs/iceoryx_hoofs/internal/relocatable_pointer/relative_pointer.inl:70 (libioxposh.so+0x000000109912)
#2 iox::mepoo::SharedChunk::getChunkHeader() const external/iceoryx/iceoryx_posh/source/mepoo/shared_chunk.cpp:132 (libioxposh.so+0x00000010979b)
#3 iox::popo::ChunkQueuePopper<iox::popo::ChunkQueueData<iox::DefaultChunkQueueConfig, iox::popo::ThreadSafePolicy> >::tryPop() bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/building_blocks/chunk_queue_popper.inl:58 (libioxposh.so+0x00000013b3c2)
#4 iox::popo::ChunkReceiver<iox::popo::ChunkReceiverData<256u, iox::popo::ChunkQueueData<iox::DefaultChunkQueueConfig, iox::popo::ThreadSafePolicy> > >::tryGet() bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/building_blocks/chunk_receiver.inl:78 (libioxposh.so+0x00000013af21)
#5 iox::popo::SubscriberPortUser::tryGetChunk() external/iceoryx/iceoryx_posh/source/popo/ports/subscriber_port_user.cpp:69 (libioxposh.so+0x00000013ab48)
#6 iox::popo::BaseSubscriber<iox::popo::SubscriberPortUser>::takeChunk() bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/base_subscriber.inl:93 (libados_cyber.so+0x00000021b4bc)
#7 iox::popo::UntypedSubscriberImpl<iox::popo::BaseSubscriber<iox::popo::SubscriberPortUser> >::take() bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/untyped_subscriber_impl.inl:37 (libados_cyber.so+0x000000219457)
#8 Listener Callback Func
#9 iox::popo::internal::TranslateAndCallTypelessCallback<iox::popo::UntypedSubscriber, apollo::cyber::transport::IceoryxDispatcher::ReaderHolder>::call(void*, void*, void (*)()) bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/notification_callback.inl:31 (libados_cyber.so+0x00000021c1df)
#10 iox::popo::internal::Event_t::executeCallback() external/iceoryx/iceoryx_posh/source/popo/listener.cpp:48 (libioxposh.so+0x00000010f258)
#11 iox::popo::ListenerImpl<1024ul>::threadLoop() bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/listener.inl:175 (libioxposh.so+0x0000001106b4)
#12 void std::__invoke_impl<void, void (iox::popo::ListenerImpl<1024ul>::*)(), iox::popo::ListenerImpl<1024ul>*>(std::__invoke_memfun_deref, void (iox::popo::ListenerImpl<1024ul>::*&&)(), iox::popo::ListenerImpl<1024ul>*&&) external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:73 (libioxposh.so+0x000000111007)
#13 std::__invoke_result<void (iox::popo::ListenerImpl<1024ul>::*)(), iox::popo::ListenerImpl<1024ul>*>::type std::__invoke<void (iox::popo::ListenerImpl<1024ul>::*)(), iox::popo::ListenerImpl<1024ul>*>(void (iox::popo::ListenerImpl<1024ul>::*&&)(), iox::popo::ListenerImpl<1024ul>*&&) external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:95 (libioxposh.so+0x000000110731)
#14 decltype (__invoke((_S_declval<0ul>)(), (_S_declval<1ul>)())) std::thread::_Invoker<std::tuple<void (iox::popo::ListenerImpl<1024ul>::*)(), iox::popo::ListenerImpl<1024ul>*> >::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:234 (libioxposh.so+0x000000111d18)
#15 std::thread::_Invoker<std::tuple<void (iox::popo::ListenerImpl<1024ul>::*)(), iox::popo::ListenerImpl<1024ul>*> >::operator()() external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:243 (libioxposh.so+0x000000111cb9)
#16 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (iox::popo::ListenerImpl<1024ul>::*)(), iox::popo::ListenerImpl<1024ul>*> > >::_M_run() external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:186 (libioxposh.so+0x000000111c6e)
#17 <null> <null> (libstdc++.so.6+0x0000000d6de3)

Previous write of size 8 at 0x7f05c4085a28 by thread T35: [failed to restore the stack]

SUMMARY: ThreadSanitizer: data race external/iceoryx/iceoryx_hoofs/source/relocatable_pointer/base_relative_pointer.cpp:193 in iox::rp::BaseRelativePointer::computeRawPtr() const ==================`

`================== WARNING: ThreadSanitizer: data race (pid=174325) Atomic write of size 8 at 0x7f05c4085a30 by thread T7 (mutexes: write M34623):

0 __tsan_atomic64_fetch_add /root/objdir/../gcc-7.5.0/libsanitizer/tsan/tsan_interface_atomic.cc:611 (node_launcher+0x0000004ff050)

#1 std::__atomic_base<unsigned long>::fetch_add(unsigned long, std::memory_order) external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/atomic_base.h:514 (libioxposh.so+0x000000109319)
#2 iox::mepoo::SharedChunk::incrementReferenceCounter() external/iceoryx/iceoryx_posh/source/mepoo/shared_chunk.cpp:49 (libioxposh.so+0x000000109319)
#3 iox::mepoo::SharedChunk::operator=(iox::mepoo::SharedChunk const&) external/iceoryx/iceoryx_posh/source/mepoo/shared_chunk.cpp:75 (libioxposh.so+0x0000001094e6)
#4 iox::mepoo::SharedChunk::SharedChunk(iox::mepoo::SharedChunk const&) external/iceoryx/iceoryx_posh/source/mepoo/shared_chunk.cpp:32 (libioxposh.so+0x00000010921b)
#5 void iox::cxx::optional<iox::mepoo::SharedChunk>::construct_value<iox::mepoo::SharedChunk&>(iox::mepoo::SharedChunk&) bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_hoofs/_virtual_includes/iceoryx_hoofs/iceoryx_hoofs/internal/cxx/optional.inl:291 (libioxposh.so+0x000000121f7e)
#6 iox::mepoo::SharedChunk& iox::cxx::optional<iox::mepoo::SharedChunk>::emplace<iox::mepoo::SharedChunk&>(iox::mepoo::SharedChunk&) bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_hoofs/_virtual_includes/iceoryx_hoofs/iceoryx_hoofs/internal/cxx/optional.inl:240 (libioxposh.so+0x000000121a4c)
#7 iox::cxx::optional<iox::mepoo::SharedChunk> iox::cxx::make_optional<iox::mepoo::SharedChunk, iox::mepoo::SharedChunk&>(iox::mepoo::SharedChunk&) bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_hoofs/_virtual_includes/iceoryx_hoofs/iceoryx_hoofs/internal/cxx/optional.inl:306 (libioxposh.so+0x000000120cd8)
#8 iox::popo::ChunkQueuePopper<iox::popo::ChunkQueueData<iox::DefaultChunkQueueConfig, iox::popo::ThreadSafePolicy> >::tryPop() bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/building_blocks/chunk_queue_popper.inl:67 (libioxposh.so+0x00000013b475)
#9 iox::popo::ChunkReceiver<iox::popo::ChunkReceiverData<256u, iox::popo::ChunkQueueData<iox::DefaultChunkQueueConfig, iox::popo::ThreadSafePolicy> > >::tryGet() bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/building_blocks/chunk_receiver.inl:78 (libioxposh.so+0x00000013af21)
#10 iox::popo::SubscriberPortUser::tryGetChunk() external/iceoryx/iceoryx_posh/source/popo/ports/subscriber_port_user.cpp:69 (libioxposh.so+0x00000013ab48)
#11 iox::popo::BaseSubscriber<iox::popo::SubscriberPortUser>::takeChunk() bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/base_subscriber.inl:93 (libados_cyber.so+0x00000021b4bc)
#12 iox::popo::UntypedSubscriberImpl<iox::popo::BaseSubscriber<iox::popo::SubscriberPortUser> >::take() bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/untyped_subscriber_impl.inl:37 (libados_cyber.so+0x000000219457)
#13 Listener Callback Func
#14 iox::popo::internal::TranslateAndCallTypelessCallback<iox::popo::UntypedSubscriber, apollo::cyber::transport::IceoryxDispatcher::ReaderHolder>::call(void*, void*, void (*)()) bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/notification_callback.inl:31 (libados_cyber.so+0x00000021c1df)
#15 iox::popo::internal::Event_t::executeCallback() external/iceoryx/iceoryx_posh/source/popo/listener.cpp:48 (libioxposh.so+0x00000010f258)
#16 iox::popo::ListenerImpl<1024ul>::threadLoop() bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/listener.inl:175 (libioxposh.so+0x0000001106b4)
#17 void std::__invoke_impl<void, void (iox::popo::ListenerImpl<1024ul>::*)(), iox::popo::ListenerImpl<1024ul>*>(std::__invoke_memfun_deref, void (iox::popo::ListenerImpl<1024ul>::*&&)(), iox::popo::ListenerImpl<1024ul>*&&) external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:73 (libioxposh.so+0x000000111007)
#18 std::__invoke_result<void (iox::popo::ListenerImpl<1024ul>::*)(), iox::popo::ListenerImpl<1024ul>*>::type std::__invoke<void (iox::popo::ListenerImpl<1024ul>::*)(), iox::popo::ListenerImpl<1024ul>*>(void (iox::popo::ListenerImpl<1024ul>::*&&)(), iox::popo::ListenerImpl<1024ul>*&&) external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:95 (libioxposh.so+0x000000110731)
#19 decltype (__invoke((_S_declval<0ul>)(), (_S_declval<1ul>)())) std::thread::_Invoker<std::tuple<void (iox::popo::ListenerImpl<1024ul>::*)(), iox::popo::ListenerImpl<1024ul>*> >::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:234 (libioxposh.so+0x000000111d18)
#20 std::thread::_Invoker<std::tuple<void (iox::popo::ListenerImpl<1024ul>::*)(), iox::popo::ListenerImpl<1024ul>*> >::operator()() external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:243 (libioxposh.so+0x000000111cb9)
#21 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (iox::popo::ListenerImpl<1024ul>::*)(), iox::popo::ListenerImpl<1024ul>*> > >::_M_run() external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:186 (libioxposh.so+0x000000111c6e)
#22 <null> <null> (libstdc++.so.6+0x0000000d6de3)

Previous write of size 8 at 0x7f05c4085a30 by thread T35: [failed to restore the stack]

SUMMARY: ThreadSanitizer: data race external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/atomic_base.h:514 in std::__atomic_base::fetch_add(unsigned long, std::memory_order) ==================`

Expected result or behaviour: no tsan warnings during the message delivery

Conditions where it occurred / Performed steps: compile with tsan and run a cross process transmission

elBoberido commented 8 months ago

@tang-qh there is #692 but currently there is no-one actively working on it. Contributions are welcome :)