search

eProsima / Fast-DDS

The most complete DDS - Proven: Plenty of success cases. Looking for commercial support? Contact info@eprosima.com
https://eprosima.com
Apache License 2.0
2.22k stars 778 forks source link

Segmentation fault (data race) while endpoints static discovery #4029

Open lexamor opened 1 year ago

lexamor commented 1 year ago

Is there an already existing issue for this?

Expected behavior

System should allow static discovery with reasonable amount of endpoints

Current behavior

Application crashes while endpoints static discovery for multiple participants/processes (above ~10, can depend on the system performance capabilities)

Steps to reproduce

  1. Enable endpoint static discovery for the DomanParticipantQoS.
  2. Set static EDP config with some amount of endpoints (~20), all of them stuck to the same topic/datatype, userIds are unique per process.
  3. Instantiate DomainParticipant (one per process).
  4. Simultaneously launch multiple processes with the described configuration.
  5. Segmentation fault happens after a few seconds of execution in some processes.

Fast DDS version/commit

2.9.1

Platform/Architecture

Ubuntu Focal 20.04 amd64, Ubuntu Focal 20.04 arm64

Transport layer

UDPv4, Shared Memory Transport (SHM)

Additional context

It seems that some data race is happened during the static endpoints processing because of lack of the implementation of the syncronization mechanizms.

Following patch locally (completely) eliminates / fixes issue for me, but it is a bit hard to prove it completely because of debugging complexity.

---
 src/cpp/rtps/builtin/discovery/endpoint/EDPStatic.cpp | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/src/cpp/rtps/builtin/discovery/endpoint/EDPStatic.cpp b/src/cpp/rtps/builtin/discovery/endpoint/EDPStatic.cpp
index 1a65b971e..2d3d6911a 100644
--- a/src/cpp/rtps/builtin/discovery/endpoint/EDPStatic.cpp
+++ b/src/cpp/rtps/builtin/discovery/endpoint/EDPStatic.cpp
@@ -433,6 +433,8 @@ void EDPStatic::assignRemoteEndpoints(
         persistence_guid.entityId.value[2] = pdata.m_userData.at(17);
     }

+    mp_PDP->getMutex()->lock();
+
     for (ParameterPropertyList_t::const_iterator pit = pdata.m_properties.begin();
             pit != pdata.m_properties.end(); ++pit)
     {
@@ -486,6 +488,8 @@ void EDPStatic::assignRemoteEndpoints(

         }
     }
+
+    mp_PDP->getMutex()->unlock();
 }

 bool EDPStatic::newRemoteReader(
--

Segmentation fault backtrace:

0x0000000000460172: eprosima::fastdds::dds::ParameterProperty_t::element_size(unsigned char const*) at ParameterTypes.hpp:1029
0x0000000000438432: eprosima::fastrtps::rtps::PDPSimple::notifyAboveRemoteEndpoints(eprosima::fastrtps::rtps::ParticipantProxyData const&) at PDPSimple.cpp:452
0x00000000002a05f1: eprosima::fastrtps::rtps::security::SecurityManager::discovered_participant(eprosima::fastrtps::rtps::ParticipantProxyData const&) at SecurityManager.cpp:581
0x0000000000438880: eprosima::fastrtps::rtps::PDPSimple::assignRemoteEndpoints(eprosima::fastrtps::rtps::ParticipantProxyData*) at PDPSimple.cpp:420
0x000000000043d0ba: eprosima::fastrtps::rtps::PDPListener::onNewCacheChangeAdded(eprosima::fastrtps::rtps::RTPSReader*, eprosima::fastrtps::rtps::CacheChange_t const*) at PDPListener.cpp:168
0x000000000037b363: eprosima::fastrtps::rtps::StatelessReader::change_received(eprosima::fastrtps::rtps::CacheChange_t*) at StatelessReader.cpp:329
0x000000000037fd8c: eprosima::fastrtps::rtps::StatelessReader::processDataMsg(eprosima::fastrtps::rtps::CacheChange_t*) at StatelessReader.cpp:557
0x000000000058f860: eprosima::fastrtps::rtps::MessageReceiver::process_data_message_without_security(eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&) at MessageReceiver.cpp:664
0x000000000059a719: eprosima::fastrtps::rtps::MessageReceiver::processCDRMsg(eprosima::fastrtps::rtps::Locator_t const&, eprosima::fastrtps::rtps::Locator_t const&, eprosima::fastrtps::rtps::CDRMessage_t*) at MessageReceiver.cpp:416
0x00000000003961ec: eprosima::fastrtps::rtps::ReceiverResource::OnDataReceived(unsigned char const*, unsigned int, eprosima::fastrtps::rtps::Locator_t const&, eprosima::fastrtps::rtps::Locator_t const&) at ReceiverResource.cpp:135
0x00000000003f8268: std::__atomic_base<bool>::load(std::memory_order) const at atomic_base.h:419
0x00000000003f87aa: std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> > >::_M_run() at thread:195

ThreadSanitizer report:

WARNING: ThreadSanitizer: data race (pid=511710)
  Write of size 4 at 0x7b60000316fc by thread T4 (mutexes: write M372526840276845008, write M361830636543018256, write M423755457836876912):
    #0 eprosima::fastrtps::rtps::SerializedPayload_t::copy(eprosima::fastrtps::rtps::SerializedPayload_t const*, bool) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/include/fastdds/rtps/common/SerializedPayload.h:113 (static+0x2fde8c)
    #1 eprosima::fastdds::dds::ParameterPropertyList_t::operator=(eprosima::fastdds::dds::ParameterPropertyList_t const&) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/include/fastdds/dds/core/policy/ParameterTypes.hpp:1343 (static+0x2fde8c)
    #2 eprosima::fastrtps::rtps::ParticipantProxyData::updateData(eprosima::fastrtps::rtps::ParticipantProxyData&) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/builtin/data/ParticipantProxyData.cpp:728 (static+0x2fde8c)
    #3 eprosima::fastrtps::rtps::PDPListener::onNewCacheChangeAdded(eprosima::fastrtps::rtps::RTPSReader*, eprosima::fastrtps::rtps::CacheChange_t const*) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/builtin/discovery/participant/PDPListener.cpp:173 (static+0x6f6fd6)
    #4 eprosima::fastrtps::rtps::StatelessReader::change_received(eprosima::fastrtps::rtps::CacheChange_t*) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/reader/StatelessReader.cpp:329 (static+0x591c0a)
    #5 eprosima::fastrtps::rtps::StatelessReader::processDataMsg(eprosima::fastrtps::rtps::CacheChange_t*) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/reader/StatelessReader.cpp:557 (static+0x599655)
    #6 operator() /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/messages/MessageReceiver.cpp:200 (static+0x942119)
    #7 findAllReaders<eprosima::fastrtps::rtps::MessageReceiver::process_data_message_without_security(const eprosima::fastrtps::rtps::EntityId_t&, eprosima::fastrtps::rtps::CacheChange_t&)::<lambda(eprosima::fastrtps::rtps::RTPSReader*)> > /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/messages/MessageReceiver.cpp:666 (static+0x942119)
    #8 eprosima::fastrtps::rtps::MessageReceiver::process_data_message_without_security(eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/messages/MessageReceiver.cpp:203 (static+0x942119)
    #9 void std::_Bind<void (eprosima::fastrtps::rtps::MessageReceiver::*(eprosima::fastrtps::rtps::MessageReceiver*, std::_Placeholder<1>, std::_Placeholder<2>))(eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&)>::operator()<eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&, void>(eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&) /usr/include/c++/9/bits/invoke.h:73 (static+0x955c3b)
    #10 std::_Function_handler<void (eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&), std::_Bind<void (eprosima::fastrtps::rtps::MessageReceiver::*(eprosima::fastrtps::rtps::MessageReceiver*, std::_Placeholder<1>, std::_Placeholder<2>))(eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&)> >::_M_invoke(std::_Any_data const&, eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&) /usr/include/c++/9/bits/std_function.h:300 (static+0x955c3b)
    #11 std::function<void (eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&)>::operator()(eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&) const /usr/include/c++/9/bits/std_function.h:688 (static+0x9518be)
    #12 eprosima::fastrtps::rtps::MessageReceiver::proc_Submsg_Data(eprosima::fastrtps::rtps::CDRMessage_t*, eprosima::fastrtps::rtps::SubmessageHeader_t*) const /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/messages/MessageReceiver.cpp:839 (static+0x9518be)
    #13 eprosima::fastrtps::rtps::MessageReceiver::processCDRMsg(eprosima::fastrtps::rtps::Locator_t const&, eprosima::fastrtps::rtps::Locator_t const&, eprosima::fastrtps::rtps::CDRMessage_t*) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/messages/MessageReceiver.cpp:416 (static+0x955593)
    #14 eprosima::fastrtps::rtps::ReceiverResource::OnDataReceived(unsigned char const*, unsigned int, eprosima::fastrtps::rtps::Locator_t const&, eprosima::fastrtps::rtps::Locator_t const&) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/network/ReceiverResource.cpp:132 (static+0x5c1b3d)
    #15 eprosima::fastdds::rtps::SharedMemChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/transport/shared_mem/SharedMemChannelResource.hpp:146 (static+0x3fc6e3)
    #16 void std::__invoke_impl<void, void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t>(std::__invoke_memfun_deref, void (eprosima::fastdds::rtps::SharedMemChannelResource::*&&)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*&&, eprosima::fastrtps::rtps::Locator_t&&) /usr/include/c++/9/bits/invoke.h:73 (static+0x3c0190)
    #17 std::__invoke_result<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t>::type std::__invoke<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t>(void (eprosima::fastdds::rtps::SharedMemChannelResource::*&&)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*&&, eprosima::fastrtps::rtps::Locator_t&&) /usr/include/c++/9/bits/invoke.h:95 (static+0x3c0190)
    #18 void std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) /usr/include/c++/9/thread:244 (static+0x3c0190)
    #19 std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::operator()() /usr/include/c++/9/thread:251 (static+0x3c0190)
    #20 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> > >::_M_run() /usr/include/c++/9/thread:195 (static+0x3c0190)
    #21 <null> <null> (libstdc++.so.6+0xd6df3)

  Previous read of size 4 at 0x7b60000316fc by thread T3 (mutexes: write M372245365300134160):
    #0 eprosima::fastrtps::rtps::EDPStatic::assignRemoteEndpoints(eprosima::fastrtps::rtps::ParticipantProxyData const&) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/builtin/discovery/endpoint/EDPStatic.cpp:437 (static+0x72ff9f)
    #1 eprosima::fastrtps::rtps::PDPSimple::notifyAboveRemoteEndpoints(eprosima::fastrtps::rtps::ParticipantProxyData const&) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/builtin/discovery/participant/PDPSimple.cpp:449 (static+0x6ee904)
    #2 eprosima::fastrtps::rtps::security::SecurityManager::discovered_participant(eprosima::fastrtps::rtps::ParticipantProxyData const&) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/security/SecurityManager.cpp:581 (static+0x423c78)
    #3 eprosima::fastrtps::rtps::PDPSimple::assignRemoteEndpoints(eprosima::fastrtps::rtps::ParticipantProxyData*) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/builtin/discovery/participant/PDPSimple.cpp:415 (static+0x6eeedc)
    #4 eprosima::fastrtps::rtps::PDPListener::onNewCacheChangeAdded(eprosima::fastrtps::rtps::RTPSReader*, eprosima::fastrtps::rtps::CacheChange_t const*) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/builtin/discovery/participant/PDPListener.cpp:168 (static+0x6f74cd)
    #5 eprosima::fastrtps::rtps::StatelessReader::change_received(eprosima::fastrtps::rtps::CacheChange_t*) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/reader/StatelessReader.cpp:329 (static+0x591c0a)
    #6 eprosima::fastrtps::rtps::StatelessReader::processDataMsg(eprosima::fastrtps::rtps::CacheChange_t*) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/reader/StatelessReader.cpp:557 (static+0x599655)
    #7 operator() /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/messages/MessageReceiver.cpp:200 (static+0x942119)
    #8 findAllReaders<eprosima::fastrtps::rtps::MessageReceiver::process_data_message_without_security(const eprosima::fastrtps::rtps::EntityId_t&, eprosima::fastrtps::rtps::CacheChange_t&)::<lambda(eprosima::fastrtps::rtps::RTPSReader*)> > /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/messages/MessageReceiver.cpp:666 (static+0x942119)
    #9 eprosima::fastrtps::rtps::MessageReceiver::process_data_message_without_security(eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/messages/MessageReceiver.cpp:203 (static+0x942119)
    #10 void std::_Bind<void (eprosima::fastrtps::rtps::MessageReceiver::*(eprosima::fastrtps::rtps::MessageReceiver*, std::_Placeholder<1>, std::_Placeholder<2>))(eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&)>::operator()<eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&, void>(eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&) /usr/include/c++/9/bits/invoke.h:73 (static+0x955c3b)
    #11 std::_Function_handler<void (eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&), std::_Bind<void (eprosima::fastrtps::rtps::MessageReceiver::*(eprosima::fastrtps::rtps::MessageReceiver*, std::_Placeholder<1>, std::_Placeholder<2>))(eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&)> >::_M_invoke(std::_Any_data const&, eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&) /usr/include/c++/9/bits/std_function.h:300 (static+0x955c3b)
    #12 std::function<void (eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&)>::operator()(eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&) const /usr/include/c++/9/bits/std_function.h:688 (static+0x9518be)
    #13 eprosima::fastrtps::rtps::MessageReceiver::proc_Submsg_Data(eprosima::fastrtps::rtps::CDRMessage_t*, eprosima::fastrtps::rtps::SubmessageHeader_t*) const /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/messages/MessageReceiver.cpp:839 (static+0x9518be)
    #14 eprosima::fastrtps::rtps::MessageReceiver::processCDRMsg(eprosima::fastrtps::rtps::Locator_t const&, eprosima::fastrtps::rtps::Locator_t const&, eprosima::fastrtps::rtps::CDRMessage_t*) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/messages/MessageReceiver.cpp:416 (static+0x955593)
    #15 eprosima::fastrtps::rtps::ReceiverResource::OnDataReceived(unsigned char const*, unsigned int, eprosima::fastrtps::rtps::Locator_t const&, eprosima::fastrtps::rtps::Locator_t const&) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/network/ReceiverResource.cpp:132 (static+0x5c1b3d)
    #16 eprosima::fastdds::rtps::SharedMemChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/transport/shared_mem/SharedMemChannelResource.hpp:146 (static+0x3fc6e3)
    #17 void std::__invoke_impl<void, void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t>(std::__invoke_memfun_deref, void (eprosima::fastdds::rtps::SharedMemChannelResource::*&&)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*&&, eprosima::fastrtps::rtps::Locator_t&&) /usr/include/c++/9/bits/invoke.h:73 (static+0x3c0190)
    #18 std::__invoke_result<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t>::type std::__invoke<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t>(void (eprosima::fastdds::rtps::SharedMemChannelResource::*&&)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*&&, eprosima::fastrtps::rtps::Locator_t&&) /usr/include/c++/9/bits/invoke.h:95 (static+0x3c0190)
    #19 void std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) /usr/include/c++/9/thread:244 (static+0x3c0190)
    #20 std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::operator()() /usr/include/c++/9/thread:251 (static+0x3c0190)
    #21 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> > >::_M_run() /usr/include/c++/9/thread:195 (static+0x3c0190)
    #22 <null> <null> (libstdc++.so.6+0xd6df3)

  Location is heap block of size 928 at 0x7b6000031400 allocated by thread T3:
    #0 operator new(unsigned long) ../../../../src/libsanitizer/tsan/tsan_new_delete.cpp:64 (libtsan.so.0+0x8bc72)
    #1 eprosima::fastrtps::rtps::PDP::add_participant_proxy_data(eprosima::fastrtps::rtps::GUID_t const&, bool, eprosima::fastrtps::rtps::ParticipantProxyData const*) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/builtin/discovery/participant/PDP.cpp:201 (static+0x6e47bd)
    #2 eprosima::fastrtps::rtps::PDPSimple::createParticipantProxyData(eprosima::fastrtps::rtps::ParticipantProxyData const&, eprosima::fastrtps::rtps::GUID_t const&) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/builtin/discovery/participant/PDPSimple.cpp:203 (static+0x6eeaa2)
    #3 eprosima::fastrtps::rtps::PDPListener::onNewCacheChangeAdded(eprosima::fastrtps::rtps::RTPSReader*, eprosima::fastrtps::rtps::CacheChange_t const*) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/builtin/discovery/participant/PDPListener.cpp:134 (static+0x6f72aa)
    #4 eprosima::fastrtps::rtps::StatelessReader::change_received(eprosima::fastrtps::rtps::CacheChange_t*) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/reader/StatelessReader.cpp:329 (static+0x591c0a)
    #5 eprosima::fastrtps::rtps::StatelessReader::processDataMsg(eprosima::fastrtps::rtps::CacheChange_t*) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/reader/StatelessReader.cpp:557 (static+0x599655)
    #6 operator() /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/messages/MessageReceiver.cpp:200 (static+0x942119)
    #7 findAllReaders<eprosima::fastrtps::rtps::MessageReceiver::process_data_message_without_security(const eprosima::fastrtps::rtps::EntityId_t&, eprosima::fastrtps::rtps::CacheChange_t&)::<lambda(eprosima::fastrtps::rtps::RTPSReader*)> > /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/messages/MessageReceiver.cpp:666 (static+0x942119)
    #8 eprosima::fastrtps::rtps::MessageReceiver::process_data_message_without_security(eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/messages/MessageReceiver.cpp:203 (static+0x942119)
    #9 void std::_Bind<void (eprosima::fastrtps::rtps::MessageReceiver::*(eprosima::fastrtps::rtps::MessageReceiver*, std::_Placeholder<1>, std::_Placeholder<2>))(eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&)>::operator()<eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&, void>(eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&) /usr/include/c++/9/bits/invoke.h:73 (static+0x955c3b)
    #10 std::_Function_handler<void (eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&), std::_Bind<void (eprosima::fastrtps::rtps::MessageReceiver::*(eprosima::fastrtps::rtps::MessageReceiver*, std::_Placeholder<1>, std::_Placeholder<2>))(eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&)> >::_M_invoke(std::_Any_data const&, eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&) /usr/include/c++/9/bits/std_function.h:300 (static+0x955c3b)
    #11 std::function<void (eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&)>::operator()(eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&) const /usr/include/c++/9/bits/std_function.h:688 (static+0x9518be)
    #12 eprosima::fastrtps::rtps::MessageReceiver::proc_Submsg_Data(eprosima::fastrtps::rtps::CDRMessage_t*, eprosima::fastrtps::rtps::SubmessageHeader_t*) const /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/messages/MessageReceiver.cpp:839 (static+0x9518be)
    #13 eprosima::fastrtps::rtps::MessageReceiver::processCDRMsg(eprosima::fastrtps::rtps::Locator_t const&, eprosima::fastrtps::rtps::Locator_t const&, eprosima::fastrtps::rtps::CDRMessage_t*) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/messages/MessageReceiver.cpp:416 (static+0x955593)
    #14 eprosima::fastrtps::rtps::ReceiverResource::OnDataReceived(unsigned char const*, unsigned int, eprosima::fastrtps::rtps::Locator_t const&, eprosima::fastrtps::rtps::Locator_t const&) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/network/ReceiverResource.cpp:132 (static+0x5c1b3d)
    #15 eprosima::fastdds::rtps::SharedMemChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/transport/shared_mem/SharedMemChannelResource.hpp:146 (static+0x3fc6e3)
    #16 void std::__invoke_impl<void, void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t>(std::__invoke_memfun_deref, void (eprosima::fastdds::rtps::SharedMemChannelResource::*&&)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*&&, eprosima::fastrtps::rtps::Locator_t&&) /usr/include/c++/9/bits/invoke.h:73 (static+0x3c0190)
    #17 std::__invoke_result<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t>::type std::__invoke<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t>(void (eprosima::fastdds::rtps::SharedMemChannelResource::*&&)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*&&, eprosima::fastrtps::rtps::Locator_t&&) /usr/include/c++/9/bits/invoke.h:95 (static+0x3c0190)
    #18 void std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) /usr/include/c++/9/thread:244 (static+0x3c0190)
    #19 std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::operator()() /usr/include/c++/9/thread:251 (static+0x3c0190)
    #20 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> > >::_M_run() /usr/include/c++/9/thread:195 (static+0x3c0190)
    #21 <null> <null> (libstdc++.so.6+0xd6df3)

  Mutex M372526840276845008 is already destroyed.

  Mutex M361830636543018256 is already destroyed.

  Mutex M423755457836876912 is already destroyed.

  Mutex M372245365300134160 is already destroyed.

  Thread T4 (tid=511999, running) created by main thread at:
    #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:962 (libtsan.so.0+0x5ea79)
    #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xd70c9)
    #2 eprosima::fastdds::rtps::SharedMemTransport::OpenInputChannel(eprosima::fastrtps::rtps::Locator_t const&, eprosima::fastdds::rtps::TransportReceiverInterface*, unsigned int) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/transport/shared_mem/SharedMemTransport.cpp:139 (static+0x3bb305)
    #3 eprosima::fastrtps::rtps::ReceiverResource::ReceiverResource(eprosima::fastdds::rtps::TransportInterface&, eprosima::fastrtps::rtps::Locator_t const&, unsigned int) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/network/ReceiverResource.cpp:43 (static+0x5c1d06)
    #4 eprosima::fastrtps::rtps::NetworkFactory::BuildReceiverResources(eprosima::fastrtps::rtps::Locator_t&, std::vector<std::shared_ptr<eprosima::fastrtps::rtps::ReceiverResource>, std::allocator<std::shared_ptr<eprosima::fastrtps::rtps::ReceiverResource> > >&, unsigned int) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/network/NetworkFactory.cpp:74 (static+0x5c09d8)
    #5 eprosima::fastrtps::rtps::RTPSParticipantImpl::createReceiverResources(eprosima::fastdds::rtps::LocatorList&, bool, bool) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/participant/RTPSParticipantImpl.cpp:1688 (static+0x5c56e4)
    #6 eprosima::fastrtps::rtps::RTPSParticipantImpl::RTPSParticipantImpl(unsigned int, eprosima::fastrtps::rtps::RTPSParticipantAttributes const&, eprosima::fastrtps::rtps::GuidPrefix_t const&, eprosima::fastrtps::rtps::GuidPrefix_t const&, eprosima::fastrtps::rtps::RTPSParticipant*, eprosima::fastrtps::rtps::RTPSParticipantListener*) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/participant/RTPSParticipantImpl.cpp:355 (static+0x5d20cf)
    #7 eprosima::fastrtps::rtps::RTPSParticipantImpl::RTPSParticipantImpl(unsigned int, eprosima::fastrtps::rtps::RTPSParticipantAttributes const&, eprosima::fastrtps::rtps::GuidPrefix_t const&, eprosima::fastrtps::rtps::RTPSParticipant*, eprosima::fastrtps::rtps::RTPSParticipantListener*) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/participant/RTPSParticipantImpl.cpp:449 (static+0x5d36d5)
    #8 eprosima::fastrtps::rtps::RTPSDomainImpl::createParticipant(unsigned int, bool, eprosima::fastrtps::rtps::RTPSParticipantAttributes const&, eprosima::fastrtps::rtps::RTPSParticipantListener*) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/RTPSDomain.cpp:216 (static+0x5e8ebb)
    #9 eprosima::fastrtps::rtps::RTPSDomain::createParticipant(unsigned int, bool, eprosima::fastrtps::rtps::RTPSParticipantAttributes const&, eprosima::fastrtps::rtps::RTPSParticipantListener*) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/RTPSDomain.cpp:88 (static+0x5eb353)
    #10 eprosima::fastdds::dds::DomainParticipantImpl::enable() /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/fastdds/domain/DomainParticipantImpl.cpp:269 (static+0x1b59ef)
    #11 eprosima::fastdds::dds::DomainParticipant::enable() /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/fastdds/domain/DomainParticipant.cpp:110 (static+0x1d3354)
    #12 eprosima::fastdds::dds::DomainParticipantFactory::create_participant(unsigned int, eprosima::fastdds::dds::DomainParticipantQos const&, eprosima::fastdds::dds::DomainParticipantListener*, eprosima::fastdds::dds::StatusMask const&) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/fastdds/domain/DomainParticipantFactory.cpp:187 (static+0x19842e)
    #13 sub(eprosima::fastdds::dds::DomainParticipantQos const&, unsigned int, unsigned int) /home/user/fastdds_static_disco_test/static.cxx:292 (static+0x1610dc)
    #14 main /home/user/fastdds_static_disco_test/static.cxx:396 (static+0xe9f3a)

  Thread T3 (tid=511956, running) created by main thread at:
    #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:962 (libtsan.so.0+0x5ea79)
    #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xd70c9)
    #2 eprosima::fastdds::rtps::SharedMemTransport::OpenInputChannel(eprosima::fastrtps::rtps::Locator_t const&, eprosima::fastdds::rtps::TransportReceiverInterface*, unsigned int) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/transport/shared_mem/SharedMemTransport.cpp:139 (static+0x3bb305)
    #3 eprosima::fastrtps::rtps::ReceiverResource::ReceiverResource(eprosima::fastdds::rtps::TransportInterface&, eprosima::fastrtps::rtps::Locator_t const&, unsigned int) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/network/ReceiverResource.cpp:43 (static+0x5c1d06)
    #4 eprosima::fastrtps::rtps::NetworkFactory::BuildReceiverResources(eprosima::fastrtps::rtps::Locator_t&, std::vector<std::shared_ptr<eprosima::fastrtps::rtps::ReceiverResource>, std::allocator<std::shared_ptr<eprosima::fastrtps::rtps::ReceiverResource> > >&, unsigned int) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/network/NetworkFactory.cpp:74 (static+0x5c09d8)
    #5 eprosima::fastrtps::rtps::RTPSParticipantImpl::createReceiverResources(eprosima::fastdds::rtps::LocatorList&, bool, bool) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/participant/RTPSParticipantImpl.cpp:1680 (static+0x5c51e9)
    #6 eprosima::fastrtps::rtps::RTPSParticipantImpl::RTPSParticipantImpl(unsigned int, eprosima::fastrtps::rtps::RTPSParticipantAttributes const&, eprosima::fastrtps::rtps::GuidPrefix_t const&, eprosima::fastrtps::rtps::GuidPrefix_t const&, eprosima::fastrtps::rtps::RTPSParticipant*, eprosima::fastrtps::rtps::RTPSParticipantListener*) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/participant/RTPSParticipantImpl.cpp:354 (static+0x5d20b6)
    #7 eprosima::fastrtps::rtps::RTPSParticipantImpl::RTPSParticipantImpl(unsigned int, eprosima::fastrtps::rtps::RTPSParticipantAttributes const&, eprosima::fastrtps::rtps::GuidPrefix_t const&, eprosima::fastrtps::rtps::RTPSParticipant*, eprosima::fastrtps::rtps::RTPSParticipantListener*) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/participant/RTPSParticipantImpl.cpp:449 (static+0x5d36d5)
    #8 eprosima::fastrtps::rtps::RTPSDomainImpl::createParticipant(unsigned int, bool, eprosima::fastrtps::rtps::RTPSParticipantAttributes const&, eprosima::fastrtps::rtps::RTPSParticipantListener*) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/RTPSDomain.cpp:216 (static+0x5e8ebb)
    #9 eprosima::fastrtps::rtps::RTPSDomain::createParticipant(unsigned int, bool, eprosima::fastrtps::rtps::RTPSParticipantAttributes const&, eprosima::fastrtps::rtps::RTPSParticipantListener*) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/rtps/RTPSDomain.cpp:88 (static+0x5eb353)
    #10 eprosima::fastdds::dds::DomainParticipantImpl::enable() /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/fastdds/domain/DomainParticipantImpl.cpp:269 (static+0x1b59ef)
    #11 eprosima::fastdds::dds::DomainParticipant::enable() /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/fastdds/domain/DomainParticipant.cpp:110 (static+0x1d3354)
    #12 eprosima::fastdds::dds::DomainParticipantFactory::create_participant(unsigned int, eprosima::fastdds::dds::DomainParticipantQos const&, eprosima::fastdds::dds::DomainParticipantListener*, eprosima::fastdds::dds::StatusMask const&) /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/src/cpp/fastdds/domain/DomainParticipantFactory.cpp:187 (static+0x19842e)
    #13 sub(eprosima::fastdds::dds::DomainParticipantQos const&, unsigned int, unsigned int) /home/user/fastdds_static_disco_test/static.cxx:292 (static+0x1610dc)
    #14 main /home/user/fastdds_static_disco_test/static.cxx:396 (static+0xe9f3a)

SUMMARY: ThreadSanitizer: data race /home/user/fastdds_static_disco_test/build/_deps/fastrtps-src/include/fastdds/rtps/common/SerializedPayload.h:113 in eprosima::fastrtps::rtps::SerializedPayload_t::copy(eprosima::fastrtps::rtps::SerializedPayload_t const*, bool)
==================
==================

XML configuration file

No response

Relevant log output

No response

Network traffic capture

No response

EduPonz commented 1 year ago

Hi @lexamor ,

Unfortunately v2.9 is EOL since July and therefore not maintained anymore. Could you please try to reproduce this on master?

lexamor commented 1 year ago

Hi @EduPonz ,

Yes, I am able to reproduce an issue on master (as well on other versions e.g. 2.10.x )

Segfault backtrace

0x000000000022f173: eprosima::fastdds::dds::ParameterProperty_t::second[abi:cxx11]() const at ParameterTypes.hpp:982
0x000000000060e5e9: eprosima::fastdds::dds::ParameterProperty_t::pair[abi:cxx11]() const at ParameterTypes.hpp:1027
0x000000000060b02b: eprosima::fastrtps::rtps::EDPStatic::assignRemoteEndpoints(eprosima::fastrtps::rtps::ParticipantProxyData const&, bool) at EDPStatic.cpp:444 (discriminator 1)
0x00000000005e62ba: eprosima::fastrtps::rtps::PDPSimple::notifyAboveRemoteEndpoints(eprosima::fastrtps::rtps::ParticipantProxyData const&, bool) at PDPSimple.cpp:501
0x000000000036144c: eprosima::fastrtps::rtps::security::SecurityManager::discovered_participant(eprosima::fastrtps::rtps::ParticipantProxyData const&) at SecurityManager.cpp:595
0x00000000005e60c3: eprosima::fastrtps::rtps::PDPSimple::assignRemoteEndpoints(eprosima::fastrtps::rtps::ParticipantProxyData*) at PDPSimple.cpp:465
0x00000000005e992d: eprosima::fastrtps::rtps::PDPListener::onNewCacheChangeAdded(eprosima::fastrtps::rtps::RTPSReader*, eprosima::fastrtps::rtps::CacheChange_t const*) at PDPListener.cpp:184
0x00000000004ad4f6: eprosima::fastrtps::rtps::StatelessReader::change_received(eprosima::fastrtps::rtps::CacheChange_t*) at StatelessReader.cpp:333
0x00000000004ae332: eprosima::fastrtps::rtps::StatelessReader::processDataMsg(eprosima::fastrtps::rtps::CacheChange_t*) at StatelessReader.cpp:557
0x00000000007dda29: eprosima::fastrtps::rtps::MessageReceiver::process_data_message_without_security(eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&, bool)::{lambda(eprosima::fastrtps::rtps::RTPSReader*)#1}::operator()(eprosima::fastrtps::rtps::RTPSReader*) const at MessageReceiver.cpp:220
0x00000000007e2f36: void eprosima::fastrtps::rtps::MessageReceiver::findAllReaders<eprosima::fastrtps::rtps::MessageReceiver::process_data_message_without_security(eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&, bool)::{lambda(eprosima::fastrtps::rtps::RTPSReader*)#1}>(eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::MessageReceiver::process_data_message_without_security(eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&, bool)::{lambda(eprosima::fastrtps::rtps::RTPSReader*)#1} const&) const at MessageReceiver.cpp:717 (discriminator 2)
0x00000000007dda77: eprosima::fastrtps::rtps::MessageReceiver::process_data_message_without_security(eprosima::fastrtps::rtps::EntityId_t const&, eprosima::fastrtps::rtps::CacheChange_t&, bool) at MessageReceiver.cpp:223
EduPonz commented 1 year ago

Hi @lexamor,

Thanks for you effort! We'll reproduce in our end and come back to you