Creating a Node in Galactic/Rolling results in ASAN errors

tylerjw commented 2 years ago

Bug report

Required Info:

Operating System:
- Ubuntu 22.04
Installation type:
- binaries
Version or commit hash:
- Rolling and Humble
DDS implementation:
- Default
Client library (if applicable):
- rclcpp/rcutils

Steps to reproduce issue

Here is a minimal project I created with this one source file to demonstrate this problem:

#include <gtest/gtest.h>
#include <rclcpp/rclcpp.hpp>

TEST(RclcppTests, CreateNode) {
  auto const node =
        std::make_shared<rclcpp::Node>("test", rclcpp::NodeOptions{});
  EXPECT_NE(node.get(), nullptr);
}

int main(int argc, char** argv) {
  rclcpp::init(argc, argv);
  ::testing::InitGoogleTest(&argc, argv);
  return RUN_ALL_TESTS();
}

https://github.com/tylerjw/ros2_sanitize Here is a CI run with the failures: https://github.com/tylerjw/ros2_sanitize/actions/runs/2450708458

Expected behavior

No ASAN errors when creating a rclcpp::Node

Actual behavior

ASAN errors. Here is a copy of the CI output from ASAN from rolling-main:

build/ros2_sanitize/Testing/20220606-2159/Test.xml: 1 test, 0 errors, 1 failure, 0 skipped
- rclcpp_tests
  <<< failure message
    -- run_test.py: invoking following command in '/root/target_ws/build/ros2_sanitize':
     - /root/target_ws/build/ros2_sanitize/rclcpp_tests --gtest_output=xml:/root/target_ws/build/ros2_sanitize/test_results/ros2_sanitize/rclcpp_tests.gtest.xml
    [==========] Running 1 test from 1 test suite.
    [----------] Global test environment set-up.
    [----------] 1 test from RclcppTests
    [ RUN      ] RclcppTests.CreateNode
    =================================================================
    ==6721==ERROR: AddressSanitizer: new-delete-type-mismatch on 0x607000010240 in thread T0:
      object passed to delete has wrong type:
      size of the allocated type:   72 bytes;
      size of the deallocated type: 1 bytes.
        #0 0x7f002d0e122f in operator delete(void*, unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cpp:172
        #1 0x7f002cc11dae in rcutils_string_map_fini (/opt/ros/rolling/lib/librcutils.so+0x9dae)
        #2 0x7f002ce2f3a5  (/opt/ros/rolling/lib/librcl.so+0x233a5)
        #3 0x7f002ce2f6c7 in rcl_node_resolve_name (/opt/ros/rolling/lib/librcl.so+0x236c7)
        #4 0x7f002ce2f898 in rcl_publisher_init (/opt/ros/rolling/lib/librcl.so+0x23898)
        #5 0x7f002cf9332d in rclcpp::PublisherBase::PublisherBase(rclcpp::node_interfaces::NodeBaseInterface*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rosidl_message_type_support_t const&, rcl_publisher_options_s const&) (/opt/ros/rolling/lib/librclcpp.so+0x14232d)
        #6 0x7f002cf667fb  (/opt/ros/rolling/lib/librclcpp.so+0x1157fb)
        #7 0x7f002cf66f30 in std::_Function_handler<std::shared_ptr<rclcpp::PublisherBase> (rclcpp::node_interfaces::NodeBaseInterface*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rclcpp::QoS const&), rclcpp::create_publisher_factory<rcl_interfaces::msg::ParameterEvent_<std::allocator<void> >, std::allocator<void>, rclcpp::Publisher<rcl_interfaces::msg::ParameterEvent_<std::allocator<void> >, std::allocator<void> > >(rclcpp::PublisherOptionsWithAllocator<std::allocator<void> > const&)::{lambda(rclcpp::node_interfaces::NodeBaseInterface*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rclcpp::QoS const&)#1}>::_M_invoke(std::_Any_data const&, rclcpp::node_interfaces::NodeBaseInterface*&&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rclcpp::QoS const&) (/opt/ros/rolling/lib/librclcpp.so+0x115f30)
        #8 0x7f002cf542ce in rclcpp::node_interfaces::NodeParameters::NodeParameters(std::shared_ptr<rclcpp::node_interfaces::NodeBaseInterface>, std::shared_ptr<rclcpp::node_interfaces::NodeLoggingInterface>, std::shared_ptr<rclcpp::node_interfaces::NodeTopicsInterface>, std::shared_ptr<rclcpp::node_interfaces::NodeServicesInterface>, std::shared_ptr<rclcpp::node_interfaces::NodeClockInterface>, std::vector<rclcpp::Parameter, std::allocator<rclcpp::Parameter> > const&, bool, bool, rclcpp::QoS const&, rclcpp::PublisherOptionsBase const&, bool, bool) (/opt/ros/rolling/lib/librclcpp.so+0x1032ce)
        #9 0x7f002cf4cc95 in rclcpp::Node::Node(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rclcpp::NodeOptions const&) (/opt/ros/rolling/lib/librclcpp.so+0xfbc95)
        #10 0x7f002cf4d4c7 in rclcpp::Node::Node(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rclcpp::NodeOptions const&) (/opt/ros/rolling/lib/librclcpp.so+0xfc4c7)
        #11 0x561dddf802ab in void __gnu_cxx::new_allocator<rclcpp::Node>::construct<rclcpp::Node, char const (&) [5], rclcpp::NodeOptions>(rclcpp::Node*, char const (&) [5], rclcpp::NodeOptions&&) (/root/target_ws/build/ros2_sanitize/rclcpp_tests+0x142ab)
        #12 0x561dddf7ff85 in void std::allocator_traits<std::allocator<rclcpp::Node> >::construct<rclcpp::Node, char const (&) [5], rclcpp::NodeOptions>(std::allocator<rclcpp::Node>&, rclcpp::Node*, char const (&) [5], rclcpp::NodeOptions&&) (/root/target_ws/build/ros2_sanitize/rclcpp_tests+0x13f85)
        #13 0x561dddf7fc16 in std::_Sp_counted_ptr_inplace<rclcpp::Node, std::allocator<rclcpp::Node>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<char const (&) [5], rclcpp::NodeOptions>(std::allocator<rclcpp::Node>, char const (&) [5], rclcpp::NodeOptions&&) (/root/target_ws/build/ros2_sanitize/rclcpp_tests+0x13c16)
        #14 0x561dddf7f0c8 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<rclcpp::Node, std::allocator<rclcpp::Node>, char const (&) [5], rclcpp::NodeOptions>(rclcpp::Node*&, std::_Sp_alloc_shared_tag<std::allocator<rclcpp::Node> >, char const (&) [5], rclcpp::NodeOptions&&) (/root/target_ws/build/ros2_sanitize/rclcpp_tests+0x130c8)
        #15 0x561dddf7eb11 in std::__shared_ptr<rclcpp::Node, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<rclcpp::Node>, char const (&) [5], rclcpp::NodeOptions>(std::_Sp_alloc_shared_tag<std::allocator<rclcpp::Node> >, char const (&) [5], rclcpp::NodeOptions&&) (/root/target_ws/build/ros2_sanitize/rclcpp_tests+0x12b11)
        #16 0x561dddf7dd04 in std::shared_ptr<rclcpp::Node>::shared_ptr<std::allocator<rclcpp::Node>, char const (&) [5], rclcpp::NodeOptions>(std::_Sp_alloc_shared_tag<std::allocator<rclcpp::Node> >, char const (&) [5], rclcpp::NodeOptions&&) (/root/target_ws/build/ros2_sanitize/rclcpp_tests+0x11d04)
        #17 0x561dddf7d038 in std::shared_ptr<rclcpp::Node> std::allocate_shared<rclcpp::Node, std::allocator<rclcpp::Node>, char const (&) [5], rclcpp::NodeOptions>(std::allocator<rclcpp::Node> const&, char const (&) [5], rclcpp::NodeOptions&&) (/root/target_ws/build/ros2_sanitize/rclcpp_tests+0x11038)
        #18 0x561dddf7befc in std::shared_ptr<rclcpp::Node> std::make_shared<rclcpp::Node, char const (&) [5], rclcpp::NodeOptions>(char const (&) [5], rclcpp::NodeOptions&&) (/root/target_ws/build/ros2_sanitize/rclcpp_tests+0xfefc)
        #19 0x561dddf791f1 in RclcppTests_CreateNode_Test::TestBody() /root/target_ws/src/ros2_sanitize/test/rclcpp_tests.cpp:6
        #20 0x561dddfb6aaa in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /opt/ros/rolling/src/gtest_vendor/./src/gtest.cc:2433
        #21 0x561dddfaf5c4 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /opt/ros/rolling/src/gtest_vendor/./src/gtest.cc:2[469](https://github.com/tylerjw/ros2_sanitize/runs/6763851100?check_suite_focus=true#step:3:478)
        #22 0x561dddf89759 in testing::Test::Run() /opt/ros/rolling/src/gtest_vendor/./src/gtest.cc:2508
        #23 0x561dddf8a19e in testing::TestInfo::Run() /opt/ros/rolling/src/gtest_vendor/./src/gtest.cc:2684
        #24 0x561dddf8a8ff in testing::TestSuite::Run() /opt/ros/rolling/src/gtest_vendor/./src/gtest.cc:2816
        #25 0x561dddf96f1e in testing::internal::UnitTestImpl::RunAllTests() /opt/ros/rolling/src/gtest_vendor/./src/gtest.cc:5338
        #26 0x561dddfb7b31 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /opt/ros/rolling/src/gtest_vendor/./src/gtest.cc:2433
        #27 0x561dddfb0816 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /opt/ros/rolling/src/gtest_vendor/./src/gtest.cc:2469
        #28 0x561dddf95700 in testing::UnitTest::Run() /opt/ros/rolling/src/gtest_vendor/./src/gtest.cc:[492](https://github.com/tylerjw/ros2_sanitize/runs/6763851100?check_suite_focus=true#step:3:501)5
        #29 0x561dddf7a4a5 in RUN_ALL_TESTS() (/root/target_ws/build/ros2_sanitize/rclcpp_tests+0xe4a5)
        #30 0x561dddf79817 in main /root/target_ws/src/ros2_sanitize/test/rclcpp_tests.cpp:13
        #31 0x7f002c0f7d8f  (/lib/x86_64-linux-gnu/libc.so.6+0x29d8f)
        #32 0x7f002c0f7e3f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x29e3f)
        #33 0x561dddf78f14 in _start (/root/target_ws/build/ros2_sanitize/rclcpp_tests+0xcf14)
    0x607000010240 is located 0 bytes inside of 72-byte region [0x607000010240,0x607000010288)
    allocated by thread T0 here:
        #0 0x7f002d0e01c7 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cpp:99
        #1 0x7f002cc11bb2 in rcutils_string_map_init (/opt/ros/rolling/lib/librcutils.so+0x9bb2)
    SUMMARY: AddressSanitizer: new-delete-type-mismatch ../../../../src/libsanitizer/asan/asan_new_delete.cpp:172 in operator delete(void*, unsigned long)
    ==6721==HINT: if you don't care about these errors you may set ASAN_OPTIONS=new_delete_type_mismatch=0
    ==6721==ABORTING
    -- run_test.py: return code 1
    -- run_test.py: generate result file '/root/target_ws/build/ros2_sanitize/test_results/ros2_sanitize/rclcpp_tests.gtest.xml' with failed test
    -- run_test.py: verify result file '/root/target_ws/build/ros2_sanitize/test_results/ros2_sanitize/rclcpp_tests.gtest.xml'
  >>>
build/ros2_sanitize/test_results/ros2_sanitize/rclcpp_tests.gtest.xml: 1 test, 1 error, 0 failures, 0 skipped
- ros2_sanitize rclcpp_tests.gtest.missing_result
  <<< error message
    The test did not generate a result file:

Additional Information

I originally reported this issue here: https://github.com/ros2/rcutils/issues/365 And I was told it is possibly related to this issue: https://github.com/ros2/rclcpp/pull/1324

Additional Requests

How difficult would it be to add a CI job to rclcpp that runs with ASAN? It would be really nice if users of ROS could use ASAN in their projects.

clalancette commented 2 years ago

How difficult would it be to add a CI job to rclcpp that runs with ASAN? It would be really nice if users of ROS could use ASAN in their projects.

I would love to add an ASAN job. It wouldn't be too hard to do; it basically requires some changes to the scripts on https://github.com/ros2/ci .

However, we would also need to have someone commit to the initial work of fixing the problems, and then maintaining that job going forward (at least for some period of time). If it is continually failing, it won't provide any value and will just impose costs on the rest of the infrastructure.

tylerjw commented 2 years ago

we would also need to have someone commit to the initial work of fixing the problems

I don't think I realistically have the time to fix all the ASAN failures in ROS (even though I would love that task), but maybe slowly I can help. I submitted this PR to class_loader to fix the memory lifetime bugs in it and add a GitHub action CI job that tests with ASAN: https://github.com/ros/class_loader/pull/199

It seems that there is already a PR directed towards this bug (or something related). I would appreciate reviews of the class_loader PR by anyone who also wants to see ASAN in the tests and by anyone who has the historical context of why certain decisions were made in class_laoder.

ros2 / rclcpp