search

ros2 / rclcpp

rclcpp (ROS Client Library for C++)
Apache License 2.0
562 stars 425 forks source link

AddressSanitizer: new-delete-type-mismatch #2220

Open fujitatomoya opened 1 year ago

fujitatomoya commented 1 year ago

Bug report

Required Info:

Steps to reproduce issue

colcon build --symlink-install --cmake-args -DCMAKE_CXX_FLAGS="-g  -fsanitize=address" -DCMAKE_C_FLAGS="-g -fsanitize=address"
source install/local_setup.bash
ros2 run demo_nodes_cpp talker
...

Expected behavior

No ASAN report generated.

Actual behavior

The following ASAN report generated.

AddressSanitizer: new-delete-type-mismatch ```bash root@tomoyafujita:~/ros2_ws/colcon_ws# ros2 run demo_nodes_cpp talker ================================================================= ==726931==ERROR: AddressSanitizer: new-delete-type-mismatch on 0x60600006c0e0 in thread T0: object passed to delete has wrong type: size of the allocated type: 64 bytes; size of the deallocated type: 1 bytes. #0 0x7f7b840bd22f in operator delete(void*, unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cpp:172 #1 0x7f7b82772587 in rcutils_string_map_fini /root/ros2_ws/colcon_ws/src/ros2/rcutils/src/string_map.c:105 #2 0x7f7b8354dfb0 in rcl_resolve_name /root/ros2_ws/colcon_ws/src/ros2/rcl/rcl/src/rcl/node_resolve_name.c:110 #3 0x7f7b8354e751 in rcl_node_resolve_name /root/ros2_ws/colcon_ws/src/ros2/rcl/rcl/src/rcl/node_resolve_name.c:152 #4 0x7f7b835483e5 in rcl_publisher_init /root/ros2_ws/colcon_ws/src/ros2/rcl/rcl/src/rcl/publisher.c:83 #5 0x7f7b83c0bc16 in rclcpp::PublisherBase::PublisherBase(rclcpp::node_interfaces::NodeBaseInterface*, std::__cxx11::basic_string, std::allocator > const&, rosidl_message_type_support_t const&, rcl_publisher_options_s const&, rclcpp::PublisherEventCallbacks const&, bool) /root/ros2_ws/colcon_ws/src/ros2/rclcpp/rclcpp/src/rclcpp/publisher_base.cpp:73 #6 0x7f7b83a55937 in rclcpp::Publisher >, std::allocator >::Publisher(rclcpp::node_interfaces::NodeBaseInterface*, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&, rclcpp::PublisherOptionsWithAllocator > const&) /root/ros2_ws/colcon_ws/src/ros2/rclcpp/rclcpp/include/rclcpp/publisher.hpp:140 #7 0x7f7b83a56e01 in void __gnu_cxx::new_allocator >, std::allocator > >::construct >, std::allocator >, rclcpp::node_interfaces::NodeBaseInterface*&, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&, rclcpp::PublisherOptionsWithAllocator > const&>(rclcpp::Publisher >, std::allocator >*, rclcpp::node_interfaces::NodeBaseInterface*&, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&, rclcpp::PublisherOptionsWithAllocator > const&) /usr/include/c++/11/ext/new_allocator.h:162 #8 0x7f7b83a56e01 in void std::allocator_traits >, std::allocator > > >::construct >, std::allocator >, rclcpp::node_interfaces::NodeBaseInterface*&, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&, rclcpp::PublisherOptionsWithAllocator > const&>(std::allocator >, std::allocator > >&, rclcpp::Publisher >, std::allocator >*, rclcpp::node_interfaces::NodeBaseInterface*&, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&, rclcpp::PublisherOptionsWithAllocator > const&) /usr/include/c++/11/bits/alloc_traits.h:516 #9 0x7f7b83a56e01 in std::_Sp_counted_ptr_inplace >, std::allocator >, std::allocator >, std::allocator > >, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace, std::allocator > const&, rclcpp::QoS const&, rclcpp::PublisherOptionsWithAllocator > const&>(std::allocator >, std::allocator > >, rclcpp::node_interfaces::NodeBaseInterface*&, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&, rclcpp::PublisherOptionsWithAllocator > const&) /usr/include/c++/11/bits/shared_ptr_base.h:519 #10 0x7f7b83a56e01 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count >, std::allocator >, std::allocator >, std::allocator > >, rclcpp::node_interfaces::NodeBaseInterface*&, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&, rclcpp::PublisherOptionsWithAllocator > const&>(rclcpp::Publisher >, std::allocator >*&, std::_Sp_alloc_shared_tag >, std::allocator > > >, rclcpp::node_interfaces::NodeBaseInterface*&, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&, rclcpp::PublisherOptionsWithAllocator > const&) /usr/include/c++/11/bits/shared_ptr_base.h:650 #11 0x7f7b83a56e01 in std::__shared_ptr >, std::allocator >, (__gnu_cxx::_Lock_policy)2>::__shared_ptr >, std::allocator > >, rclcpp::node_interfaces::NodeBaseInterface*&, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&, rclcpp::PublisherOptionsWithAllocator > const&>(std::_Sp_alloc_shared_tag >, std::allocator > > >, rclcpp::node_interfaces::NodeBaseInterface*&, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&, rclcpp::PublisherOptionsWithAllocator > const&) /usr/include/c++/11/bits/shared_ptr_base.h:1342 #12 0x7f7b83a56e01 in std::shared_ptr >, std::allocator > >::shared_ptr >, std::allocator > >, rclcpp::node_interfaces::NodeBaseInterface*&, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&, rclcpp::PublisherOptionsWithAllocator > const&>(std::_Sp_alloc_shared_tag >, std::allocator > > >, rclcpp::node_interfaces::NodeBaseInterface*&, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&, rclcpp::PublisherOptionsWithAllocator > const&) /usr/include/c++/11/bits/shared_ptr.h:409 #13 0x7f7b83a56e01 in std::shared_ptr >, std::allocator > > std::allocate_shared >, std::allocator >, std::allocator >, std::allocator > >, rclcpp::node_interfaces::NodeBaseInterface*&, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&, rclcpp::PublisherOptionsWithAllocator > const&>(std::allocator >, std::allocator > > const&, rclcpp::node_interfaces::NodeBaseInterface*&, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&, rclcpp::PublisherOptionsWithAllocator > const&) /usr/include/c++/11/bits/shared_ptr.h:863 #14 0x7f7b83a56e01 in std::shared_ptr >, std::allocator > > std::make_shared >, std::allocator >, rclcpp::node_interfaces::NodeBaseInterface*&, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&, rclcpp::PublisherOptionsWithAllocator > const&>(rclcpp::node_interfaces::NodeBaseInterface*&, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&, rclcpp::PublisherOptionsWithAllocator > const&) /usr/include/c++/11/bits/shared_ptr.h:879 #15 0x7f7b83a56e01 in rclcpp::create_publisher_factory >, std::allocator, rclcpp::Publisher >, std::allocator > >(rclcpp::PublisherOptionsWithAllocator > const&)::{lambda(rclcpp::node_interfaces::NodeBaseInterface*, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&)#1}::operator()(rclcpp::node_interfaces::NodeBaseInterface*, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&) const /root/ros2_ws/colcon_ws/src/ros2/rclcpp/rclcpp/include/rclcpp/publisher_factory.hpp:76 #16 0x7f7b83a56e01 in std::shared_ptr >, std::allocator > > std::__invoke_impl >, std::allocator > >, rclcpp::create_publisher_factory >, std::allocator, rclcpp::Publisher >, std::allocator > >(rclcpp::PublisherOptionsWithAllocator > const&)::{lambda(rclcpp::node_interfaces::NodeBaseInterface*, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&)#1}&, rclcpp::node_interfaces::NodeBaseInterface*, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&>(std::__invoke_other, rclcpp::create_publisher_factory >, std::allocator, rclcpp::Publisher >, std::allocator > >(rclcpp::PublisherOptionsWithAllocator > const&)::{lambda(rclcpp::node_interfaces::NodeBaseInterface*, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&)#1}&, rclcpp::node_interfaces::NodeBaseInterface*&&, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&) /usr/include/c++/11/bits/invoke.h:61 #17 0x7f7b83a56e01 in std::enable_if, rclcpp::create_publisher_factory >, std::allocator, rclcpp::Publisher >, std::allocator > >(rclcpp::PublisherOptionsWithAllocator > const&)::{lambda(rclcpp::node_interfaces::NodeBaseInterface*, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&)#1}&, rclcpp::node_interfaces::NodeBaseInterface*, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&>, std::shared_ptr >::type std::__invoke_r, rclcpp::create_publisher_factory >, std::allocator, rclcpp::Publisher >, std::allocator > >(rclcpp::PublisherOptionsWithAllocator > const&)::{lambda(rclcpp::node_interfaces::NodeBaseInterface*, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&)#1}&, rclcpp::node_interfaces::NodeBaseInterface*, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&>(rclcpp::create_publisher_factory >, std::allocator, rclcpp::Publisher >, std::allocator > >(rclcpp::PublisherOptionsWithAllocator > const&)::{lambda(rclcpp::node_interfaces::NodeBaseInterface*, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&)#1}&, rclcpp::node_interfaces::NodeBaseInterface*&&, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&) /usr/include/c++/11/bits/invoke.h:116 #18 0x7f7b83a56e01 in std::_Function_handler (rclcpp::node_interfaces::NodeBaseInterface*, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&), rclcpp::create_publisher_factory >, std::allocator, rclcpp::Publisher >, std::allocator > >(rclcpp::PublisherOptionsWithAllocator > const&)::{lambda(rclcpp::node_interfaces::NodeBaseInterface*, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&)#1}>::_M_invoke(std::_Any_data const&, rclcpp::node_interfaces::NodeBaseInterface*&&, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&) /usr/include/c++/11/bits/std_function.h:291 #19 0x7f7b83a5f128 in std::function (rclcpp::node_interfaces::NodeBaseInterface*, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&)>::operator()(rclcpp::node_interfaces::NodeBaseInterface*, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&) const /usr/include/c++/11/bits/std_function.h:590 #20 0x7f7b83a5f128 in rclcpp::node_interfaces::NodeTopics::create_publisher(std::__cxx11::basic_string, std::allocator > const&, rclcpp::PublisherFactory const&, rclcpp::QoS const&) /root/ros2_ws/colcon_ws/src/ros2/rclcpp/rclcpp/src/rclcpp/node_interfaces/node_topics.cpp:50 #21 0x7f7b83a143a8 in std::shared_ptr >, std::allocator > > rclcpp::detail::create_publisher >, std::allocator, rclcpp::Publisher >, std::allocator >, std::shared_ptr, std::shared_ptr >(std::shared_ptr&, std::shared_ptr&, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&, rclcpp::PublisherOptionsWithAllocator > const&) /root/ros2_ws/colcon_ws/src/ros2/rclcpp/rclcpp/include/rclcpp/create_publisher.hpp:65 #22 0x7f7b83a143a8 in std::shared_ptr >, std::allocator > > rclcpp::create_publisher >, std::allocator, rclcpp::Publisher >, std::allocator >, std::shared_ptr&>(std::shared_ptr&, std::__cxx11::basic_string, std::allocator > const&, rclcpp::QoS const&, rclcpp::PublisherOptionsWithAllocator > const&) /root/ros2_ws/colcon_ws/src/ros2/rclcpp/rclcpp/include/rclcpp/create_publisher.hpp:104 #23 0x7f7b83a143a8 in rclcpp::node_interfaces::NodeParameters::NodeParameters(std::shared_ptr, std::shared_ptr, std::shared_ptr, std::shared_ptr, std::shared_ptr, std::vector > const&, bool, bool, rclcpp::QoS const&, rclcpp::PublisherOptionsBase const&, bool, bool) /root/ros2_ws/colcon_ws/src/ros2/rclcpp/rclcpp/src/rclcpp/node_interfaces/node_parameters.cpp:99 #24 0x7f7b83983162 in rclcpp::Node::Node(std::__cxx11::basic_string, std::allocator > const&, std::__cxx11::basic_string, std::allocator > const&, rclcpp::NodeOptions const&) /root/ros2_ws/colcon_ws/src/ros2/rclcpp/rclcpp/src/rclcpp/node.cpp:197 #25 0x7f7b83988223 in rclcpp::Node::Node(std::__cxx11::basic_string, std::allocator > const&, rclcpp::NodeOptions const&) /root/ros2_ws/colcon_ws/src/ros2/rclcpp/rclcpp/src/rclcpp/node.cpp:115 #26 0x7f7b7e576f64 in demo_nodes_cpp::Talker::Talker(rclcpp::NodeOptions const&) /root/ros2_ws/colcon_ws/src/ros2/demos/demo_nodes_cpp/src/topics/talker.cpp:38 #27 0x7f7b7e5794d3 in void __gnu_cxx::new_allocator::construct(demo_nodes_cpp::Talker*, rclcpp::NodeOptions const&) /usr/include/c++/11/ext/new_allocator.h:162 #28 0x7f7b7e5794d3 in void std::allocator_traits >::construct(std::allocator&, demo_nodes_cpp::Talker*, rclcpp::NodeOptions const&) /usr/include/c++/11/bits/alloc_traits.h:516 #29 0x7f7b7e5794d3 in std::_Sp_counted_ptr_inplace, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace(std::allocator, rclcpp::NodeOptions const&) /usr/include/c++/11/bits/shared_ptr_base.h:519 #30 0x7f7b7e5794d3 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count, rclcpp::NodeOptions const&>(demo_nodes_cpp::Talker*&, std::_Sp_alloc_shared_tag >, rclcpp::NodeOptions const&) /usr/include/c++/11/bits/shared_ptr_base.h:650 #31 0x7f7b7e5794d3 in std::__shared_ptr::__shared_ptr, rclcpp::NodeOptions const&>(std::_Sp_alloc_shared_tag >, rclcpp::NodeOptions const&) /usr/include/c++/11/bits/shared_ptr_base.h:1342 #32 0x7f7b7e5794d3 in std::shared_ptr::shared_ptr, rclcpp::NodeOptions const&>(std::_Sp_alloc_shared_tag >, rclcpp::NodeOptions const&) /usr/include/c++/11/bits/shared_ptr.h:409 #33 0x7f7b7e5794d3 in std::shared_ptr std::allocate_shared, rclcpp::NodeOptions const&>(std::allocator const&, rclcpp::NodeOptions const&) /usr/include/c++/11/bits/shared_ptr.h:863 #34 0x7f7b7e5794d3 in std::shared_ptr std::make_shared(rclcpp::NodeOptions const&) /usr/include/c++/11/bits/shared_ptr.h:879 #35 0x7f7b7e5794d3 in rclcpp_components::NodeFactoryTemplate::create_node_instance(rclcpp::NodeOptions const&) /root/ros2_ws/colcon_ws/src/ros2/rclcpp/rclcpp_components/include/rclcpp_components/node_factory_template.hpp:45 #36 0x55b82f7d87f7 in main /root/ros2_ws/colcon_ws/build/demo_nodes_cpp/rclcpp_components/node_main_talker.cpp:55 #37 0x7f7b81c3dd8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 #38 0x7f7b81c3de3f in __libc_start_main_impl ../csu/libc-start.c:392 #39 0x55b82f7db5c4 in _start (/root/ros2_ws/colcon_ws/build/demo_nodes_cpp/talker+0xd5c4) 0x60600006c0e0 is located 0 bytes inside of 64-byte region [0x60600006c0e0,0x60600006c120) allocated by thread T0 here: #0 0x7f7b840bc1c7 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cpp:99 #1 0x7f7b82772023 in rcutils_string_map_init /root/ros2_ws/colcon_ws/src/ros2/rcutils/src/string_map.c:67 #2 0x7f7b8354dc37 in rcl_resolve_name /root/ros2_ws/colcon_ws/src/ros2/rcl/rcl/src/rcl/node_resolve_name.c:47 #3 0x610000000a3f () SUMMARY: AddressSanitizer: new-delete-type-mismatch ../../../../src/libsanitizer/asan/asan_new_delete.cpp:172 in operator delete(void*, unsigned long) ==726931==HINT: if you don't care about these errors you may set ASAN_OPTIONS=new_delete_type_mismatch=0 ==726931==ABORTING [ros2run]: Process exited with failure 1 ```
fujitatomoya commented 1 year ago

this can also happen with humble branch as well.

fujitatomoya commented 1 year ago

CC: @iuhilnehc-ynos @Barry-Xu-2018

iuhilnehc-ynos commented 1 year ago

Actually, I noticed this issue a long time ago while adding the zero_allocate.

The deallocate just deallocate buffer size with 1. I don't know how to fix this issue elegantly. Any suggestion?

iuhilnehc-ynos commented 1 year ago

How about incorporating an additional size field at the beginning of the buffer to store the size value during buffer allocation?

(NOTE: It results in a significant waste of memory when there are numerous small buffer allocations.)

CursedRock17 commented 1 year ago

According to cppreference the ending parameter of both allocate and deallocate is a size_type n where n is:

the number of objects to allocate storage for

So is there anyway to extract the size of the typename T given in retyped_deallocate like 

std::allocator_traits<Alloc>::deallocate(*typed_allocator, typed_ptr, sizeof(T));

or a some way to find the type of the object that was allocated, or is that beyond the reach of the function.

iuhilnehc-ynos commented 1 year ago

What if the type is char and then rcl uses the allocator to allocate a buffer with count? Even if we can add an explicit specialization template get_rcl_allocator for char to use rcl_get_default_allocator(), which seems like a workaround, what if using get_rcl_allocator<rcl_node_t> in the future, and then use the allocator to allocate a buffer with count(2)?

It seems that using sizeof(T) can't fix this issue.

kfabian commented 11 months ago

I stumbled on this issue, because I encountered problems when I wanted to use ROS2 >= humble with jemalloc. (Apparently I am not the first one: ROS 2 migration stories: The struggles of moving to Humble)

I think std::allocator_traits<Alloc>::deallocate(*typed_allocator, typed_ptr, sizeof(T)); won't work because n is the number of objects of type T to deallocate and not the number of bytes.

I also think that https://github.com/ros2/rclcpp/blob/978439191fdb3924af900a506dd35b68f8da725c/rclcpp/include/rclcpp/allocator/allocator_common.hpp#L67 only works because the default implementation of operator delete(void* ptr, std::size_t size) just calls operator delete(void* ptr) and ignores the size argument. Jemalloc instead explicitly implements a sized delete, which leads to memory errors when used together with rclcpp, because the size argument does not match the previously allocated size. So I wonder if there is a safe way to create an rcl_allocator_t struct from an STL allocator, because the STL allocator requires a size argument for deallocate.

iluetkeb commented 11 months ago

I'm assuming that the C++ standard writers had a good reason to make the number of elements a parameter. Therefore, I would suggest to address this by adding a corresponding method which takes this as an argument. Further, we may want to consider deprecating the existing version of the function that does not have this argument.

ralwing commented 9 months ago

There are many errors related to this issue:

volleypaul commented 1 month ago

Is this still an issue in Jazzy? Thanks.

fujitatomoya commented 1 month ago

yes, this issue is still open.

russkel commented 1 week ago

So what is required for this to be fixed? I am running into some issues and would like to be able to run our stack with these sanitisers.

skndi commented 1 day ago

Does anybody have a workaround for this problem? We're having some memory issues in one of our executables and I have a strong suspicion that the cause might be this bug. We're not using any custom allocators in our project so I would be okay even with replacing the body of the retyped_* functions with malloc/free/realloc.

kfabian commented 1 day ago

We work around it by changing https://github.com/ros2/rclcpp/blob/64dd644469d7fb6f2ed64e609e5bd48b6b23f4ce/rclcpp/include/rclcpp/allocator/allocator_common.hpp#L91 to 

#if 0

Which should have the exact same effect as changing the implementation of the retyped_* functions