Segfault in driver while launching controller

[schornakj]

Saw this crash in the logs for one of the Skills Center systems, apparently from when it was first launching the drivers after startup:

[launcher_node-2] [ros2_control_node-1] terminate called after throwing an instance of 'std::runtime_error'
[launcher_node-2] [ros2_control_node-1]   what():  sendto() failed with error code : 101
[launcher_node-2] [ros2_control_node-1] Stack trace (most recent call last) in thread 179:
[launcher_node-2] [ros2_control_node-1] #18   Object "", at 0xffffffffffffffff, in 
[launcher_node-2] [ros2_control_node-1] #17   Object "/usr/lib/x86_64-linux-gnu/libc.so.6", at 0x7fa6687229ff, in 
[launcher_node-2] [ros2_control_node-1] #16   Object "/usr/lib/x86_64-linux-gnu/libc.so.6", at 0x7fa668690b42, in 
[launcher_node-2] [ros2_control_node-1] #15   Object "/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30", at 0x7fa6689202b2, in 
[launcher_node-2] [ros2_control_node-1] #14   Object "/opt/ros/humble/lib/controller_manager/ros2_control_node", at 0x55a399602406, in 
[launcher_node-2] [ros2_control_node-1] #13   Object "/opt/ros/humble/lib/libhardware_interface.so", at 0x7fa668528489, in hardware_interface::ResourceManager::write(rclcpp::Time const&, rclcpp::Duration const&)
[launcher_node-2] [ros2_control_node-1] #12   Object "/opt/ros/humble/lib/libhardware_interface.so", at 0x7fa66854f854, in hardware_interface::System::write(rclcpp::Time const&, rclcpp::Duration const&)
[launcher_node-2] [ros2_control_node-1] #11   Object "/opt/underlay_ws/install/kortex2_driver/lib/libkortex2_driver.so", at 0x7fa656c6811d, in kortex2_driver::KortexMultiInterfaceHardware::write(rclcpp::Time const&, rclcpp::Duration const&)
[launcher_node-2] [ros2_control_node-1] #10   Object "/opt/underlay_ws/install/kortex2_driver/lib/libkortex2_driver.so", at 0x7fa656c69bbc, in kortex2_driver::KortexMultiInterfaceHardware::sendJointCommands() [clone .cold]
[launcher_node-2] [ros2_control_node-1] #9    Object "/opt/underlay_ws/install/kortex2_driver/lib/libkortex2_driver.so", at 0x7fa656dda71e, in Kinova::Api::BaseCyclic::BaseCyclicClient::RefreshFeedback(unsigned int, Kinova::Api::RouterClientSendOptions const&)
[launcher_node-2] [ros2_control_node-1] #8    Object "/opt/underlay_ws/install/kortex2_driver/lib/libkortex2_driver.so", at 0x7fa656d23807, in Kinova::Api::RouterClient::send(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int, unsigned int, unsigned int, Kinova::Api::RouterClientSendOptions const&)
[launcher_node-2] [ros2_control_node-1] #7    Object "/opt/underlay_ws/install/kortex2_driver/lib/libkortex2_driver.so", at 0x7fa656d2c816, in Kinova::Api::TransportClientUdp::send(char const*, unsigned int)
[launcher_node-2] [ros2_control_node-1] #6    Object "/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30", at 0x7fa6688f2517, in __cxa_throw
[launcher_node-2] [ros2_control_node-1] #5    Object "/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30", at 0x7fa6688f22b6, in std::terminate()
[launcher_node-2] [ros2_control_node-1] #4    Object "/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30", at 0x7fa6688f224b, in 
[launcher_node-2] [ros2_control_node-1] #3    Object "/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30", at 0x7fa6688e6bbd, in 
[launcher_node-2] [ros2_control_node-1] #2    Object "/usr/lib/x86_64-linux-gnu/libc.so.6", at 0x7fa6686247f2, in abort
[launcher_node-2] [ros2_control_node-1] #1    Object "/usr/lib/x86_64-linux-gnu/libc.so.6", at 0x7fa66863e475, in raise
[launcher_node-2] [ros2_control_node-1] #0    Object "/usr/lib/x86_64-linux-gnu/libc.so.6", at 0x7fa668692a7c, in pthread_kill
[launcher_node-2] [ros2_control_node-1] Aborted (Signal sent by tkill() 105 0)
[launcher_node-2] [ERROR] [ros2_control_node-1]: process has died [pid 105, exit code -6, cmd '/opt/ros/humble/lib/controller_manager/ros2_control_node --ros-args --params-file /tmp/launch_params_1yeiu6dv --params-file /tmp/launch_params_4x0vz0dj --params-file /opt/moveit_studio/user_ws/install/kinova_gen3_lite_base_config/share/kinova_gen3_lite_base_config/config/control/kinova_gen3_lite.ros2_control.yaml --params-file /tmp/launch_params_4k_ixde9 --params-file /tmp/launch_params_nkds895e'].

[moriarty]

@schornakj do you have any version information?

[schornakj]

@moriarty this happened with the version of the package that's shipped with MoveIt Studio 1.11.4.

[fzoric8]

Hi @schornakj dd you manage to solve this? :)

[aalmrad]

Hello, Apologies for the late reply. We are trying to tackle the github issues on our repositories more actively. The error you are getting suggests a network communication failure between the ROS 2 control node and the Kinova arm. The error was replicated on my end and i will be working towards a solution.

Best, Abed