search

orbbec / OrbbecSDK_ROS2

OrbbecSDK ROS2 wrapper
Apache License 2.0
83 stars 33 forks source link

orbbec SDK ROS2 crash within 1 minute with Signal 11 error by subscribing /depth/registred_point and /depth/color_point together #60

Closed tarikbeijing closed 1 month ago

tarikbeijing commented 1 month ago

by subscribing /depth/registred_point and /depth/color_point the orbbec SDK ROS2 crash always within 1 minute with same error Signal 11 .I have this problem on normal pc and on my jetson orin nx, with different gemini2 devices. Can somebody help?

ON Jetson orin nx : [component_container-2] Received signal: 11 [component_container-2] Log crash stack trace to Log/tata_nx1_AY3Z331009L_crash_stack_trace_2024_09_18_09_34_13.log [component_container-2] Received signal: 11

Received signal: 11 Stack trace (most recent call last) in thread 3626:

10 Object "[0xffffffffffffffff]", at 0xffffffffffffffff, in

9 Object "/usr/lib/aarch64-linux-gnu/libc.so.6", at 0xffffa35c5edb, in

8 Object "/usr/lib/aarch64-linux-gnu/libc.so.6", at 0xffffa355d5c7, in

7 Object "/usr/lib/aarch64-linux-gnu/libstdc++.so.6", at 0xffffa37931fb, in

6 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0xffff91b6703f, in orbbec_camera::OBCameraNode::onNewColorFrameCallback()

5 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0xffff91b5d353, in orbbec_camera::OBCameraNode::publishPointCloud(std::shared_ptr const&)

4 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0xffff91b5a3fb, in orbbec_camera::OBCameraNode::publishDepthPointCloud(std::shared_ptr const&)

3 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0xffff91b7b06f, in std::shared_ptr ob::Frame::as()

2 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0xffff91b7aec8, in bool ob::Frame::is()

1 Object "linux-vdso.so.1", at 0xffffa3b8b7bb, in

0 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0xffff91b069d3, in signalHandler(int)

PLATFORM INFORMATION system : Linux platform info : Linux-5.15.122-tegra-aarch64-with-glibc2.35 release : 5.15.122-tegra processor : aarch64

RMW MIDDLEWARE middleware name : rmw_cyclonedds_cpp

ROS 2 INFORMATION distribution name : humble distribution type : ros2 distribution status : active release platforms : {'rhel': ['8'], 'ubuntu': ['jammy']}

Device Orbbec Gemini 2 connected Serial number: AY3Z331009L Firmware version: 1.4.92 Hardware version: 0.1 Ros_sdk_version : 1.5.11 ob_sdk_version : 1.10.14 device unique id: 2-1-2 current node pid: 3543 usb connect type: USB3.0

On PC

Received signal: 11 Stack trace (most recent call last) in thread 33906:

10 Object "", at 0xffffffffffffffff, in

9 Source "../sysdeps/unix/sysv/linux/x86_64/clone3.S", line 81, in __clone3

8 Source "./nptl/pthread_create.c", line 442, in start_thread

7 Object "/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30", at 0x7cbac62dc252, in

6 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0x7cbabbde00ae, in orbbec_camera::OBCameraNode::onNewColorFrameCallback()

5 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0x7cbabbdd6fa3, in orbbec_camera::OBCameraNode::publishPointCloud(std::shared_ptr const&)

4 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0x7cbabbdd3ff7, in orbbec_camera::OBCameraNode::publishDepthPointCloud(std::shared_ptr const&)

3 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0x7cbabbdf3feb, in std::shared_ptr ob::Frame::as()

2 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0x7cbabbdf3e9e, in bool ob::Frame::is()

1 Object "/usr/lib/x86_64-linux-gnu/libc.so.6", at 0x7cbac5e4251f, in

0 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0x7cbabbd8bcc6, in signalHandler(int)

Received signal: 11 Stack trace (most recent call last) in thread 10018:

10 Object "", at 0xffffffffffffffff, in

9 Source "../sysdeps/unix/sysv/linux/x86_64/clone3.S", line 81, in __clone3

8 Source "/home/tarik/Downloads/glibc-2.35/nptl/pthread_create.c", line 442, in start_thread

7 Object "/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30", at 0x7ae29dedc252, in

6 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0x7ae2987df0de, in orbbec_camera::OBCameraNode::onNewColorFrameCallback()

5 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0x7ae2987d6083, in orbbec_camera::OBCameraNode::publishPointCloud(std::shared_ptr const&)

4 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0x7ae2987d3107, in orbbec_camera::OBCameraNode::publishDepthPointCloud(std::shared_ptr const&)

3 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0x7ae2987f2eeb, in std::shared_ptr ob::Frame::as()

2 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0x7ae2987f2d9e, in bool ob::Frame::is()

1 Object "/opt/glibc/lib/libc.so.6", at 0x7ae29da41e7f, in

0 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0x7ae29878ae96, in signalHandler(int)

PLATFORM INFORMATION system : Linux platform info : Linux-6.8.0-40-generic-x86_64-with-glibc2.35 release : 6.8.0-40-generic processor : x86_64

RMW MIDDLEWARE middleware name : rmw_cyclonedds_cpp

ROS 2 INFORMATION distribution name : humble distribution type : ros2 distribution status : active release platforms : {'rhel': ['8'], 'ubuntu': ['jammy']}

Device Orbbec Gemini 2 connected Serial number: AY3N331002H Firmware version: 1.4.92 Hardware version: 0.1 Ros_sdk_version : 1.5.11 ob_sdk_version : 1.10.14 device unique id: 6-1-3 Current node pid: 56771 usb connect type: USB3.0

jian-dong commented 1 month ago

@tarikbeijing Thanks for the information; I will check soon.

jian-dong commented 1 month ago

@tarikbeijing, can you paste your color and depth resolution and fps here?

tarikbeijing commented 1 month ago

sure

color

    DeclareLaunchArgument('color_width', default_value='1920'),
    DeclareLaunchArgument('color_height', default_value='1080'),
    DeclareLaunchArgument('color_fps', default_value='10'),
    DeclareLaunchArgument('color_format', default_value='RGB888'),
    DeclareLaunchArgument('enable_color', default_value='true'),
    DeclareLaunchArgument('flip_color', default_value='false'),
    DeclareLaunchArgument('color_qos', default_value='SENSOR_DATA'),
    DeclareLaunchArgument('color_camera_info_qos', default_value='default'),
    DeclareLaunchArgument('enable_color_auto_exposure', default_value='true'),
    DeclareLaunchArgument('color_exposure', default_value='-1'),
    DeclareLaunchArgument('color_gain', default_value='-1'),
    DeclareLaunchArgument('enable_color_auto_white_balance', default_value='true'),
    DeclareLaunchArgument('color_white_balance', default_value='-1'),
    DeclareLaunchArgument('color_info_url', default_value=''),
    DeclareLaunchArgument('color_delay_us', default_value='0'),
    #depth point cloud
    DeclareLaunchArgument('enable_point_cloud', default_value='true'),
    DeclareLaunchArgument('enable_colored_point_cloud', default_value='true'),
    DeclareLaunchArgument('point_cloud_qos', default_value='SENSOR_DATA'),
    DeclareLaunchArgument('depth_registration', default_value='true'),
    DeclareLaunchArgument('depth_width', default_value='1280'),
    DeclareLaunchArgument('depth_height', default_value='800'),
    DeclareLaunchArgument('depth_fps', default_value='10'),
    DeclareLaunchArgument('depth_format', default_value='Y16'),
    DeclareLaunchArgument('enable_depth', default_value='true'),
    DeclareLaunchArgument('flip_depth', default_value='false'),
    DeclareLaunchArgument('depth_qos', default_value='SENSOR_DATA'),
    DeclareLaunchArgument('depth_precision', default_value='0.8mm"'),
tarikbeijing commented 1 month ago

set align mode to ALIGN_D2C_HW_MODE enable depth scale ON Enable color stream Stream color width: 1920 height: 1080 fps: 10 format: RGB888 Enable depth stream Stream depth width: 1280 height: 800 fps: 10 format: Y16 Enable ir stream Stream ir width: 640 height: 400 fps: 15 format: Y8 Enable frame sync

jian-dong commented 1 month ago

@tarikbeijing can you try increase usb buffer by

echo 128 | sudo tee /sys/module/usbcore/parameters/usbfs_memory_mb

jjiszjj commented 1 month ago

@tarikbeijing I will use your camera configuration to verify it later.

tarikbeijing commented 1 month ago

Received signal: 11 Stack trace (most recent call last) in thread 8809:

10 Object "[0xffffffffffffffff]", at 0xffffffffffffffff, in

9 Object "/usr/lib/aarch64-linux-gnu/libc.so.6", at 0xffffaf185edb, in

8 Object "/usr/lib/aarch64-linux-gnu/libc.so.6", at 0xffffaf11d5c7, in

7 Object "/usr/lib/aarch64-linux-gnu/libstdc++.so.6", at 0xffffaf3531fb, in

6 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0xffffa56a703f, in orbbec_camera::OBCameraNode::onNewColorFrameCallback()

5 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0xffffa569d353, in orbbec_camera::OBCameraNode::publishPointCloud(std::shared_ptr const&)

4 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0xffffa569a3fb, in orbbec_camera::OBCameraNode::publishDepthPointCloud(std::shared_ptr const&)

3 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0xffffa56bb06f, in std::shared_ptr ob::Frame::as()

2 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0xffffa56baec8, in bool ob::Frame::is()

1 Object "linux-vdso.so.1", at 0xffffaf7487bb, in

0 Object "/home/tarik/ROS-WS/dev_ws/install/orbbec_camera/lib/liborbbec_camera.so", at 0xffffa56469d3, in signalHandler(int)

tarikbeijing commented 1 month ago

both system are with-glibc2.35,

tarikbeijing commented 1 month ago

i just made a test reduce the size, it work stable with this configuration set align mode to ALIGN_D2C_HW_MODE enable depth scale ON Enable color stream Stream color width: 640 height: 480 fps: 10 format: RGB888 Enable depth stream Stream depth width: 640 height: 400 fps: 10 format: Y16 Enable ir stream Stream ir width: 640 height: 400 fps: 15 format: Y8 Enable frame sync

tarikbeijing commented 1 month ago

i tried your fix, till now no crash any more, most of the null frame are from void OBCameraNode::publishDepthPointCloud(const std::shared_ptr &frame_set) and not from void OBCameraNode::publishColoredPointCloud(const std::shared_ptr &frame_set).

thank you the issue can be closed.

[component_container-2] [INFO] [1726705085.298624847] : ########################################### [component_container-2] [INFO] [1726705085.298859220] : set align mode to ALIGN_D2C_HW_MODE [component_container-2] [INFO] [1726705085.299004791] : enable depth scale ON [component_container-2] [INFO] [1726705085.299044792] : Enable color stream [component_container-2] [INFO] [1726705085.299192891] : Stream color width: 1920 height: 1080 fps: 10 format: RGB888 [component_container-2] [INFO] [1726705085.299226844] : Enable depth stream [component_container-2] [INFO] [1726705085.299306205] : Stream depth width: 1280 height: 800 fps: 10 format: Y16 [component_container-2] [INFO] [1726705085.299386399] : Enable ir stream [component_container-2] [INFO] [1726705085.299418368] : Stream ir width: 640 height: 400 fps: 15 format: Y8 [component_container-2] [INFO] [1726705085.406694220] : Enable frame sync [component_container-2] [INFO] [1726705085.413161968] : #########################################

[component_container-2] [INFO] [1726705177.891531253] : 100 frames used 0.100000 ms, max fps : 10.000000 frames/sec [componentcontainer-2] [ERROR] [1726705180.665784618] : depth PointCloud frame is null [componentcontainer-2] [ERROR] [1726705185.011854817] : depth PointCloud frame is null [componentcontainer-2] [ERROR] [1726705185.650730667] : depth PointCloud frame is null [componentcontainer-2] [ERROR] [1726705186.205303036] : depth PointCloud frame is null [componentcontainer-2] [ERROR] [1726705186.841139112] : depth PointCloud frame is null [componentcontainer-2] [ERROR] [1726705187.000144916] : depth PointCloud frame is null [component_container-2] [INFO] [1726705187.895626288] : 100 frames used 0.100000 ms, max fps : 10.000000 frames/sec [componentcontainer-2] [ERROR] [1726705189.020389586] : depth PointCloud frame is null [componentcontainer-2] [ERROR] [1726705191.257263252] : depth PointCloud frame is null [componentcontainer-2] [ERROR] [1726705195.413435914] : depth PointCloud frame is null [componentcontainer-2] [ERROR] [1726705197.030653930] : depth PointCloud frame is null [component_container-2] [INFO] [1726705197.803589219] : 100 frames used 0.100000 ms, max fps : 10.000000 frames/sec [componentcontainer-2] [ERROR] [1726705198.867962901] : depth PointCloud frame is null [componentcontainer-2] [ERROR] [1726705202.674785620] : depth PointCloud frame is null [componentcontainer-2] [ERROR] [1726705203.696862567] : depth PointCloud frame is null [componentcontainer-2] [ERROR] [1726705207.070119256] : depth PointCloud frame is null [componentcontainer-2] [ERROR] [1726705207.872199460] : depth PointCloud frame is null [component_container-2] [INFO] [1726705207.888008966] : 100 frames used 0.100000 ms, max fps : 10.000000 frames/sec [component_container-2] [INFO] [1726705217.975131041] : 100 frames used 0.100000 ms, max fps : 10.000000 frames/sec [component_container-2] [INFO] [1726705228.051148346] : 100 frames used 0.110000 ms, max fps : 9.090909 frames/sec [component_container-2] [INFO] [1726705237.971610062] : 100 frames used 0.090000 ms, max fps : 11.111111 frames/sec

tarikbeijing commented 1 month ago

i also made another test, modified the fps from 10 to 15fps for all frame type, there is also no one error message more. it look very stable [component_container-2] [INFO] [1726706222.602703362] : set align mode to ALIGN_D2C_HW_MODE [component_container-2] [INFO] [1726706222.602760419] : enable depth scale ON [component_container-2] [INFO] [1726706222.602799779] : Enable color stream [component_container-2] [INFO] [1726706222.602841092] : Stream color width: 1920 height: 1080 fps: 15 format: RGB888 [component_container-2] [INFO] [1726706222.602887653] : Enable depth stream [component_container-2] [INFO] [1726706222.602919558] : Stream depth width: 1280 height: 800 fps: 15 format: Y16 [component_container-2] [INFO] [1726706222.602947206] : Enable ir stream [component_container-2] [INFO] [1726706222.602973511] : Stream ir width: 1280 height: 800 fps: 15 format: Y8 [component_container-2] [INFO] [1726706222.697950873] : Enable frame sync

[component_container-2] [INFO] [1726706234.571784029] : 100 frames used 17267062.340000 ms, max fps : 0.000000 frames/sec [component_container-2] [INFO] [1726706244.352732395] : 100 frames used 0.100000 ms, max fps : 10.000000 frames/sec [component_container-2] [INFO] [1726706251.168242903] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec [component_container-2] [INFO] [1726706258.813267427] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec [component_container-2] [INFO] [1726706282.753039039] : 100 frames used 0.240000 ms, max fps : 4.166667 frames/sec [component_container-2] [INFO] [1726706289.499249253] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec [component_container-2] [INFO] [1726706296.325986040] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec [component_container-2] [INFO] [1726706303.106980581] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec [component_container-2] [INFO] [1726706309.775351960] : 100 frames used 0.060000 ms, max fps : 16.666667 frames/sec [component_container-2] [INFO] [1726706316.451023744] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec [component_container-2] [INFO] [1726706323.044581852] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec [component_container-2] [INFO] [1726706331.240245904] : 100 frames used 0.080000 ms, max fps : 12.500000 frames/sec [component_container-2] [INFO] [1726706339.187737915] : 100 frames used 0.080000 ms, max fps : 12.500000 frames/sec [component_container-2] [INFO] [1726706345.855211933] : 100 frames used 0.060000 ms, max fps : 16.666667 frames/sec [component_container-2] [INFO] [1726706352.528517878] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec [component_container-2] [INFO] [1726706359.271038580] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec [component_container-2] [INFO] [1726706366.008012577] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec [component_container-2] [INFO] [1726706372.742812514] : 100 frames used 0.060000 ms, max fps : 16.666667 frames/sec [component_container-2] [INFO] [1726706379.480271673] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec [component_container-2] [INFO] [1726706386.217153892] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec [component_container-2] [INFO] [1726706393.413270764] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec [component_container-2] [INFO] [1726706400.156714046] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec [component_container-2] [INFO] [1726706406.831627642] : 100 frames used 0.060000 ms, max fps : 16.666667 frames/sec [component_container-2] [INFO] [1726706414.897422582] : 100 frames used 0.080000 ms, max fps : 12.500000 frames/sec [component_container-2] [INFO] [1726706421.794757102] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec [component_container-2] [INFO] [1726706428.304094007] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec [component_container-2] [INFO] [1726706435.045254619] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec [component_container-2] [INFO] [1726706441.637414591] : 100 frames used 0.060000 ms, max fps : 16.666667 frames/sec [component_container-2] [INFO] [1726706448.310467894] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec [component_container-2] [INFO] [1726706454.976935271] : 100 frames used 0.060000 ms, max fps : 16.666667 frames/sec [component_container-2] [INFO] [1726706461.653725099] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec [component_container-2] [INFO] [1726706468.394497193] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec [component_container-2] [INFO] [1726706475.064120315] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec [component_container-2] [INFO] [1726706481.817920515] : 100 frames used 0.060000 ms, max fps : 16.666667 frames/sec [component_container-2] [INFO] [1726706488.396953116] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec [component_container-2] [INFO] [1726706495.629566320] : 100 frames used 0.070000 ms, max fps : 14.285714 frames/sec