dual zedx mini setup works only sporadically on zed box orin

JHeuverRiwo commented 12 months ago

Preliminary Checks

[X] This issue is not a duplicate. Before opening a new issue, please search existing issues.
[X] This issue is not a question, feature request, or anything other than a bug report directly related to this project.

Description

I am trying to launch two cameras at once. Both get different camera names. one publishes TFs and the other doesn't. The main problem is that often either one or both cameras do not open properly.

[zed_wrapper-4] [ZED] [ERROR] [ZED] sl::Camera::Open has not been called, no Camera instance running. [zed_wrapper-4] [WARN] [cam1.node]: Error opening camera: CAMERA NOT DETECTED [zed_wrapper-4] [WARN] [cam1.node]: Please verify the camera connection.

Similarly when I for instance choose to set the depth to NONE in the configs, the color cams may actually both open, although it complains about the depth cam settings. After a while the whole zed box may actually crash and restart. This is obviously not the intended behavior.

Right now when I set the SDK grab rate to 60, the video quality to SVGA or HD1200, and the depth to PERFORMANCE it seems to start up the nodes properly, but when checking the topics in rviz2 I am getting the message that the message filter drops the message due to EmptyFrameID.

I tried asking in the examples repo, but I am getting no answer there. The launch file I am using right now is a bit simpler than the tutorials zed_multi_camera launch file, in that it simply tries to launch two cameras, so I think it is also appropriate for this repo.

Checking a single camera I can run them without errors.

Steps to Reproduce

Launch two zed nodes i.e.:

# Camera model (force value)
camera_model = 'zedxm'  

# ZED Wrapper node
zed_wrapper_launch = IncludeLaunchDescription(
    launch_description_source=PythonLaunchDescriptionSource([
        get_package_share_directory('arha_orchard_robot_demo'),
        '/launch/zed_camera.launch.py'
    ]),
    launch_arguments={
        'camera_name': 'cam0',
        'camera_model': camera_model,
        # 'zed_id': '1', #should be 'int'
        'serial_number': '54806473', 
        'node_name': 'node', #crashes on empty name.
        'publish_tf': 'True',
        'publish_map_tf': 'True',
        'cam_pose': '[0,0,0,0,0,0]',
    }.items()
)

zed_wrapper_launch2 = IncludeLaunchDescription(
    launch_description_source=PythonLaunchDescriptionSource([
        get_package_share_directory('arha_orchard_robot_demo'),
        '/launch/zed_camera.launch.py'
    ]),
    launch_arguments={
        'camera_name': 'cam1',
        'camera_model': camera_model,
        # 'zed_id': '1', #should be 'int'
        'serial_number': '56272705', # on stack
        'node_name': 'node',
        'publish_tf': 'False',
        'publish_map_tf': 'False',
        'cam_pose': '[0,0.1,0,0,0,0]',
    }.items()
)

note: changed the location of the zed_camera.launch.py, file has not been changed in any functional way, only some path changes.

Expected Result

cameras are launched, relevant topics are published at the settings configured in the yaml files ...

Actual Result

First the input configuration gets dumped to terminal as expected, then the lines below will follow:

[zed_wrapper-4] [ZED] [ERROR] [ZED] sl::Camera::Open has not been called, no Camera instance running. [zed_wrapper-4] [WARN] [cam1.node]: Error opening camera: CAMERA NOT DETECTED [zed_wrapper-4] [WARN] [cam1.node]: Please verify the camera connection.

After this, the node will crash after a few seconds with a camera detection timeout.

ZED Camera model

ZED Mini

Environment

OS: Ubuntu 20.04 variant pre-installed on the zedbox
ROS distro: ROS2 foxy
SDK version: 4.0.5, l4t 35.3
Jetpack version: 5.1.1
zed ros wrapper: master (1b8e1ae9e42f24b9bf4843f2022c52125c622a3c)
camera: zed-x mini

Anything else?

sometimes changing settings like turning off network connections seem to impact the problem, sometimes not. It's all feeling rather unstable in that sense.

Additionally, I think the max settings for multicam systems should also be noted more clearly. I notice for instance that running two cameras on the highest settings will cause the orin to draw too much power, sometimes drawing up to 26 watts. This may be part of the problem, but even at lower settings this issue will occur.

Myzhar commented 12 months ago

Hi @JHeuverRiwo there is a problem in the SDK while launching cameras simultaneously. You can try to add a delay and launch them in sequence.

A fix will be provided soon.

JHeuverRiwo commented 12 months ago

Hey,

Thanks for the suggestion and the quick answer. However just putting a delay between the nodes does not seem to resolve the issues for me. The result of launching them with a delay is a 'Critical camera error: CAMERA NOT INITIALIZED. Node stopped.'. The node crashing is the first camera launched, and it happens when the second camera launches. The second camera then is actually properly up by the looks of it though, although no pointcloud is published, but i'm not sure that's related.

github-actions[bot] commented 11 months ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment otherwise it will be automatically closed in 5 days

stereolabs / zed-ros-wrapper