search

IntelRealSense / realsense-ros

ROS Wrapper for Intel(R) RealSense(TM) Cameras
http://wiki.ros.org/RealSense
Apache License 2.0
2.51k stars 1.74k forks source link

D455 doesn't publish depth image when resolution is higher than 640x480 and infra is enabled #1377

Closed genzoo closed 3 years ago

genzoo commented 4 years ago

When i run: roslaunch realsense2_camera rs_camera.launch depth_width:=848 depth_height:=480 enable_infra1:=true

the ros topic /camera/depth/image_rect_raw doesn't publish data:

rostopic hz /camera/depth/image_rect_raw subscribed to [/camera/depth/image_rect_raw] no new messages no new messages no new messages no new messages ...

All topics publish data for depth resolutions lower than 848x480 with infra enabled. When infra is not enabled everything works well for all available depth resolutions. I experienced the same behavior using a D435i.

The sensor is attached to a USB3 port, I don't think this is a bandwidth problem.

MartyG-RealSense commented 4 years ago

Hi @genzoo Could you run rs-enumerate-devices please and see whether the "USB Descriptor" is USB 3, in order to check whether the camera is being detected as being on a USB 2.1 connection even though it is plugged into a USB3 port (this can happen). This will help to eliminate the possibility that it is related to USB2 bandwidth limitations.

image

genzoo commented 4 years ago

Hi @MartyG-RealSense:

Device info: Name : Intel RealSense D455 Serial Number : 026222250596 Firmware Version : 05.12.07.100 Recommended Firmware Version : 05.12.07.100 Physical Port : /sys/devices/pci0000:00/0000:00:14.0/usb2/2-2/2-2:1.0/video4linux/video0 Debug Op Code : 15 Advanced Mode : YES Product Id : 0B5C Camera Locked : YES Usb Type Descriptor : 3.2 Product Line : D400 Asic Serial Number : 025523051283 Firmware Update Id : 025523051283

MartyG-RealSense commented 3 years ago

Thank you very much. Do you experience any change in the behavior if you add a hardware reset instruction to your roslaunch statement please?

initial_reset:=true

genzoo commented 3 years ago

With the initial_reset statement the depth is published but not the infra (without initial_reset it's the opposite). I verified and as before the problem occurs when the depth resolution is higher than 640x480. The D435i has the same behavior. This is the roslaunch output:

roslaunch realsense2_camera rs_camera.launch depth_width:=848 depth_height:=480 enable_infra1:=true initial_reset:=true ... logging to /root/.ros/log/65cccba6-f740-11ea-8229-02421c8c91f5/roslaunch-IRBT-84087-7271.log Checking log directory for disk usage. This may take awhile. Press Ctrl-C to interrupt Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://IRBT-84087:40455/

SUMMARY

PARAMETERS

  • /camera/realsense2_camera/accel_fps: 250
  • /camera/realsense2_camera/accel_frame_id: camera_accel_frame
  • /camera/realsense2_camera/accel_optical_frame_id: camera_accel_opti...
  • /camera/realsense2_camera/align_depth: False
  • /camera/realsense2_camera/aligned_depth_to_color_frame_id: camera_aligned_de...
  • /camera/realsense2_camera/aligned_depth_to_fisheye1_frame_id: camera_aligned_de...
  • /camera/realsense2_camera/aligned_depth_to_fisheye2_frame_id: camera_aligned_de...
  • /camera/realsense2_camera/aligned_depth_to_fisheye_frame_id: camera_aligned_de...
  • /camera/realsense2_camera/aligned_depth_to_infra1_frame_id: camera_aligned_de...
  • /camera/realsense2_camera/aligned_depth_to_infra2_frame_id: camera_aligned_de...
  • /camera/realsense2_camera/allow_no_texture_points: False
  • /camera/realsense2_camera/base_frame_id: camera_link
  • /camera/realsense2_camera/calib_odom_file:
  • /camera/realsense2_camera/clip_distance: -2.0
  • /camera/realsense2_camera/color_fps: 30
  • /camera/realsense2_camera/color_frame_id: camera_color_frame
  • /camera/realsense2_camera/color_height: 480
  • /camera/realsense2_camera/color_optical_frame_id: camera_color_opti...
  • /camera/realsense2_camera/color_width: 640
  • /camera/realsense2_camera/depth_fps: 30
  • /camera/realsense2_camera/depth_frame_id: camera_depth_frame
  • /camera/realsense2_camera/depth_height: 480
  • /camera/realsense2_camera/depth_optical_frame_id: camera_depth_opti...
  • /camera/realsense2_camera/depth_width: 848
  • /camera/realsense2_camera/device_type:
  • /camera/realsense2_camera/enable_accel: False
  • /camera/realsense2_camera/enable_color: True
  • /camera/realsense2_camera/enable_depth: True
  • /camera/realsense2_camera/enable_fisheye1: False
  • /camera/realsense2_camera/enable_fisheye2: False
  • /camera/realsense2_camera/enable_fisheye: False
  • /camera/realsense2_camera/enable_gyro: False
  • /camera/realsense2_camera/enable_infra1: True
  • /camera/realsense2_camera/enable_infra2: False
  • /camera/realsense2_camera/enable_infra: False
  • /camera/realsense2_camera/enable_pointcloud: False
  • /camera/realsense2_camera/enable_pose: False
  • /camera/realsense2_camera/enable_sync: False
  • /camera/realsense2_camera/filters:
  • /camera/realsense2_camera/fisheye1_frame_id: camera_fisheye1_f...
  • /camera/realsense2_camera/fisheye1_optical_frame_id: camera_fisheye1_o...
  • /camera/realsense2_camera/fisheye2_frame_id: camera_fisheye2_f...
  • /camera/realsense2_camera/fisheye2_optical_frame_id: camera_fisheye2_o...
  • /camera/realsense2_camera/fisheye_fps: 30
  • /camera/realsense2_camera/fisheye_frame_id: camera_fisheye_frame
  • /camera/realsense2_camera/fisheye_height: 480
  • /camera/realsense2_camera/fisheye_optical_frame_id: camera_fisheye_op...
  • /camera/realsense2_camera/fisheye_width: 640
  • /camera/realsense2_camera/gyro_fps: 400
  • /camera/realsense2_camera/gyro_frame_id: camera_gyro_frame
  • /camera/realsense2_camera/gyro_optical_frame_id: camera_gyro_optic...
  • /camera/realsense2_camera/imu_optical_frame_id: camera_imu_optica...
  • /camera/realsense2_camera/infra1_frame_id: camera_infra1_frame
  • /camera/realsense2_camera/infra1_optical_frame_id: camera_infra1_opt...
  • /camera/realsense2_camera/infra2_frame_id: camera_infra2_frame
  • /camera/realsense2_camera/infra2_optical_frame_id: camera_infra2_opt...
  • /camera/realsense2_camera/infra_fps: 30
  • /camera/realsense2_camera/infra_height: 480
  • /camera/realsense2_camera/infra_width: 640
  • /camera/realsense2_camera/initial_reset: True
  • /camera/realsense2_camera/json_file_path:
  • /camera/realsense2_camera/linear_accel_cov: 0.01
  • /camera/realsense2_camera/odom_frame_id: camera_odom_frame
  • /camera/realsense2_camera/pointcloud_texture_index: 0
  • /camera/realsense2_camera/pointcloud_texture_stream: RS2_STREAM_COLOR
  • /camera/realsense2_camera/pose_frame_id: camera_pose_frame
  • /camera/realsense2_camera/pose_optical_frame_id: camera_pose_optic...
  • /camera/realsense2_camera/publish_odom_tf: True
  • /camera/realsense2_camera/publish_tf: True
  • /camera/realsense2_camera/rosbag_filename:
  • /camera/realsense2_camera/serial_no:
  • /camera/realsense2_camera/tf_publish_rate: 0.0
  • /camera/realsense2_camera/topic_odom_in: odom_in
  • /camera/realsense2_camera/unite_imu_method:
  • /camera/realsense2_camera/usb_port_id:
  • /rosdistro: kinetic
  • /rosversion: 1.12.14

NODES /camera/ realsense2_camera (nodelet/nodelet) realsense2_camera_manager (nodelet/nodelet)

auto-starting new master process[master]: started with pid [7281] ROS_MASTER_URI=http://localhost:11311

setting /run_id to 65cccba6-f740-11ea-8229-02421c8c91f5 process[rosout-1]: started with pid [7294] started core service [/rosout] process[camera/realsense2_camera_manager-2]: started with pid [7311] [ INFO] [1600166656.028668300]: Initializing nodelet with 12 worker threads. process[camera/realsense2_camera-3]: started with pid [7312] [ INFO] [1600166656.691101681]: RealSense ROS v2.2.17 [ INFO] [1600166656.691139911]: Built with LibRealSense v2.38.1 [ INFO] [1600166656.691167458]: Running with LibRealSense v2.38.1 [ INFO] [1600166656.699707484]:
[ INFO] [1600166656.710108791]: Device with serial number 026222250596 was found.

[ INFO] [1600166656.710142853]: Device with physical ID /sys/devices/pci0000:00/0000:00:14.0/usb2/2-2/2-2:1.0/video4linux/video0 was found. [ INFO] [1600166656.710162382]: Device with name Intel RealSense D455 was found. [ INFO] [1600166656.710555870]: Device with port number 2-2 was found. [ INFO] [1600166656.710584860]: Resetting device... [ INFO] [1600166662.729086019]:
[ INFO] [1600166662.746529216]: Device with serial number 026222250596 was found.

[ INFO] [1600166662.746635606]: Device with physical ID /sys/devices/pci0000:00/0000:00:14.0/usb2/2-2/2-2:1.0/video4linux/video0 was found. [ INFO] [1600166662.746709656]: Device with name Intel RealSense D455 was found. [ INFO] [1600166662.747768558]: Device with port number 2-2 was found.

[ INFO] [1600166662.910736342]: JSON file is not provided [ INFO] [1600166662.910772505]: ROS Node Namespace: camera [ INFO] [1600166662.910828826]: Device Name: Intel RealSense D455 [ INFO] [1600166662.910867822]: Device Serial No: 026222250596 [ INFO] [1600166662.910902785]: Device physical port: /sys/devices/pci0000:00/0000:00:14.0/usb2/2-2/2-2:1.0/video4linux/video0 [ INFO] [1600166662.910952607]: Device FW version: 05.12.07.100 [ INFO] [1600166662.910997512]: Device Product ID: 0x0B5C [ INFO] [1600166662.911029201]: Enable PointCloud: Off [ INFO] [1600166662.911105110]: Align Depth: Off [ INFO] [1600166662.911145118]: Sync Mode: Off [ INFO] [1600166662.911378065]: Device Sensors: [ INFO] [1600166662.949200295]: Stereo Module was found. [ INFO] [1600166662.979369051]: RGB Camera was found. [ INFO] [1600166662.980278637]: Motion Module was found. [ INFO] [1600166662.980310407]: num_filters: 0 [ INFO] [1600166662.980322377]: Setting Dynamic reconfig parameters. [ INFO] [1600166663.094980765]: Done Setting Dynamic reconfig parameters. [ INFO] [1600166663.096041351]: depth stream is enabled - width: 848, height: 480, fps: 30, Format: Z16 [ INFO] [1600166663.096366271]: infra1 stream is enabled - width: 640, height: 480, fps: 30, Format: Y8 [ INFO] [1600166663.097174248]: color stream is enabled - width: 640, height: 480, fps: 30, Format: RGB8

[ INFO] [1600166663.101609942]: Expected frequency for depth = 30.00000 [ INFO] [1600166663.151789238]: Expected frequency for infra1 = 30.00000 [ INFO] [1600166663.197959251]: Expected frequency for color = 30.00000

[ INFO] [1600166663.249950832]: insert Depth to Stereo Module [ INFO] [1600166663.250015018]: insert Color to RGB Camera [ INFO] [1600166663.250042542]: insert Infrared to Stereo Module [ INFO] [1600166663.282583536]: SELECTED BASE:Depth, 0 [ INFO] [1600166663.297697102]: RealSense Node Is Up!

Thanks for your help!

MartyG-RealSense commented 3 years ago

Do you experience the same behavior if you launch with roslaunch realsense2_camera rs_rgbd.launch instead of roslaunch realsense2_camera rs_camera.launch

genzoo commented 3 years ago

Yes the same behavior.

MartyG-RealSense commented 3 years ago

Going through the roslaunch output, I noticed something curious. The depth width and height is reported as 840x480 but the infra width and height is reported as 640x480.

image

genzoo commented 3 years ago

@MartyG-RealSense you found the problem! When the infra is enabled you have to specify the same resolution used by the depth, otherwise infra uses the default (640x480) and this cause the depth to be not published. Thanks!

MartyG-RealSense commented 3 years ago

I'm very pleased that the information was helpful to you. Thanks for the update!

Ankush1909IIT commented 1 year ago

D435_vs_D455_Depth_Error_Comparison

Ankush1909IIT commented 1 year ago

Depth RMS error comparison for a D435 vs D455 Intel Realsense Camera

Ankush1909IIT commented 1 year ago
Screen Shot 2022-10-03 at 12 28 13 AM
Ankush1909IIT commented 1 year ago

D435 - Subpixel = 0.08 Xres = 1280, HFOV = 90 degree, Baseline = 50 mm, Range = 0.3. to 3 m

D455 - Subpixel = 0.08 Xres = 1280, HFOV = 90 degree, Baseline = 95 mm, Range = 0.6 to 6 m

MartyG-RealSense commented 1 year ago

Hi @Ankush1909IIT There is not an RMS comparison chart available for D435 versus D455, though one could be plotted with the formula. However, the D415 has around 2x less error over distance than the D435 and the D455 is 2x as accurate over distance than D435. So it could be assumed that the green D415 line may be roughly representative of D435 versus D455.