CPU usage with two d435i in Jetson AGX Xavier

[tugbakara]

hi, the issue briefly is the title but I got observation like, when two cam runs in ROS wrapper at the same time, SDK runs twice behind don't know whether is correct or not? If it is correct running two cam with running SDK at once could be the solution? Firstly could you make it clear, then in this issue you mentioned about using usb_port_id instead serial no for multi cams makes more CPU friend approach. I couldn't find what to put usb_port_id two run it with two cam? Also to reduce CPU usage I tried different ros wrappers, SDKs and firmwares based on recommendation but nothing gives proper results. Briefly, what other suggestions can reduce CPU usage with two d435i? Kernel:4.9.253 JetPack: 4.6 Current RealSense Versions: SDK: 2.50 Firmware: 5.13.0.50 ROS-Wrapper: 2.3.2

[MartyG-RealSense]

Hi @tugbakara Using two cameras at the same time should not result in half the performance, especially on an Nvidia Jetson, as Jetsons can have multiple RealSense 400 Series cameras attached to them.

---

If the SDK has CUDA support enabled then it can offload work from the CPU onto the Jetson's Nvidia graphics GPU for three types of processing: YUY to RGB color conversion, depth-color alignment and pointcloud generation. These are the only types of processing that benefit from offloading from the CPU to GPU.

CUDA support is included automatically if the SDK is installed from the SDK's Jetson packages.

https://github.com/IntelRealSense/librealsense/blob/master/doc/installation_jetson.md#4-install-with-debian-packages

It can also be added to the SDK when building it from source code by including the build flag -DBUILD_WITH_CUDA=true in the CMake build instruction.

---

You could define a usb_port_id in the roslaunch instruction. For example: usb_port_id:=4-1

For a multiple camera launch, you can open two separate ROS terminals and have an individual roslaunch instruction for each camera, with a different usb_port_id in each roslaunch.

https://github.com/IntelRealSense/realsense-ros/tree/ros1-legacy#work-with-multiple-cameras

---

If you are using pointclouds or alignment on Jetson then you may experience better performance on Jetson if you use the launch file rs_rgbd.launch instead of rs_camera.launch. To use rs_rgbd.launch, you should first install RGBD launch support for the ROS1 branch (Kinetic, Melodic or Noetic) that you are using, as described at https://github.com/IntelRealSense/librealsense/issues/10223#issuecomment-1152864261

[tugbakara]

Hi @MartyG-RealSense :) I set usb_port_id instead serial-no in launch and used SDK 2.48 with 2.3.1 wrapper seperately and now realsenses look nice in CPU side, I use rs_rgbd.launch to create pointclouds and when I echo (subscribe into terminal) the topic of this launch CPU increases not so critically but my other package in ROS does not accept the data type. So I need to use filter that is voxel filter to decrease data and fit to package but it gives almost 2 times much more CPU load than realsense's pointcloud when I subscribe. I found the reason that voxel creates unordered pointcloud but realsense creates ordered that's why their array shapes are different, how can I achieve voxel type data in realsense? Also I searched about the filters that realsense has but these are for SDK pointclouds and works in SDK side, is it possible to use them in wrapper side with rs_rgbd.launch?

Here is voxel filter message type and realsense message type, respectively:

header: 
  seq: 0
  stamp: 
    secs: 1683110402
    nsecs: 222108841
  frame_id: "camera_front_color_optical_frame"
height: 1
width: 32608
fields: 
  - 
    name: "x"
    offset: 0
    datatype: 7
    count: 1
  - 
    name: "y"
    offset: 4
    datatype: 7
    count: 1
  - 
    name: "z"
    offset: 8
    datatype: 7
    count: 1
  - 
    name: "rgb"
    offset: 16
    datatype: 7
    count: 1
is_bigendian: False
point_step: 32
row_step: 1043456
data: [194, 188, 132, 62, 33, 251, 17, 190, 211, 77, 2, 63, 0, 0, 0, 0, 121, 128, 157, 255, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 105, 152, 133, 62, 33, 251, 17, 190, 211, 77, 2,...too much data to put that's why I cut]
is_dense: True

and also in the message there is another difference between the messages is dense , don't know whether affect package or not.

header: 
  seq: 3997
  stamp: 
    secs: 1683110025
    nsecs: 158569098
  frame_id: "camera_front_color_optical_frame"
height: 480
width: 640
fields: 
  - 
    name: "x"
    offset: 0
    datatype: 7
    count: 1
  - 
    name: "y"
    offset: 4
    datatype: 7
    count: 1
  - 
    name: "z"
    offset: 8
    datatype: 7
    count: 1
  - 
    name: "rgb"
    offset: 16
    datatype: 7
    count: 1
is_bigendian: False
point_step: 32
row_step: 20480
data: [238, 232, 73, 191, 253, 38, 24, 191, 169, 198, 187, 63, 0, 0, 0, 0, 204, 196, 203, 255, 0, ......too much data to put that's why I cut]
is_dense: False

and here is my cpu usage when I subscribe voxel (left side of the pic) and realsense(right side of the pic) , respectively:

[MartyG-RealSense]

The generation of a voxel pointcloud with librealsense using C++ and PCL is discussed at https://github.com/IntelRealSense/librealsense/issues/6411 though this would not be helpful if you need to produce voxel data in ROS.

rs_rgbd launch publishes an ordered pointcloud by default, whilst rs_camera.launch publishes an unordered pointcloud by default.

Data can be downsampled to reduce depth image complexity with the post-processing Decimation filter, as described for the ROS1 wrapper at https://github.com/IntelRealSense/realsense-ros/issues/1924#issuecomment-858369624

[tugbakara]

Is there any chance to change rs_rgbd to unordered?

[MartyG-RealSense]

I believe that an unordered cloud is 'dense' and an ordered cloud is not dense. So you could try adding to the rs_rgbd roslaunch instruction is_dense:=true

See https://github.com/IntelRealSense/realsense-ros/issues/1563#issuecomment-742439647 for further information.

[MartyG-RealSense]

Hi @tugbakara Do you require further assistance with this case, please? Thanks!

[tugbakara]

Thanks @MartyG-RealSense

[MartyG-RealSense]

You are very welcome, @tugbakara - thanks very much for the update.