Arducam-IMX219 fps reduction with isaac_ros_image_proc

[kaganerunsal]

Hello,

I am using a Jetson Xavier NX development board for my project. The power mode I set is "Mode 20W 6Core". I am using Arducam-IMX219, which are natively supported. I have built and run the docker successfully and then cloned and built "isaac_ros_image_pipeline" without any error. When the camera is seen at /dev/video0, I run the gscam2 node with the following command to obtain raw image topic:

export GSCAM_CONFIG="nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=NV12, framerate=(fraction)60/1 ! nvvidconv flip-method=0 ! video/x-raw, format=BGRx ! videoconvert"
ros2 launch gscam node_param_launch.py

which corresponds to 1280x720 resolution at 60fps according to v4l2-ctl --dev?dev/video0 --list-format-ext. Luckily, I obtain nearly 60fps with it (/image_raw topic).

However, when I run "isaac_ros_imag_proc" to obtain the rectified images with the command below, I see that /image_rect topic is publlshed at most 7-8 Hz which is very low compared to actual camera fps. Here, the frequency of "/image_raw" topic also reduces to 40fps. Possibly the board already uses its max resources.

What could be the reason for these reduction? Could it be my power settings or any GPU related settings? I am not sure if really my GPU is used? I would be glad if you enlighten me.

[hemalshahNV]

Could you try running the "jetson_clocks" script shown here (search for jetson_clocks) to make sure your NX is configured properly in addition to the power setting? It is possible this is affecting the rectification performance.

[hemalshahNV]

You can also use the isaac_ros_argus_camera_mono package instead of gscam as well as a more convenient option to access the IMX219. There is a significant performance impact from ROS2 middleware which we are working with Open Robotics (the organization that maintains ROS) as well.

[kaganerunsal]

Hello again,

I am not able use isaac_ros_argus_camera_mono because I am not able to get enough fps from the package. I opened another issue for this package. Currently I run gscam.

When I run sudo /usr/bin/jetson_clocksand sudo /usr/bin/jetson_clocks --show, i got the following output:

SOC family:tegra194  Machine:NVIDIA Jetson Xavier NX Developer Kit
Online CPUs: 0-5
cpu0: Online=1 Governor=schedutil MinFreq=1907200 MaxFreq=1907200 CurrentFreq=1907200 IdleStates: C1=0 c6=0 
cpu1: Online=1 Governor=schedutil MinFreq=1907200 MaxFreq=1907200 CurrentFreq=1907200 IdleStates: C1=0 c6=0 
cpu2: Online=1 Governor=schedutil MinFreq=1907200 MaxFreq=1907200 CurrentFreq=1907200 IdleStates: C1=0 c6=0 
cpu3: Online=1 Governor=schedutil MinFreq=1907200 MaxFreq=1907200 CurrentFreq=1907200 IdleStates: C1=0 c6=0 
cpu4: Online=1 Governor=schedutil MinFreq=1907200 MaxFreq=1907200 CurrentFreq=1907200 IdleStates: C1=0 c6=0 
cpu5: Online=1 Governor=schedutil MinFreq=1907200 MaxFreq=1907200 CurrentFreq=1907200 IdleStates: C1=0 c6=0 
GPU MinFreq=1109250000 MaxFreq=1109250000 CurrentFreq=1109250000
EMC MinFreq=204000000 MaxFreq=1866000000 CurrentFreq=1866000000 FreqOverride=1
Fan: PWM=0
NV Power Mode: MODE_20W_6CORE_MAX

This is a custom mode I set to get most power. I know this might be not healthy to work long periods with this setting but I wanted to see how much fps I get.

Still, I am able get at most 4-5 Hz for "/image_rect" topic. There are two strange things here. First jtop says that gpu is used very little:

Second, without using docker and ROS foxy, when i use ROS melodic and cpu implementation of image_pipeline, I am able to get at least 30 fps. I was expecting to get more fps from hardware accelerated version. Still, I am suspecting my hardware settings are not correct. Do you have any other suggestion? Do you have any test results with Xavier NX development board?

[kaganerunsal]

Note: I run jetson_clocks outside of docker and then run the container.

[hemalshahNV]

We can reproduce the issue on NX and still diagnosing the issue in the related issue. 4-5fps indicates something is very different between our qual setup. Thanks for bringing this to our attention!

[kaganerunsal]

You are welcome. Thank you for developing this tool. Good luck with the diagnosis.

[hemalshahNV]

https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_argus_camera/issues/1

[iemirosman]

Hi, @kaganGH @hemalshahNV is it fixed? I have the same issue with IMX477 and IMX219. My setup is Jetson Xavier NX 16GB, Jetpack 5.0.2, jetson-clocks on (frequencies are same with the kaganGH's Xavier).