Closed GEngels closed 1 year ago
Hi @GEngels Problems with pointcloud generation, and also enabling align_depth, on Jetson boards specifically are a known issue in the RealSense ROS wrapper. Symptoms can iclude very low FPS or missing color. The issue does not occur on non-Jetson computers such as laptop and desktop PCs. Detailed information about it can be found at https://github.com/IntelRealSense/realsense-ros/issues/1967
For ROS1 (Kinetic, Melodic, Noetic) the current best solution available is to use librealsense version 2.43.0 and ROS1 wrapper version 2.2.23.
Hi @MartyG-RealSense, I am a colleague of @GEngels. Thank you for your quick answer! We are currently using ROS2, is there also a version of the ROS2 wrapper that works flawlessly with the Jetson? And I guess the librealsense version will still be 2.43.0 in that case.
Hi @Doch88 The older RealSense ros2 wrapper branch would have to be used with older SDK versions, as the current ros2_beta wrapper branch is for SDK 2.50.0.
https://github.com/IntelRealSense/realsense-ros/tree/ros2
If SDK 2.43.0 is being used with ROS2 then it would have to be matched with ROS2 wrapper version 3.1.5
https://github.com/IntelRealSense/realsense-ros/releases/tag/3.1.5
This wrapper version only supports Foxy, Eloquent and Dashing though. So it would not be suitable for newer ROS2 versions such as Galactic and Rolling as support for those two ROS2 versions was introduced in more recent wrapper versions.
As far as I am aware, Jetson pointcloud generation has not been tested with the combination of SDK 2.43.0 and ROS2 wrapper 3.1.5 though, so I cannot offer a guarantee about pointcloud performance with that configuration.
Hi @MartyG-RealSense, I built the SDK 2.43.0 version from source (using RSUSB backend), with CUDA support, and I also used the ROS2 wrapper version 3.1.5 (with Foxy), built from source too. The problem seems to be still there. However, it seems to change depending on which Jetson power option is enabled.
What looks strange to me is that the 15W is performing better than the 30W one, I would guess that it depends on some specific power limitation set with these modes on the USB port or on the CPU cores.
EDIT: With the realsense-viewer everything works perfectly.
Thanks so much for the feedback from your tests!
When you refer to occluding an infrared camera, do you mean having it disabled in the ROS wrapper or covering over the lens on the outside of the camera?
Sorry, I mean covering the lens on the outside of the camera.
New update: We are using 1280x720 15 fps for the Depth and 1280x800 15 fps for the color. Disabling the spatial and temporal filtering seems to improve a lot the performance (still some delay tho) with those resolutions and FPS. But the problem appears again if I set the FPS to 30.
The Realsense-viewer works perfectly in the 3D view with all the post-processing filters enabled, the same resolutions, and the FPS set to 30.
How does it perform if depth and color are both set to 848x480 and 30 FPS?
MAXN: It works well with that resolution and FPS. MODE 30W ALL: There are 1 or 2 seconds of delay, but the point cloud and the RGB frame are published.
However, we need at least a resolution of 1280x720 with 15 FPS, preferably using MODE 30W ALL. With the realsense-viewer every setting that I tried worked perfectly, so nothing to notice there.
How about using 1280x720 and applying a Decimation filter to reduce the complexity of the depth scene? The Decimation filter will not work with align_depth = true though if you are using alignment.
Yes, we are using alignment, and also the Decimation filter will reduce the resolution of the depth image, which is not something that we want.
In your project, is 1280x720 a requirement for both depth and color or only for depth?
In your project, is 1280x720 a requirement for both depth and color or only for depth?
Yes, unfortunately it is a requirement for both frames.
Can you confirm please that when you built librealsense from source code with RSUSB, you included the build flag -DBUILD_WITH_CUDA=TRUE in the CMake build instruction? An RSUSB build of librealsense will automatically support metadata but require the DBUILD_WITH_CUDA flag to be used in order to add CUDA support to the build too.
I used this command to build the library:
cmake -DBUILD_WITH_CUDA=True -DCMAKE_BUILD_TYPE=Release -DFORCE_LIBUVC=true ..
I checked the differences between MODE 30W ALL and MODE 15W DESKTOP. The first one uses 8 core with a capped frequency for all of them of 1.2 Ghz. The latter, instead, uses 4 cores, and two of them have a capped frequency of 2.18 Ghz (The other twos still 1.2 Ghz). As the 15W DESKTOP one works better than the 30W one I guess that the problem is some CPU-intensive operation. As I know the spatial filtering and the temporal filtering are executed on the CPU, while I know nothing about the point cloud generation (I see that there is a function called rs2::gl::pointcloud for pointcloud generation using GPU, which is not used directly in the ROS2 node, but maybe it is used indirectly when building the library with CUDA).
Also I looked a bit into the code of the realsense-viewer and at a first sight I noticed that there a difference thread is used for the post-processing part of the code, while in the node (as I understood) everything is done in the same thread (except for some part not related to filtering and point cloud generation). Maybe that could also help a bit to understand what is the actual problem.
You mentioed earlier at https://github.com/IntelRealSense/realsense-ros/issues/2396#issuecomment-1175108099 that you could run well with 1280x720 depth and 1280x800 color (both at 15 FPS) but with spatial and temporal filters disabled. Are these filters vital to your ROS project, please?
Normally, Infra and Infra2 are disabled by default when using the rs_launch.py launch file. Are you using that launch file or a custom launch file, and if you are using a custom launch then are Infra and Infra2 enabled in it?
You mentioed earlier at https://github.com/IntelRealSense/realsense-ros/issues/2396#issuecomment-1175108099 that you could run well with 1280x720 depth and 1280x800 color (both at 15 FPS) but with spatial and temporal filters disabled. Are these filters vital to your ROS project, please?
It runs better, but there is still a bit of delay which we would like to avoid. Also, it would be nice to run everything with the best settings possible, as it works well with the realsense-viewer. So if it is possible to run it at 30 FPS that would be amazing. The spatial and temporal filters are not essential, tho.
By the way thank you for your support, I really like the fact that you are trying to reply as quickly and precisely as possible, so I am sorry if I am being a bit annoying!
Normally, Infra and Infra2 are disabled by default when using the rs_launch.py launch file. Are you using that launch file or a custom launch file, and if you are using a custom launch then are Infra and Infra2 enabled in it?
We are using a custom launch file that uses rs_launch.py inside, we are not touching the settings regarding infra and infra2.
These are the launch arguments, many of them are useless for the current camera but they are still provided for internal purposes:
launch_arguments={
'device_type': 'd455',
'enable_gyro': 'false',
'enable_accel': 'false',
'enable_sync': 'true',
'initial_reset': 'true',
'align_depth': 'true',
'enable_pointcloud': 'true',
'ordered_pc': 'true',
'enable_fisheye1': 'false',
'enable_fisheye2': 'false',
'enable_confidence': 'false',
'depth_fps': '15.0',
'depth_width': '1280',
'depth_height': '720',
'color_fps': '15.0',
'color_width': '1280',
'color_height': '800',
'filters': '""',
'clip_distance': '10.0',
}
It's no trouble at all :)
If librealsense is built with the flag -DBUILD_WITH_OPENMP=TRUE then YUY to RGB color conversion and depth-color alignment can take advantage of multiple CPU cores, at the expense of slightly higher CPU percentage utilization.
rs2::gl, also known as GLSL, is an alternative acceleration system to CUDA that is built into librealsense and works with any graphics GPU brand instead of just Nvidia. It is not involved in CUDA processing. There would not be a significant performance advantage to using GLSL over CUDA and since GLSL only works with rs2:: instructions in librealsense scripts, CUDA will be the best choice for ROS.
Unless you need an ordered pointcloud instead of the default unordered one then you could try removing the ordered_pc instruction to see if it makes a positive difference.
As you are using both depth and color, if auto-exposure is enabled then you could also try disabling an RGB setting called auto_exposure_priority to force the FPS of the two streams to remain at the defined FPS (such as '30') instead of being permitted to drop to a lower FPS value. Information about defining this setting in a launch file as a rosparam is at https://github.com/IntelRealSense/realsense-ros/issues/2308#issuecomment-1086628767
Thank you for your clarifications! I will try to build librealsense today with OpenMP enabled and see if it improves the results.
Yes, we need ordered_pc and I have also tried to disable it and the performances did not improve visibly.
I cannot find how to set that parameter with ROS2, that setting in the launch file seems to be a ROS1 one. Also, I cannot find the definition of that parameter inside of the Realsense node, can you provide me a way to do that with ROS2?
Are you able to test disabling auto_exposure_priority by opening the rqt_reconfigure interface in the 3.x ROS2 wrapper with the command ros2 run rqt_reconfigure rqt_reconfigure
If you can open the rqt_reconfigure interface and the side-panel has an rgb_camera category in its side-panel (camera > rgb_camera) then the auto_exposure_priority option should be under that section's options.
For some reason rqt_reconfigure was not working, but I managed to change the setting with
ros2 param set /camera/camera rgb_camera.auto_exposure_priority false
Although nothing really changed that much.
Also, I just saw that disabling everything (including pointcloud generation and depth alignment) on Rviz appears to be quite ok, but instead, doing ros2 topic hz /camera/color/image_raw
, I realize that the publishing rate is always lower than expected.
If I set the camera to run at 15 fps, the publishing rate is around 10 Hz; if I set the camera to run at 30 fps the publishing rate is around 20 Hz (1280x720 for both cases). Don't know then if it is a problem of ros2 topic hz
or of the realsense node.
That happens also if I disable auto_exposure_priority and if I build with -DBUILD_WITH_OPENMP.
Thanks very much for your tests.
When auto_exposure_priority is set to False, is the RGB auto_exposure also set to True? If auto_exposure is true and auto_exposure_priority is false then the FPS rate should be forced to remain constant.
Thanks very much for your tests.
When auto_exposure_priority is set to False, is the RGB auto_exposure also set to True? If auto_exposure is true and auto_exposure_priority is false then the FPS rate should be forced to remain constant.
Yes, the RGB auto_exposure is also set to True, I already checked them because I saw another issue where someone (I think you) replied that it should be True when auto_exposure_priority is false, in order to get constant FPS. Here's a picture of the hz of the camera image with everything disabled. This is a screenshot of the startup of the camera. (wrapper v. 3.1.5, librealsense 2.43.0) I also tried to run the camera both with ROS Foxy and ROS Elegant, using docker containers, nothing really changed.
On realsense-viewer the reported FPS are always ~30.0 for both frames. Although something that I noticed is that disabling "Use GLSL for processing" on the realsense-viewer the frame drop rate becomes quite high, around 30%, with sometimes 45% and sometimes 20%. If I use "Camera timestamp rate" as a metric to measure the frame drop rate it becomes constantly 50% instead.
That seems strange to me, since as you said it should use CUDA by default and using GLSL should not make any difference, having built it from source with CUDA enabled.
GLSL differs from CUDA in that programs will not make use of GLSL acceleration unless support for doing so has been deliberately coded into that program, like it is in the RealSense Viewer. So the ROS wrapper would not be benefiting from GLSL.
CUDA acceleration of pointclouds, alignment and color conversion is provided automatically though if the librealsense SDK has CUDA support enabled (by building the SDK from packages, or from source code with the DBUILD_WITH_CUDA=true flag included).
CUDA support is not included though if the SDK and the ROS wrapper were installed together from packages at the same time using the wrapper's Method 1 instructions.
Then I do not really understand this difference of performance between the node and the realsense-viewer. Another thing that I saw is that the USB power consumption seems to be quite low, even when used:
jetsonxavier@ubuntu:~$ lsusb -v | egrep "^Bus|MaxPower"
Bus 002 Device 004: ID 8086:0b5c Intel Corp.
MaxPower 180mA
I would expect it to be around 700 mA, as it is when I run it in a laptop. Apart from that, I do not really know what else to look, I feel like I checked everything.
The RealSense Viewer tool runs directly in the RealSense SDK, whilst the ROS wrapper runs as a layer atop the SDK in the background. The wrapper also handles some functions differently from librealsense in order to maintain compatibility with ROS standards. So the camera may work correctly in the Viewer but have some performance issues in the ROS wrapper for some RealSense users.
A Jetson user at https://github.com/IntelRealSense/realsense-ros/issues/1964 who was also using MAXN mode had performance issues such as very high CPU % utilization. They provided extensive testing logs in that case. In the end, they found that performance improved if they removed the ROS wrapper with the command sudo apt remove ros-$ROS_DISTRO-librealsense2 and rebuilt it from source code.
Hi @GEngels and @Doch88 Do you require further assistance with this case, please? Thanks!
Hi @MartyG-RealSense,
I'm a colleague of @GEngels and @Doch88, and I'm taking over this issue. I'm currently writing a first long message to try to be the most complete possible.
Thank you.
Thanks very much @iraadit for the introduction. I'm pleased to meet you and look forward to your forthcoming message!
Hi @MartyG-RealSense,
Here is a recap of my setup:
First, should ros2_beta be considered production ready? Is ros2-beta branch having the last development / being compatible with the last librealsense version, while ros2 branch is stale?
We aim at getting the maximum output quality. Our application should target precision and reactivity. Therefore, we want to be able to get the point cloud at max resolution (1280x720) and 30FPS. It would be preferable to get a colored texture stream (_RS2_STREAMCOLOR), but no texture stream (_RS2_STREAMANY) would be acceptable for now (with align_depth:=true though). Soon, we will want to be able to use 2 realsense cameras for one Jetson AGX Xavier. And later on, possibly 4 cameras. It could later be considered to get higher FPS at the expense of the resolution (up to 90FPS at 848x480).
I checked several issues on this Github repository as well as on the Nvidia forum. I didn't find any definitive answer on how to get good point cloud performances on Jetson, without sacrificing output quality or downgrading librealsense and realsense-ros (which seemed to be a solution for ROS1). With your help, I'd like to go to the end of this issue and to find a working configuration. I'd be ready to delve into code if necessary and to publish a pull request if we find a solution.
Here is a collection of the observations I made until now:
$ launch realsense2_camera rs_launch.py enable_pointcloud:=true initial_reset:=true pointcloud_texture_stream:=RS2_STREAM_COLOR depth_width:=1280 depth_height:=720 depth_fps:=30.0
$ ros2 launch realsense2_camera rs_launch.py enable_pointcloud:=true initial_reset:=true pointcloud_texture_stream:=RS2_STREAM_ANY depth_width:=1280 depth_height:=720 depth_fps:=30.0
$ ros2 launch realsense2_camera rs_launch.py enable_pointcloud:=true initial_reset:=true pointcloud_texture_stream:=RS2_STREAM_ANY depth_width:=1280 depth_height:=720 depth_fps:=30.0 align_depth:=true
$ ros2 launch realsense2_camera rs_launch.py pointcloud.enable:=true depth_module.profile:=1280x720x30 decimation_filter.enable:=false pointcloud.stream_filter:=2
The publication rates are still slow and variable though, as shown by the next command:
$ ros2 topic hz /camera/depth/color/points -w 10
average rate: 9.935
min: 0.055s max: 0.141s std dev: 0.03570s window: 10
average rate: 7.864
min: 0.050s max: 0.265s std dev: 0.07520s window: 10
average rate: 6.858
min: 0.064s max: 0.334s std dev: 0.08832s window: 10
average rate: 11.541
min: 0.063s max: 0.209s std dev: 0.04425s window: 10
average rate: 6.488
min: 0.064s max: 0.397s std dev: 0.10397s window: 10
average rate: 7.923
min: 0.057s max: 0.271s std dev: 0.08429s window: 10
average rate: 7.141
min: 0.061s max: 0.472s std dev: 0.12704s window: 10
average rate: 5.008
min: 0.062s max: 0.530s std dev: 0.18022s window: 10
$ ros2 topic hz /camera/color/image_raw -w 10
average rate: 18.758
min: 0.027s max: 0.108s std dev: 0.02769s window: 10
average rate: 26.809
min: 0.021s max: 0.068s std dev: 0.01301s window: 10
average rate: 30.423
min: 0.020s max: 0.040s std dev: 0.00570s window: 10
average rate: 23.287
min: 0.020s max: 0.100s std dev: 0.02405s window: 10
average rate: 13.814
min: 0.020s max: 0.228s std dev: 0.06253s window: 10
average rate: 21.465
min: 0.025s max: 0.086s std dev: 0.01977s window: 10
average rate: 21.363
min: 0.025s max: 0.069s std dev: 0.01700s window: 10
average rate: 18.522
min: 0.027s max: 0.109s std dev: 0.02873s window: 10
For the delays:
$ ros2 topic delay /camera/depth/color/points -w 10
average delay: 0.063
min: 0.055s max: 0.071s std dev: 0.00505s window: 10
average delay: 0.067
min: 0.060s max: 0.071s std dev: 0.00361s window: 10
average delay: 0.064
min: 0.060s max: 0.070s std dev: 0.00307s window: 10
average delay: 0.064
min: 0.059s max: 0.070s std dev: 0.00333s window: 10
average delay: 0.065
min: 0.061s max: 0.074s std dev: 0.00410s window: 10
$ ros2 topic delay /camera/color/image_raw -w 10
average delay: 0.059
min: 0.053s max: 0.068s std dev: 0.00410s window: 10
average delay: 0.058
min: 0.053s max: 0.065s std dev: 0.00451s window: 10
average delay: 0.055
min: 0.051s max: 0.063s std dev: 0.00399s window: 10
$ ros2 launch realsense2_camera rs_launch.py pointcloud.enable:=true decimation_filter.enable:=false depth_module.profile:=1280x720x30 pointcloud.stream_filter:=2 diagnostics_period:=1.0 enable_sync:=true align_depth:=true
$ ros2 topic echo /diagnostics
header:
stamp:
sec: 1658760550
nanosec: 738734464
frame_id: ''
status:
- level: "\0"
name: 'amera: Temperatures'
message: OK
hardware_id: '146222254978'
values:
- key: Asic Temperature
value: '45'
- key: Projector Temperature
value: '40'
- level: "\x01"
name: 'amera: depth'
message: Frequency too low.
hardware_id: '146222254978'
values:
- key: Events in window
value: '150'
- key: Events since startup
value: '8139'
- key: Duration of window (s)
value: '9.999573'
- key: Actual frequency (Hz)
value: '15.000640'
- key: Target frequency (Hz)
value: '30.000000'
- key: Minimum acceptable frequency (Hz)
value: '27.000000'
- key: Maximum acceptable frequency (Hz)
value: '33.000000'
- level: "\0"
name: 'amera: color'
message: Desired frequency met
hardware_id: '146222254978'
values:
- key: Events in window
value: '300'
- key: Events since startup
value: '16281'
- key: Duration of window (s)
value: '9.999632'
- key: Actual frequency (Hz)
value: '30.001105'
- key: Target frequency (Hz)
value: '30.000000'
- key: Minimum acceptable frequency (Hz)
value: '27.000000'
- key: Maximum acceptable frequency (Hz)
value: '33.000000'
With ros2 topic echo /diagnostics, the actual frequency for the color camera is around 30 Hz and is the same as the target frequency (good!). But the depth camera actual frequency is around 15 Hz, half what it should be. I tried running with smaller depth resolutions and enabled decimation, but the actual depth frequency is always staying at 15 Hz. So, what numbers should I trust? The ones from ros2 topic hz, or from ros2 topic echo /diagnostics ? Any idea why the actual depth frequency would saturate at 15 Hz? I also tried with combinations of ordered_pc:=true, align_depth:=true and enable_sync:=true, without seeing any drastic changes. Looking at Rviz visualization, the output seems good to me. I would know like to be sure we get a 30FPS pointcloud.
$ ros2 run rqt_image_view rqt_image_view
There is a more visible delay in Rviz with the pointcloud
I'll continue to look into it in the coming days.
Thank you for your help.
Thanks so much @iraadit for your very detailed report.
A good rule to apply in a production situation is "one version back" - using one version older than the current version in order to give time for reports of any problems to emerge with a new version before committing to using it in a production or critical reliability environment. So it may be worth sticking with 4.0.4 for a while even if a newer beta wrapper is released.
The current known issues with using pointcloud and align with Jetson boards specifically (such as FPS drop or missing color) should apply to both the ros2 and ros2_beta wrappers though. Also, it is likely that there will be a performance drop with using 1280x720 compared to 848x480, especially if using 90 FPS.
Typically, the maximum number of RealSense 400 Series cameras that have been attached to the same Jetson boad via a hub has been six. So four cameras would certainly be an achievable target.
If performance degrades and then is corrected by the auto-exposure after placing a hand over the lenses, this can indicate that the infrared sensor may have become saturated with light by a cause such as a strong light source (such as sunlight) flashing directly into the camera's view.
As mentioned at https://github.com/IntelRealSense/realsense-ros/issues/2396#issuecomment-1176356778 earlier in this discussion, a common method used to deal with differences between depth and RGB streams, such as FPS lag or one stream being 30 FPS and the other 15, is to have auto-exposure enabled and the RGB option auto_exposure_priority disabled.
Hi @iraadit Do you require further assistance with this case, please? Thanks!
Hi @MartyG-RealSense,
Yes, I'm still working on it / trying to find a solution.
First, answering your last comments:
It is good to know that 6 cameras have already been achieved on Jetson. Was it while using poincloud though?
The problem doesn't seem to come from a saturated infrared sensor, the luminosity of the room isn't changing the results. it works with smaller resolutions though.
rgb_camera.enable_auto_exposure is enabled and rgb_camera.auto_exposure_priority disabled; these are their default values. I tried playing with those values (as well as with rgb_camera.exposure) with rqt_reconfigure and didn't see real changes; BUT I saw changes when changing depth_module.exposure. By default, depth_module.enable_auto_exposure is disabled and the value of depth_module.exposure is 33000 (33ms). Reducing this value to 32000 will increase the FPS of depth at the expense at the FPS of color, and more instability (pointcloud freezing sometime). Do you have an idea of why chaning this parameter would change the FPS of the color camera?
It seems indeed that the color and depth frames are going out of sync, leading to the pointcloud problem.
I reinstalled my Jetson with Jetpack 4.6.2 (the last version 4) to gain access to the latest Nvidia Nsight profiling tools.
I'm now able to debug and profile realsense-ros as well as librealsense.
I can also share that the problem is present with:
From here, I'll dive into code to try to find what is going wrong.
By the way, as it seems it is a problem that occurred often for other users too, and that it is a showstopper for the use of pointclouds with Realsense and Jetson, do you know if the internal dev time of Intel Realsense are working on it?
Thank you
Typically the 6 cameras have been used with basic depth / color streams rather than pointclouds. If CUDA support is enabled in the librealsense SDK though then the processing burden of generating a pointcloud should be low, as the Nvidia graphics GPU on the Jetson can process the pointcloud instead of the CPU.
CUDA support is automatically included in librealsense if it is built from packages or built from source code with CMake with the build term -DBUILD_WITH_CUDA included in the CMake build instruction.
CUDA support is not included if building librealsense and the wrapper together at the same time from packages with the RealSense ROS wrapper's Method 1 installation instructions though, but that would only be applicable to the previous-generation ros2 branch rather than ros2_beta.
If auto-exposure is enabled then setting a manual exposure value will automatically disable auto- exposure. When using manual exposure, setting values in a certain range can increase or decrease FPS. More information about this is at https://github.com/IntelRealSense/librealsense/issues/1957#issuecomment-400715579
The Jetson-specific issues with pointclouds and alignment are being worked on, though there is not a date for implementation at the time of writing this.
Hi @iraadit Was the information in the comment above helpful to you, please? Thanks!
Hi @MartyG-RealSense I'll answer tomorrow Still not there, but slowly going in the right direction :)
Hi @iraadit Do you have an update about this case that you can provide, please? Thanks!
Hi @MartyG-RealSense, Again sorry for the delayed answer.
I passed the last days installing Jetpack 5 and Isaac ROS (Humble with Docker) That took me some time but I now have librealsense and realsense-ros working in the Docker. I hoped new versions of ROS packages would maybe help, it seems at first it isn't the case. I'll continue to explore with this version though.
In the discoveries of the last weeks:
I can get 30 FPS in color and depth (without pointcloud) by setting _depth_module.profile:=1280x720x30 depth_module.exposure:=32000 depth_module.enable_autoexposure:=false in the _ros2 launch realsense2_camera rslaunch.py command (confirmed using the topic /diagnostics). I also had to modifiy _rslaunch.py to accept those new parameters.
I profiled realsense-viewer and realsense-ros with Nsight, running on the Jetson. realsense-viewer is using the OpenGL version of the filters, while realsense-ros is using a "basic CPU" version. For poincloud, on a x86_64 host computer, the executed code is in SSE3 (what is not possible on aarch64/Jetson), already better than the "basic CPU" version. On Jetson with realsense-ros, poincloud operations is what is using must of the CPU time and the problematic function is _get_texturemap. The CUDA flag is only changing the function _depth_topoints. While the OpenGL and SSE3 versions are changing _get_texturemap. I modified rs-benchmark to take a 1280x720 depth image as input, here is the output:
$ rs-benchmark
CPU | ARMv8 Processor rev 0 (v8l) |
GPU | NVIDIA Tegra Xavier (nvgpu)/integrated |
Graphics Driver | 4.6.0 NVIDIA 32.7.2 |
Device Name | Intel RealSense D455 |
Stream Type: Depth, Resolution: 1280 x 720
Filter Name | Step | Median(m) | Mean(m) | STD(m) | Max(m) | Max FPS |
---|---|---|---|---|---|---|
colorizer | Calculate | 19.138 | 19.773 | 4.766 | 76.215 | 30 |
pointcloud | Calculate | 4.561 | 8.947 | 49.191 | 609.226 | 6 |
spatial_filter | Calculate | 56.714 | 56.782 | 4.908 | 103.292 | 15 |
temporal_filter | Calculate | 7.653 | 7.970 | 0.753 | 10.122 | 90 |
disparity_transform | Calculate | 4.532 | 4.764 | 0.716 | 11.145 | 90 |
threshold_filter | Calculate | 3.982 | 4.537 | 0.831 | 8.814 | 90 |
decimation_filter | Calculate | 3.402 | 3.603 | 0.538 | 6.285 | 90 |
gl::colorizer | Upload | 3.602 | 4.205 | 1.912 | 21.720 | 90 |
Calculate | 1.357 | 1.951 | 1.291 | 8.280 | 90 | |
Download | 1.685 | 2.339 | 1.325 | 9.559 | 90 | |
Total | 8.373 | 8.495 | 3.116 | 30.229 | 60 | |
gl::pointcloud | Upload | 3.556 | 4.226 | 1.446 | 9.747 | 90 |
Calculate | 1.223 | 1.704 | 2.445 | 29.984 | 90 | |
Download | 0.002 | 0.008 | 0.073 | 0.903 | 90 | |
Total | 5.043 | 5.939 | 3.059 | 37.744 | 90 |
Stream Type: Color, Resolution: 1280 x 800
Filter Name | Step | Median(m) | Mean(m) | STD(m) | Max(m) | Max FPS |
---|---|---|---|---|---|---|
yuy_decoder | Calculate | 2.108 | 2.462 | 1.046 | 6.506 | 90 |
gl::yuy_decoder | Upload | 1.834 | 2.377 | 1.131 | 6.112 | 90 |
Calculate | 1.159 | 1.583 | 0.991 | 5.516 | 90 | |
Download | 1.651 | 2.032 | 1.166 | 10.268 | 90 | |
Total | 5.276 | 5.993 | 1.970 | 17.424 | 90 |
It can be seen that _gl::pointcloud_ is faster than _pointcloud_ (6ms vs 9ms, maximum of 38 vs 609ms)
From there, I changed the code of realsense-ros to use OpenGL, getting some inspiration from https://github.com/IntelRealSense/librealsense/tree/master/examples/gl
I know have _get_texture_map_ going faster, less CPU usage and better streams. I'm still not satisfied with the result though.
I also tried to change _align_depth_ by its OpenGL version, but with no luck until now, because of an error about an OpenGL texture.
OpenGL upload should also be used to avoid sending the frames several times to the GPU.
I'll also maybe try to convert the SSE3 instructions to NEON instructions to make it compatible with aarch64.
3. Now, hz of poincloud is higher, but FastDDS seemed to badly cope with the transfer of the data, I switched to Cyclone DDS and got better results.
4. When running some example app, it is possible to load a config file to the camera. Behavior of the camera will be different after applying this config (and until a reset). Is is possible to get a faster pointcloud but losing a certain amount of points (higher confidence threshold). It could be explored but we would like to keep the highest quality possible.
I'll be on vacations until the start of September and will continue to work on this when I will be back. Please, do not close the ticket :)
Thank you
Thanks so much @iraadit for the highly detailed feedback of your findings. It is no problem at all to keep the ticket open whilst you are on vacation and resume when you return.
@iraadit Thank you for sharing @MartyG-RealSense what about Humble ROS support on Jetson Docker? Just about Humble aarch64 support? Could you add on it, please?
Hi @AndreV84 Humble is now supported by the RealSense ros2_beta wrapper, though not by earlier wrapper branches (ROS1 and ros2). At the time of writing this, only building from source code is supported for Humble. Ubuntu 22.04 (Jammy) Debians are on the roadmap. https://github.com/IntelRealSense/librealsense/issues/10439#issuecomment-1186588624 and the comments beneath it provide further information.
Gld it can help you @AndreV84 :)
Hi @MartyG-RealSense
I've been able to install the last librealsense v2.51.1 as well as the branch ros2-beta of realsense-ros inside a docker container created with Nvidia Isaac ROS (with ROS 2 Humble). I could install following the first method of the Jetson install (with _FORCE_RSUSBBACKEND=false), without needing to apply the kernel patching. Running on a Nvidia Jetson AGX Xavier.
Sadly, the performances are staying the same.
ROS2 logs are now working correctly though and I observed something in the output of the command ros2 launch realsense2_camera rs_launch.py pointcloud.enable:=true decimation_filter.enable:=false pointcloud.stream_filter:=2 diagnostics_period:=1.0 enable_sync:=true align_depth.enable:=false ordered_pc:=true log_level:=debug output:=log camera_name:=realsense_cam depth_module.profile:=1280x720x30 depth_module.exposure:=32000 depth_module.enable_auto_exposure:=false
:
[realsense2_camera_node-1] [DEBUG] [1661793531.267126566] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793531.267218956] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 1
[realsense2_camera_node-1] [DEBUG] [1661793531.267279728] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8748 ; frame_TS: 1661793531226.631592 ; ros_TS(NSec): 1661793531226631680
[realsense2_camera_node-1] [DEBUG] [1661793531.267353108] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793531.276654409] [FrequencyStatus_debug_logger]: TICK 5941
[realsense2_camera_node-1] [DEBUG] [1661793531.288720997] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 2
[realsense2_camera_node-1] [DEBUG] [1661793531.288904336] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8748 ; frame_TS: 1661793531226.631592 ; ros_TS(NSec): 1661793531226631680
[realsense2_camera_node-1] [DEBUG] [1661793531.289124126] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16) frame. frame_number: 8748 ; frame_TS: 1661793531226.631592 ; ros_TS(NSec): 1661793531226631680
[realsense2_camera_node-1] [DEBUG] [1661793531.289208483] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.289259942] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.291451211] [realsense_cam.realsense_cam]: Depth stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.291857412] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793531.291943465] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 1
[realsense2_camera_node-1] [DEBUG] [1661793531.291999340] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8749 ; frame_TS: 1661793531259.886963 ; ros_TS(NSec): 1661793531259886848
[realsense2_camera_node-1] [DEBUG] [1661793531.292080369] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793531.297640419] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 2
[realsense2_camera_node-1] [DEBUG] [1661793531.297817774] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8749 ; frame_TS: 1661793531259.886963 ; ros_TS(NSec): 1661793531259886848
[realsense2_camera_node-1] [DEBUG] [1661793531.298029306] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16) frame. frame_number: 8749 ; frame_TS: 1661793531259.886963 ; ros_TS(NSec): 1661793531259886848
[realsense2_camera_node-1] [DEBUG] [1661793531.298109599] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.298160610] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.301872740] [realsense_cam.realsense_cam]: Depth stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.302473512] [FrequencyStatus_debug_logger]: TICK 6498
[realsense2_camera_node-1] [DEBUG] [1661793531.306614820] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793531.306867859] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 1
[realsense2_camera_node-1] [DEBUG] [1661793531.306991674] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8 3) frame. frame_number: 8740 ; frame_TS: 1661793530860.797363 ; ros_TS(NSec): 1661793530860797440
[realsense2_camera_node-1] [DEBUG] [1661793531.307089664] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793531.307221416] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 1
[realsense2_camera_node-1] [DEBUG] [1661793531.307278764] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8) frame. frame_number: 8740 ; frame_TS: 1661793530860.797363 ; ros_TS(NSec): 1661793530860797440
[realsense2_camera_node-1] [DEBUG] [1661793531.307358673] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.307410324] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.309945102] [FrequencyStatus_debug_logger]: TICK 5942
[realsense2_camera_node-1] [DEBUG] [1661793531.311142390] [realsense_cam.realsense_cam]: Color stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.311711065] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793531.311790046] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 1
[realsense2_camera_node-1] [DEBUG] [1661793531.311861218] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8750 ; frame_TS: 1661793531293.141846 ; ros_TS(NSec): 1661793531293141760
[realsense2_camera_node-1] [DEBUG] [1661793531.311935783] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793531.322884767] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 2
[realsense2_camera_node-1] [DEBUG] [1661793531.323059754] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8750 ; frame_TS: 1661793531293.141846 ; ros_TS(NSec): 1661793531293141760
[realsense2_camera_node-1] [DEBUG] [1661793531.323296984] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16) frame. frame_number: 8750 ; frame_TS: 1661793531293.141846 ; ros_TS(NSec): 1661793531293141760
[realsense2_camera_node-1] [DEBUG] [1661793531.323403039] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.323478755] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.325643815] [realsense_cam.realsense_cam]: Depth stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.325952793] [FrequencyStatus_debug_logger]: TICK 6499
[realsense2_camera_node-1] [DEBUG] [1661793531.329300837] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793531.329459534] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 1
[realsense2_camera_node-1] [DEBUG] [1661793531.329527410] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8 3) frame. frame_number: 8742 ; frame_TS: 1661793530927.321289 ; ros_TS(NSec): 1661793530927321344
[realsense2_camera_node-1] [DEBUG] [1661793531.329597911] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793531.329693212] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 1
[realsense2_camera_node-1] [DEBUG] [1661793531.329755520] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8) frame. frame_number: 8742 ; frame_TS: 1661793530927.321289 ; ros_TS(NSec): 1661793530927321344
[realsense2_camera_node-1] [DEBUG] [1661793531.329834693] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.329915338] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.333937406] [realsense_cam.realsense_cam]: Color stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.334200558] [FrequencyStatus_debug_logger]: TICK 6500
[realsense2_camera_node-1] [DEBUG] [1661793531.338594201] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793531.338807686] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 1
[realsense2_camera_node-1] [DEBUG] [1661793531.338920845] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8 3) frame. frame_number: 8745 ; frame_TS: 1661793531027.102539 ; ros_TS(NSec): 1661793531027102464
[realsense2_camera_node-1] [DEBUG] [1661793531.339019315] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793531.339171644] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 1
[realsense2_camera_node-1] [DEBUG] [1661793531.339247328] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8) frame. frame_number: 8745 ; frame_TS: 1661793531027.102539 ; ros_TS(NSec): 1661793531027102464
[realsense2_camera_node-1] [DEBUG] [1661793531.339446829] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.339590261] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.344026787] [realsense_cam.realsense_cam]: Color stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.344322133] [FrequencyStatus_debug_logger]: TICK 5943
[realsense2_camera_node-1] [DEBUG] [1661793531.344363575] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793531.345281103] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 1
[realsense2_camera_node-1] [DEBUG] [1661793531.345347987] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8751 ; frame_TS: 1661793531326.465576 ; ros_TS(NSec): 1661793531326465536
[realsense2_camera_node-1] [DEBUG] [1661793531.345439704] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793531.360340065] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 2
[realsense2_camera_node-1] [DEBUG] [1661793531.360553998] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8751 ; frame_TS: 1661793531326.465576 ; ros_TS(NSec): 1661793531326465536
[realsense2_camera_node-1] [DEBUG] [1661793531.360782716] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16) frame. frame_number: 8751 ; frame_TS: 1661793531326.465576 ; ros_TS(NSec): 1661793531326465536
[realsense2_camera_node-1] [DEBUG] [1661793531.360865697] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.360918020] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.363152236] [realsense_cam.realsense_cam]: Depth stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.363438013] [FrequencyStatus_debug_logger]: TICK 6501
[realsense2_camera_node-1] [DEBUG] [1661793531.366930833] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793531.367130077] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 1
[realsense2_camera_node-1] [DEBUG] [1661793531.367227299] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8 3) frame. frame_number: 8754 ; frame_TS: 1661793531326.479492 ; ros_TS(NSec): 1661793531326479616
[realsense2_camera_node-1] [DEBUG] [1661793531.367320969] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793531.367446960] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 1
[realsense2_camera_node-1] [DEBUG] [1661793531.367522229] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8) frame. frame_number: 8754 ; frame_TS: 1661793531326.479492 ; ros_TS(NSec): 1661793531326479616
[realsense2_camera_node-1] [DEBUG] [1661793531.367663581] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.367767076] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.371095886] [realsense_cam.realsense_cam]: Color stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.371365982] [FrequencyStatus_debug_logger]: TICK 6502
[realsense2_camera_node-1] [DEBUG] [1661793531.376538712] [FrequencyStatus_debug_logger]: TICK 5944
[realsense2_camera_node-1] [DEBUG] [1661793531.392993599] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793531.393191179] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 2
[realsense2_camera_node-1] [DEBUG] [1661793531.393264592] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8 3) frame. frame_number: 8755 ; frame_TS: 1661793531359.752686 ; ros_TS(NSec): 1661793531359752704
[realsense2_camera_node-1] [DEBUG] [1661793531.393350613] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8755 ; frame_TS: 1661793531359.752686 ; ros_TS(NSec): 1661793531359752704
[realsense2_camera_node-1] [DEBUG] [1661793531.393424569] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793531.398061619] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 3
[realsense2_camera_node-1] [DEBUG] [1661793531.398253246] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8752 ; frame_TS: 1661793531359.752686 ; ros_TS(NSec): 1661793531359752704
[realsense2_camera_node-1] [DEBUG] [1661793531.409840254] [FrequencyStatus_debug_logger]: TICK 5945
[realsense2_camera_node-1] [DEBUG] [1661793531.443292588] [FrequencyStatus_debug_logger]: TICK 5946
[realsense2_camera_node-1] [DEBUG] [1661793531.449920863] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8) frame. frame_number: 8755 ; frame_TS: 1661793531359.752686 ; ros_TS(NSec): 1661793531359752704
[realsense2_camera_node-1] [DEBUG] [1661793531.450174094] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.450246643] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.453366480] [realsense_cam.realsense_cam]: Color stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.453643873] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16) frame. frame_number: 8752 ; frame_TS: 1661793531359.752686 ; ros_TS(NSec): 1661793531359752704
[realsense2_camera_node-1] [DEBUG] [1661793531.453765640] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.453823404] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.455817861] [realsense_cam.realsense_cam]: Depth stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.456126168] [FrequencyStatus_debug_logger]: TICK 6503
[realsense2_camera_node-1] [DEBUG] [1661793531.461992540] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793531.462154309] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 2
[realsense2_camera_node-1] [DEBUG] [1661793531.462248395] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8 3) frame. frame_number: 8756 ; frame_TS: 1661793531393.024170 ; ros_TS(NSec): 1661793531393024256
[realsense2_camera_node-1] [DEBUG] [1661793531.462417013] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8756 ; frame_TS: 1661793531393.024170 ; ros_TS(NSec): 1661793531393024256
[realsense2_camera_node-1] [DEBUG] [1661793531.462491002] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793531.476574641] [FrequencyStatus_debug_logger]: TICK 5947
[realsense2_camera_node-1] [DEBUG] [1661793531.479878009] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 3
[realsense2_camera_node-1] [DEBUG] [1661793531.480060164] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8753 ; frame_TS: 1661793531393.024170 ; ros_TS(NSec): 1661793531393024256
[realsense2_camera_node-1] [DEBUG] [1661793531.510095844] [FrequencyStatus_debug_logger]: TICK 5948
[realsense2_camera_node-1] [DEBUG] [1661793531.534572914] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8) frame. frame_number: 8756 ; frame_TS: 1661793531393.024170 ; ros_TS(NSec): 1661793531393024256
[realsense2_camera_node-1] [DEBUG] [1661793531.534795519] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.534886661] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.537705712] [realsense_cam.realsense_cam]: Color stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.537940318] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16) frame. frame_number: 8753 ; frame_TS: 1661793531393.024170 ; ros_TS(NSec): 1661793531393024256
[realsense2_camera_node-1] [DEBUG] [1661793531.538035844] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.538118217] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.540286828] [realsense_cam.realsense_cam]: Depth stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.540573502] [FrequencyStatus_debug_logger]: TICK 6504
[realsense2_camera_node-1] [DEBUG] [1661793531.543320613] [FrequencyStatus_debug_logger]: TICK 5949
[realsense2_camera_node-1] [DEBUG] [1661793531.544009678] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793531.544397318] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 2
[realsense2_camera_node-1] [DEBUG] [1661793531.544471914] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8 3) frame. frame_number: 8757 ; frame_TS: 1661793531426.296143 ; ros_TS(NSec): 1661793531426296064
[realsense2_camera_node-1] [DEBUG] [1661793531.544561392] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8757 ; frame_TS: 1661793531426.296143 ; ros_TS(NSec): 1661793531426296064
[realsense2_camera_node-1] [DEBUG] [1661793531.544631860] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793531.550036860] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 3
[realsense2_camera_node-1] [DEBUG] [1661793531.550208039] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8754 ; frame_TS: 1661793531426.296143 ; ros_TS(NSec): 1661793531426296064
[realsense2_camera_node-1] [DEBUG] [1661793531.576790100] [FrequencyStatus_debug_logger]: TICK 5950
[realsense2_camera_node-1] [DEBUG] [1661793531.603993160] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8) frame. frame_number: 8757 ; frame_TS: 1661793531426.296143 ; ros_TS(NSec): 1661793531426296064
[realsense2_camera_node-1] [DEBUG] [1661793531.604220149] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.604296378] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.607290448] [realsense_cam.realsense_cam]: Color stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.607554912] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16) frame. frame_number: 8754 ; frame_TS: 1661793531426.296143 ; ros_TS(NSec): 1661793531426296064
[realsense2_camera_node-1] [DEBUG] [1661793531.607694664] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.607756076] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.609915855] [realsense_cam.realsense_cam]: Depth stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.609984275] [FrequencyStatus_debug_logger]: TICK 5951
[realsense2_camera_node-1] [DEBUG] [1661793531.610455536] [FrequencyStatus_debug_logger]: TICK 6505
[realsense2_camera_node-1] [DEBUG] [1661793531.616107367] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793531.616285266] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 2
[realsense2_camera_node-1] [DEBUG] [1661793531.616376983] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8 3) frame. frame_number: 8758 ; frame_TS: 1661793531459.566406 ; ros_TS(NSec): 1661793531459566336
[realsense2_camera_node-1] [DEBUG] [1661793531.616465181] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8758 ; frame_TS: 1661793531459.566406 ; ros_TS(NSec): 1661793531459566336
[realsense2_camera_node-1] [DEBUG] [1661793531.616578372] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793531.622573520] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 3
[realsense2_camera_node-1] [DEBUG] [1661793531.622763515] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8755 ; frame_TS: 1661793531459.566406 ; ros_TS(NSec): 1661793531459566336
[realsense2_camera_node-1] [DEBUG] [1661793531.643326299] [FrequencyStatus_debug_logger]: TICK 5952
[realsense2_camera_node-1] [DEBUG] [1661793531.674932122] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8) frame. frame_number: 8758 ; frame_TS: 1661793531459.566406 ; ros_TS(NSec): 1661793531459566336
[realsense2_camera_node-1] [DEBUG] [1661793531.675179337] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.675331026] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.676982678] [FrequencyStatus_debug_logger]: TICK 5953
[realsense2_camera_node-1] [DEBUG] [1661793531.680709113] [realsense_cam.realsense_cam]: Color stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.681003146] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16) frame. frame_number: 8755 ; frame_TS: 1661793531459.566406 ; ros_TS(NSec): 1661793531459566336
[realsense2_camera_node-1] [DEBUG] [1661793531.681140595] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.681208983] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.683849207] [realsense_cam.realsense_cam]: Depth stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.684263216] [FrequencyStatus_debug_logger]: TICK 6506
[realsense2_camera_node-1] [DEBUG] [1661793531.689181979] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793531.689349765] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 2
[realsense2_camera_node-1] [DEBUG] [1661793531.689416777] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8 3) frame. frame_number: 8759 ; frame_TS: 1661793531492.836914 ; ros_TS(NSec): 1661793531492836864
[realsense2_camera_node-1] [DEBUG] [1661793531.689485037] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8759 ; frame_TS: 1661793531492.836914 ; ros_TS(NSec): 1661793531492836864
[realsense2_camera_node-1] [DEBUG] [1661793531.689549425] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793531.696969268] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 3
[realsense2_camera_node-1] [DEBUG] [1661793531.697335530] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8756 ; frame_TS: 1661793531492.836914 ; ros_TS(NSec): 1661793531492836864
[realsense2_camera_node-1] [DEBUG] [1661793531.710121138] [FrequencyStatus_debug_logger]: TICK 5954
[realsense2_camera_node-1] [DEBUG] [1661793531.743275023] [FrequencyStatus_debug_logger]: TICK 5955
[realsense2_camera_node-1] [DEBUG] [1661793531.750541736] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8) frame. frame_number: 8759 ; frame_TS: 1661793531492.836914 ; ros_TS(NSec): 1661793531492836864
[realsense2_camera_node-1] [DEBUG] [1661793531.750802136] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.750906910] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.754113025] [realsense_cam.realsense_cam]: Color stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.754429140] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16) frame. frame_number: 8756 ; frame_TS: 1661793531492.836914 ; ros_TS(NSec): 1661793531492836864
[realsense2_camera_node-1] [DEBUG] [1661793531.754589149] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.754649729] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.756765090] [realsense_cam.realsense_cam]: Depth stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.758204505] [FrequencyStatus_debug_logger]: TICK 6507
[realsense2_camera_node-1] [DEBUG] [1661793531.764000921] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793531.764199109] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 1
[realsense2_camera_node-1] [DEBUG] [1661793531.764270441] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8757 ; frame_TS: 1661793531526.091797 ; ros_TS(NSec): 1661793531526091776
[realsense2_camera_node-1] [DEBUG] [1661793531.764375280] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793531.769367263] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 2
[realsense2_camera_node-1] [DEBUG] [1661793531.769550506] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8757 ; frame_TS: 1661793531526.091797 ; ros_TS(NSec): 1661793531526091776
[realsense2_camera_node-1] [DEBUG] [1661793531.769764311] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16) frame. frame_number: 8757 ; frame_TS: 1661793531526.091797 ; ros_TS(NSec): 1661793531526091776
[realsense2_camera_node-1] [DEBUG] [1661793531.769852252] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.769906911] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.772351604] [realsense_cam.realsense_cam]: Depth stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.772752364] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793531.772844306] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 2
[realsense2_camera_node-1] [DEBUG] [1661793531.772903893] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8 3) frame. frame_number: 8761 ; frame_TS: 1661793531559.375000 ; ros_TS(NSec): 1661793531559375104
[realsense2_camera_node-1] [DEBUG] [1661793531.772976858] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8761 ; frame_TS: 1661793531559.375000 ; ros_TS(NSec): 1661793531559375104
[realsense2_camera_node-1] [DEBUG] [1661793531.773048478] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793531.777326626] [FrequencyStatus_debug_logger]: TICK 5956
[realsense2_camera_node-1] [DEBUG] [1661793531.780650796] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 3
[realsense2_camera_node-1] [DEBUG] [1661793531.780817430] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8758 ; frame_TS: 1661793531559.375000 ; ros_TS(NSec): 1661793531559375104
[realsense2_camera_node-1] [DEBUG] [1661793531.810080998] [FrequencyStatus_debug_logger]: TICK 5957
[realsense2_camera_node-1] [DEBUG] [1661793531.828247893] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8) frame. frame_number: 8761 ; frame_TS: 1661793531559.375000 ; ros_TS(NSec): 1661793531559375104
[realsense2_camera_node-1] [DEBUG] [1661793531.828606443] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.828702448] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.831939189] [realsense_cam.realsense_cam]: Color stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.832302603] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16) frame. frame_number: 8758 ; frame_TS: 1661793531559.375000 ; ros_TS(NSec): 1661793531559375104
[realsense2_camera_node-1] [DEBUG] [1661793531.832426195] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.832484534] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.834560980] [realsense_cam.realsense_cam]: Depth stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.834835269] [FrequencyStatus_debug_logger]: TICK 6508
[realsense2_camera_node-1] [DEBUG] [1661793531.839021795] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793531.839204174] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 1
[realsense2_camera_node-1] [DEBUG] [1661793531.839279795] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8759 ; frame_TS: 1661793531592.628662 ; ros_TS(NSec): 1661793531592628736
[realsense2_camera_node-1] [DEBUG] [1661793531.839399610] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793531.843386988] [FrequencyStatus_debug_logger]: TICK 5958
[realsense2_camera_node-1] [DEBUG] [1661793531.846532139] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 2
[realsense2_camera_node-1] [DEBUG] [1661793531.846704533] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8759 ; frame_TS: 1661793531592.628662 ; ros_TS(NSec): 1661793531592628736
[realsense2_camera_node-1] [DEBUG] [1661793531.846949988] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16) frame. frame_number: 8759 ; frame_TS: 1661793531592.628662 ; ros_TS(NSec): 1661793531592628736
[realsense2_camera_node-1] [DEBUG] [1661793531.847031689] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.847083084] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.850507612] [realsense_cam.realsense_cam]: Depth stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.851031932] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793531.851142563] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 2
[realsense2_camera_node-1] [DEBUG] [1661793531.851205222] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8 3) frame. frame_number: 8763 ; frame_TS: 1661793531625.910400 ; ros_TS(NSec): 1661793531625910528
[realsense2_camera_node-1] [DEBUG] [1661793531.851270186] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8763 ; frame_TS: 1661793531625.910400 ; ros_TS(NSec): 1661793531625910528
[realsense2_camera_node-1] [DEBUG] [1661793531.851339439] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793531.855985161] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 3
[realsense2_camera_node-1] [DEBUG] [1661793531.856151635] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8760 ; frame_TS: 1661793531625.910400 ; ros_TS(NSec): 1661793531625910528
[realsense2_camera_node-1] [DEBUG] [1661793531.876756566] [FrequencyStatus_debug_logger]: TICK 5959
[realsense2_camera_node-1] [DEBUG] [1661793531.910037978] [FrequencyStatus_debug_logger]: TICK 5960
[realsense2_camera_node-1] [DEBUG] [1661793531.918694631] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8) frame. frame_number: 8763 ; frame_TS: 1661793531625.910400 ; ros_TS(NSec): 1661793531625910528
[realsense2_camera_node-1] [DEBUG] [1661793531.919066174] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.919172740] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.923230491] [realsense_cam.realsense_cam]: Color stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.923565679] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16) frame. frame_number: 8760 ; frame_TS: 1661793531625.910400 ; ros_TS(NSec): 1661793531625910528
[realsense2_camera_node-1] [DEBUG] [1661793531.923705656] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.923779548] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.925892732] [realsense_cam.realsense_cam]: Depth stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.926176174] [FrequencyStatus_debug_logger]: TICK 6509
[realsense2_camera_node-1] [DEBUG] [1661793531.930176672] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793531.930512597] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 1
[realsense2_camera_node-1] [DEBUG] [1661793531.930593466] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8761 ; frame_TS: 1661793531659.162842 ; ros_TS(NSec): 1661793531659162880
[realsense2_camera_node-1] [DEBUG] [1661793531.930669854] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793531.937992475] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 2
[realsense2_camera_node-1] [DEBUG] [1661793531.938165029] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8761 ; frame_TS: 1661793531659.162842 ; ros_TS(NSec): 1661793531659162880
[realsense2_camera_node-1] [DEBUG] [1661793531.938479320] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16) frame. frame_number: 8761 ; frame_TS: 1661793531659.162842 ; ros_TS(NSec): 1661793531659162880
[realsense2_camera_node-1] [DEBUG] [1661793531.938591263] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793531.938645250] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793531.940764931] [realsense_cam.realsense_cam]: Depth stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793531.941261793] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793531.941406794] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 2
[realsense2_camera_node-1] [DEBUG] [1661793531.941472110] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8 3) frame. frame_number: 8765 ; frame_TS: 1661793531692.443359 ; ros_TS(NSec): 1661793531692443392
[realsense2_camera_node-1] [DEBUG] [1661793531.941538482] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8765 ; frame_TS: 1661793531692.443359 ; ros_TS(NSec): 1661793531692443392
[realsense2_camera_node-1] [DEBUG] [1661793531.941607126] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793531.945714288] [FrequencyStatus_debug_logger]: TICK 5961
[realsense2_camera_node-1] [DEBUG] [1661793531.962190424] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 3
[realsense2_camera_node-1] [DEBUG] [1661793531.962468297] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8762 ; frame_TS: 1661793531692.443359 ; ros_TS(NSec): 1661793531692443392
[realsense2_camera_node-1] [DEBUG] [1661793532.011014476] [FrequencyStatus_debug_logger]: TICK 5962
[realsense2_camera_node-1] [DEBUG] [1661793532.017573306] [realsense_cam.realsense_cam]: Frameset contain (Color, 0, RGB8) frame. frame_number: 8765 ; frame_TS: 1661793531692.443359 ; ros_TS(NSec): 1661793531692443392
[realsense2_camera_node-1] [DEBUG] [1661793532.017819177] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793532.017921935] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793532.021353311] [realsense_cam.realsense_cam]: Color stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793532.021601422] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16) frame. frame_number: 8762 ; frame_TS: 1661793531692.443359 ; ros_TS(NSec): 1661793531692443392
[realsense2_camera_node-1] [DEBUG] [1661793532.021714581] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793532.021772953] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793532.023902842] [realsense_cam.realsense_cam]: Depth stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793532.024312275] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793532.024404184] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 1
[realsense2_camera_node-1] [DEBUG] [1661793532.024465500] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8763 ; frame_TS: 1661793531725.694336 ; ros_TS(NSec): 1661793531725694208
[realsense2_camera_node-1] [DEBUG] [1661793532.024535072] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793532.029865252] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 2
[realsense2_camera_node-1] [DEBUG] [1661793532.030088465] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8763 ; frame_TS: 1661793531725.694336 ; ros_TS(NSec): 1661793531725694208
[realsense2_camera_node-1] [DEBUG] [1661793532.030448775] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16) frame. frame_number: 8763 ; frame_TS: 1661793531725.694336 ; ros_TS(NSec): 1661793531725694208
[realsense2_camera_node-1] [DEBUG] [1661793532.030639347] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793532.030709751] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793532.032810391] [realsense_cam.realsense_cam]: Depth stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793532.033190830] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793532.033279539] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 1
[realsense2_camera_node-1] [DEBUG] [1661793532.033370905] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8764 ; frame_TS: 1661793531758.958496 ; ros_TS(NSec): 1661793531758958592
[realsense2_camera_node-1] [DEBUG] [1661793532.033466366] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793532.047225986] [FrequencyStatus_debug_logger]: TICK 5963
[realsense2_camera_node-1] [DEBUG] [1661793532.048988589] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 2
[realsense2_camera_node-1] [DEBUG] [1661793532.049175416] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8764 ; frame_TS: 1661793531758.958496 ; ros_TS(NSec): 1661793531758958592
[realsense2_camera_node-1] [DEBUG] [1661793532.049443592] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16) frame. frame_number: 8764 ; frame_TS: 1661793531758.958496 ; ros_TS(NSec): 1661793531758958592
[realsense2_camera_node-1] [DEBUG] [1661793532.049529838] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793532.049583345] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793532.051818040] [realsense_cam.realsense_cam]: Depth stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793532.052351897] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793532.052496770] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 1
[realsense2_camera_node-1] [DEBUG] [1661793532.052594664] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8765 ; frame_TS: 1661793531792.223145 ; ros_TS(NSec): 1661793531792223232
[realsense2_camera_node-1] [DEBUG] [1661793532.052672268] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793532.058241726] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 2
[realsense2_camera_node-1] [DEBUG] [1661793532.058535728] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8765 ; frame_TS: 1661793531792.223145 ; ros_TS(NSec): 1661793531792223232
[realsense2_camera_node-1] [DEBUG] [1661793532.058842083] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16) frame. frame_number: 8765 ; frame_TS: 1661793531792.223145 ; ros_TS(NSec): 1661793531792223232
[realsense2_camera_node-1] [DEBUG] [1661793532.058961290] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793532.059037775] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793532.061652525] [realsense_cam.realsense_cam]: Depth stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793532.062123018] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793532.062222288] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 1
[realsense2_camera_node-1] [DEBUG] [1661793532.062391130] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8766 ; frame_TS: 1661793531825.485840 ; ros_TS(NSec): 1661793531825485824
[realsense2_camera_node-1] [DEBUG] [1661793532.062482464] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793532.076754850] [FrequencyStatus_debug_logger]: TICK 5964
[realsense2_camera_node-1] [DEBUG] [1661793532.079330751] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 2
[realsense2_camera_node-1] [DEBUG] [1661793532.079498857] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8766 ; frame_TS: 1661793531825.485840 ; ros_TS(NSec): 1661793531825485824
[realsense2_camera_node-1] [DEBUG] [1661793532.079705781] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16) frame. frame_number: 8766 ; frame_TS: 1661793531825.485840 ; ros_TS(NSec): 1661793531825485824
[realsense2_camera_node-1] [DEBUG] [1661793532.079790202] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793532.079838141] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793532.081999777] [realsense_cam.realsense_cam]: Depth stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793532.082755822] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793532.082888535] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 1
[realsense2_camera_node-1] [DEBUG] [1661793532.082961339] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8767 ; frame_TS: 1661793531858.749268 ; ros_TS(NSec): 1661793531858749184
[realsense2_camera_node-1] [DEBUG] [1661793532.083038048] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793532.091445534] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 2
[realsense2_camera_node-1] [DEBUG] [1661793532.091630185] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8767 ; frame_TS: 1661793531858.749268 ; ros_TS(NSec): 1661793531858749184
[realsense2_camera_node-1] [WARN] [1661793532.091838070] [realsense_cam.realsense_cam]: No stream match for pointcloud chosen texture Process - Color
[realsense2_camera_node-1] [DEBUG] [1661793532.091913274] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16) frame. frame_number: 8767 ; frame_TS: 1661793531858.749268 ; ros_TS(NSec): 1661793531858749184
[realsense2_camera_node-1] [DEBUG] [1661793532.092010208] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793532.092062627] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793532.094226791] [realsense_cam.realsense_cam]: Depth stream published, message address: 0xffff41209ea0
[realsense2_camera_node-1] [DEBUG] [1661793532.094640448] [realsense_cam.realsense_cam]: Frameset arrived.
[realsense2_camera_node-1] [DEBUG] [1661793532.094728837] [realsense_cam.realsense_cam]: List of frameset before applying filters: size: 1
[realsense2_camera_node-1] [DEBUG] [1661793532.094792457] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8768 ; frame_TS: 1661793531892.010742 ; ros_TS(NSec): 1661793531892010752
[realsense2_camera_node-1] [DEBUG] [1661793532.094866862] [realsense_cam.realsense_cam]: num_filters: 11
[realsense2_camera_node-1] [DEBUG] [1661793532.106918505] [realsense_cam.realsense_cam]: List of frameset after applying filters: size: 2
[realsense2_camera_node-1] [DEBUG] [1661793532.107124022] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8768 ; frame_TS: 1661793531892.010742 ; ros_TS(NSec): 1661793531892010752
[realsense2_camera_node-1] [DEBUG] [1661793532.107353732] [realsense_cam.realsense_cam]: Frameset contain (Depth, 0, Z16) frame. frame_number: 8768 ; frame_TS: 1661793531892.010742 ; ros_TS(NSec): 1661793531892010752
[realsense2_camera_node-1] [DEBUG] [1661793532.107474827] [realsense_cam.realsense_cam]: frame: 1280 x 720
[realsense2_camera_node-1] [DEBUG] [1661793532.107585394] [realsense_cam.realsense_cam]: publishFrame(...)
[realsense2_camera_node-1] [DEBUG] [1661793532.109725844] [realsense_cam.realsense_cam]: Depth stream published, message address: 0xffff41209ea0
The following snippets are cleaned for brievety.
Scenario 1 is the best scenario would be, with depth and color "arriving at the same time":
Frameset arrived.
List of frameset before applying filters: size: 2
Frameset contain (Color, 0, RGB8 3) frame. frame_number: 8756
Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8756
List of frameset after applying filters: size: 3
Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8753
Frameset contain (Color, 0, RGB8) frame. frame_number: 8756
frame: 1280 x 720
publishFrame(...)
Color stream published
Frameset contain (Depth, 0, Z16) frame. frame_number: 8753
frame: 1280 x 720
publishFrame(...)
Depth stream published
But two other scenarios can happen, and are not desirable.
Scenario 2 has only depth before filters:
Frameset arrived.
List of frameset before applying filters: size: 1
Frameset contain (Depth, 0, Z16 0) frame. frame_number: 8751
num_filters: 11
List of frameset after applying filters: size: 2
Frameset contain (Depth, 0, XYZ32F) frame. frame_number: 8751
Frameset contain (Depth, 0, Z16) frame. frame_number: 8751
frame: 1280 x 720
publishFrame(...)
Depth stream published
Scenario 3 has only color before filters:
Frameset arrived.
List of frameset before applying filters: size: 1
Frameset contain (Color, 0, RGB8 3) frame. frame_number: 8742
num_filters: 11
List of frameset after applying filters: size: 1
Frameset contain (Color, 0, RGB8) frame. frame_number: 8742
frame: 1280 x 720
publishFrame(...)
Color stream published
Perfect execution should always be scenario 1, always having pointcloud data with color texture available as an output. The frames seem to not be in sync, and it can be seen that the frame number of the filtered frames are going back and forth..
Would you have any idea on how to avoid that?
Thank you
@iraadit You should not need to set enable_sync to true in your launch instruction as the ROS wrapper documentation for this parameter states that it happens automatically when filters such as pointcloud are enabled. Likewise, the decimation filter is false by default in the rs_launch.py file and so should not need setting to false in the launch instruction.
How does it perform if you remove depth_module.exposure:=32000 depth_module.enable_auto_exposure:=false and use auto-exposure?
Hi @MartyG-RealSense,
_enblesync as well as _decimationfilter.enable are in the launch command because it is easier for me that way to try out different parameters (I don't have to remember their name when I want to try them out, but just to change true to false (and vice-versa)).
By the way, I think that pointcloud.enable:=true must also turn _aligndepth.enable to true, what is happening then if I add _aligndepth.enable:=false in the launch command? Is it ignored?
As explained in the point 1 of https://github.com/IntelRealSense/realsense-ros/issues/2396#issuecomment-1219842074, I have to add depth_module.exposure:=32000 depth_module.enable_auto_exposure:=false to be able to get 30 FPS in input for depth, if not it saturates at 15FPS. By default, depth_module.enable_auto_exposure is true, but doesn't seem to work. (???) If I put depth_module.enable_auto_exposure to false, it will use the default value of depth_module.exposure that is 33000 (33ms) and seems to be too much to get 30FPS. If I change the value of depth_module.exposure to be lower (here 32000), it can runt 30FPS. These FPS values are seen thanks to the /diagnostics topic.
I would assume that a 'false' state of align_depth will be overriden to 'true' if the pointcloud filter is enabled.
Your colleague @Doch88 reported earlier in the discussion that enabling auto_exposure and disabling auto_exposure_priority instead of using manual exposure in order to enforce a constant FPS) did not seem to work.
https://github.com/IntelRealSense/realsense-ros/issues/2396#issuecomment-1175140329 states that you cannot reduce your resolution from 1280x720 to 848x480 to improve performance.
I believe we have covered almost all of the available possibilities for improving performance during this 2 month discussion, unfortunately.
Hi @MartyG-RealSense , I believe there is still more into this, by modifying code if needed, I'll continue to look into it :)
Thanks so much @iraadit :) Please do feedback any findings that you make.
Hi @iraadit Do you have an update about this case that you can provide, please? Thanks!
I am trying to use the Intel Realsense camera on the Jetson Xavier AGX. When I only run the RGB camera & depth image it seems to run quite fast (30fps). However, when I enable the pointcloud it becomes very slow.
I tried to run it at 15 fps for RGB & Depth at various resolutions but the result is the same. The RGB camera freezes and no point cloud is displayed in RVIZ. When I block part of the depth camera (so that it doesn't have to create as many points in the point cloud) it seems to work again. The creation of the pointcloud in the realsense node seems to be a bottleneck, I am running the Jetson on max performance.
Some of the errors and warnings that appear are the following:
"incomplete frame received: Incomplete video frame detected! Size 99080 out of 2048255 bytes (4%):" "Hardware Notification:Depth stream start failure,1.65668e+12,Error,Hardware Error" "Out of frame resources"
I wonder if it is expected behaviour that you can only run it with maybe 5 FPS on the Jetson on max settings.