(synthetic-stream.cpp:48) Exception was thrown during user processing callback: Out of frame resources!

gitsr-sys commented 3 years ago

Do you have any idea about this error?

(synthetic-stream.cpp:48) Exception was thrown during user processing callback: Out of frame resources!

What is this and what cause this problem?

I tried to use a usb hub with a power cable for avoiding power supply problem, and I changed the following parameters;

    <!-- d435 -->
    <arg name="color_width"               default="640"/>
    <arg name="color_height"              default="480"/>
    <arg name="depth_width"               default="640"/>
    <arg name="depth_height"              default="480"/>
    <arg name="clip_distance"             default="-2"/>
    <arg name="infra_fps"                 default="45"/>

RealSenseSupport commented 3 years ago

@srnetsr Which camera you are working with?

gitsr-sys commented 3 years ago

@RealSenseCustomerSupport , sorry for the late response. As you can see above,

<!-- d435 -->

gitsr-sys commented 3 years ago

Hi, @MartyG-RealSense I saw your reply in #1161 You mentioned this error message and reducing frame rate and resolution as a solution. FYI, I am not using NVIDIA machienes.

Please refer to the above parameter values and advise me to resolve this issue?

MartyG-RealSense commented 3 years ago

Hi @srnetsr Looking at your arg definitons in the opening comment, I note that you have set infra_fps to a value of 45. 45 FPS is not supported as a default option on RealSense cameras. On a D435 camera on a USB 3 connection, the supported FPS settings for infrared at 640x480 resolution are:

6, 15, 30, 60, 90

Could you therefore try setting infra_fps to '30' please?

MartyG-RealSense commented 3 years ago

Hi @srnetsr Do you require further assistance with this case, please? Thanks!

MartyG-RealSense commented 3 years ago

Case closed due to no further comments received.

AndreV84 commented 3 years ago

we can not get ros2 color topic passing anything with the error like that it streams the depth, however


(synthetic-stream.cpp:48) Exception was thrown during user processing callback: 1kernel_oth

MartyG-RealSense commented 3 years ago

Hi @AndreV84 If you are using ROS2, have you checked the list of topics being published with ros2 topic list to see whether the color topics are on the list?

AndreV84 commented 3 years ago

yes, the color topic was published, but empty

MartyG-RealSense commented 3 years ago

@AndreV84 Can you confirm whether you are using the current RealSense ROS2 wrapper please and not the old obsolete ROS2 wrapper in the link below that is no longer supported.

https://github.com/intel/ros2_intel_realsense

AndreV84 commented 3 years ago

@MartyG-RealSense yes, but we run from docker, also on arm64

MartyG-RealSense commented 3 years ago

@AndreV84 I researched your problem carefully but could not find a solution. Is the problem limited to just color or do other types of stream such as depth have empty topics too?

AndreV84 commented 3 years ago

it goit addressed by adding more swap memory as it turned out it was due to cuda memory allocation error it seems without extra swap it was posible to get only part of topics published

MartyG-RealSense commented 3 years ago

Okay, great news that you were able to find a solution! Thanks for the update @AndreV84

gitsr-sys commented 3 years ago

@MartyG-RealSense I am sorry for my late response. I am using 90 fps for infra_fps, and tried small numbers as well. However, the error keep displaying.. I am just ignoring this message, but I do not know why it is coming up.

MartyG-RealSense commented 3 years ago

Hi @srnetsr Could you test whether resetting the camera at launch makes a difference please by adding the command below to the end of your roslaunch instruction:

initial_reset:=true

mzahana commented 3 years ago

@AndreV84 & @MartyG-RealSense I am getting the same error (or similar)

(synthetic-stream.cpp:48) Exception was thrown during user processing callback:

setup

Camera: D435i with firmware 5.12.7 (I also tried the latest D435i firmware 5.12.13.5 as well as 5.12.11
Using Xavier NX with l4t-r32.5.0
Using a docker container with Nvidia base image l4t-r32.5.0 (inspired by jetson-containers )
Compiled libearlsense v2.45.0 (with -DFORCE_RSUSB_BACKEND=ON -DBUILD_WITH_CUDA=true) and realsense-ros 2.3.0 from source
I run the default rs_camera.launch

I get a continuous stream of the above error. The RGB ROS topic /camera/color/image_raw is there but empty, but the depth image topic is there and has messages at 30Hz.

When I use realsense_viewer, it seems RGB and depth images are being streamed, but the motion module is not recognized.

I am not sure what the problem is with the ROS side. Any help is appreciated

Below is the output of roslaunch realsense2_camera rs_camera.launch

started roslaunch server http://riot-desktop:45809/

SUMMARY
========

PARAMETERS
 * /camera/realsense2_camera/accel_fps: -1
 * /camera/realsense2_camera/accel_frame_id: camera_accel_frame
 * /camera/realsense2_camera/accel_optical_frame_id: camera_accel_opti...
 * /camera/realsense2_camera/align_depth: False
 * /camera/realsense2_camera/aligned_depth_to_color_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_fisheye1_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_fisheye2_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_fisheye_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_infra1_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_infra2_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/allow_no_texture_points: False
 * /camera/realsense2_camera/base_frame_id: camera_link
 * /camera/realsense2_camera/calib_odom_file: 
 * /camera/realsense2_camera/clip_distance: -2.0
 * /camera/realsense2_camera/color_fps: -1
 * /camera/realsense2_camera/color_frame_id: camera_color_frame
 * /camera/realsense2_camera/color_height: -1
 * /camera/realsense2_camera/color_optical_frame_id: camera_color_opti...
 * /camera/realsense2_camera/color_width: -1
 * /camera/realsense2_camera/confidence_fps: -1
 * /camera/realsense2_camera/confidence_height: -1
 * /camera/realsense2_camera/confidence_width: -1
 * /camera/realsense2_camera/depth_fps: -1
 * /camera/realsense2_camera/depth_frame_id: camera_depth_frame
 * /camera/realsense2_camera/depth_height: -1
 * /camera/realsense2_camera/depth_optical_frame_id: camera_depth_opti...
 * /camera/realsense2_camera/depth_width: -1
 * /camera/realsense2_camera/device_type: 
 * /camera/realsense2_camera/enable_accel: False
 * /camera/realsense2_camera/enable_color: True
 * /camera/realsense2_camera/enable_confidence: True
 * /camera/realsense2_camera/enable_depth: True
 * /camera/realsense2_camera/enable_fisheye1: False
 * /camera/realsense2_camera/enable_fisheye2: False
 * /camera/realsense2_camera/enable_fisheye: False
 * /camera/realsense2_camera/enable_gyro: False
 * /camera/realsense2_camera/enable_infra1: False
 * /camera/realsense2_camera/enable_infra2: False
 * /camera/realsense2_camera/enable_infra: False
 * /camera/realsense2_camera/enable_pointcloud: False
 * /camera/realsense2_camera/enable_pose: False
 * /camera/realsense2_camera/enable_sync: False
 * /camera/realsense2_camera/filters: 
 * /camera/realsense2_camera/fisheye1_frame_id: camera_fisheye1_f...
 * /camera/realsense2_camera/fisheye1_optical_frame_id: camera_fisheye1_o...
 * /camera/realsense2_camera/fisheye2_frame_id: camera_fisheye2_f...
 * /camera/realsense2_camera/fisheye2_optical_frame_id: camera_fisheye2_o...
 * /camera/realsense2_camera/fisheye_fps: -1
 * /camera/realsense2_camera/fisheye_frame_id: camera_fisheye_frame
 * /camera/realsense2_camera/fisheye_height: -1
 * /camera/realsense2_camera/fisheye_optical_frame_id: camera_fisheye_op...
 * /camera/realsense2_camera/fisheye_width: -1
 * /camera/realsense2_camera/gyro_fps: -1
 * /camera/realsense2_camera/gyro_frame_id: camera_gyro_frame
 * /camera/realsense2_camera/gyro_optical_frame_id: camera_gyro_optic...
 * /camera/realsense2_camera/imu_optical_frame_id: camera_imu_optica...
 * /camera/realsense2_camera/infra1_frame_id: camera_infra1_frame
 * /camera/realsense2_camera/infra1_optical_frame_id: camera_infra1_opt...
 * /camera/realsense2_camera/infra2_frame_id: camera_infra2_frame
 * /camera/realsense2_camera/infra2_optical_frame_id: camera_infra2_opt...
 * /camera/realsense2_camera/infra_fps: 30
 * /camera/realsense2_camera/infra_height: 480
 * /camera/realsense2_camera/infra_rgb: False
 * /camera/realsense2_camera/infra_width: 848
 * /camera/realsense2_camera/initial_reset: False
 * /camera/realsense2_camera/json_file_path: 
 * /camera/realsense2_camera/linear_accel_cov: 0.01
 * /camera/realsense2_camera/odom_frame_id: camera_odom_frame
 * /camera/realsense2_camera/ordered_pc: False
 * /camera/realsense2_camera/pointcloud_texture_index: 0
 * /camera/realsense2_camera/pointcloud_texture_stream: RS2_STREAM_COLOR
 * /camera/realsense2_camera/pose_frame_id: camera_pose_frame
 * /camera/realsense2_camera/pose_optical_frame_id: camera_pose_optic...
 * /camera/realsense2_camera/publish_odom_tf: True
 * /camera/realsense2_camera/publish_tf: True
 * /camera/realsense2_camera/rosbag_filename: 
 * /camera/realsense2_camera/serial_no: 
 * /camera/realsense2_camera/stereo_module/exposure/1: 7500
 * /camera/realsense2_camera/stereo_module/exposure/2: 1
 * /camera/realsense2_camera/stereo_module/gain/1: 16
 * /camera/realsense2_camera/stereo_module/gain/2: 16
 * /camera/realsense2_camera/tf_publish_rate: 0.0
 * /camera/realsense2_camera/topic_odom_in: odom_in
 * /camera/realsense2_camera/unite_imu_method: 
 * /camera/realsense2_camera/usb_port_id: 
 * /rosdistro: melodic
 * /rosversion: 1.14.11

NODES
  /camera/
    realsense2_camera (nodelet/nodelet)
    realsense2_camera_manager (nodelet/nodelet)

auto-starting new master
process[master]: started with pid [281]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to 0ebfcf82-cc5d-11eb-af22-d8c0a6b126c1
process[rosout-1]: started with pid [294]
started core service [/rosout]
process[camera/realsense2_camera_manager-2]: started with pid [311]
[ INFO] [1623598566.298330232]: Initializing nodelet with 6 worker threads.
process[camera/realsense2_camera-3]: started with pid [312]
[ INFO] [1623598567.179209428]: RealSense ROS v2.3.0
[ INFO] [1623598567.179621336]: Built with LibRealSense v2.45.0
[ INFO] [1623598567.179742905]: Running with LibRealSense v2.45.0
[ INFO] [1623598567.250975590]:  
[ INFO] [1623598567.395925369]: Device with serial number 048122071215 was found.

[ INFO] [1623598567.396076507]: Device with physical ID 2-3.4-10 was found.
[ INFO] [1623598567.396393598]: Device with name Intel RealSense D435I was found.
[ INFO] [1623598567.398672372]: Device with port number 2-3.4 was found.
[ INFO] [1623598567.398835605]: Device USB type: 3.2
[ INFO] [1623598567.406840418]: getParameters...
[ INFO] [1623598567.764438387]: setupDevice...
[ INFO] [1623598567.764590548]: JSON file is not provided
[ INFO] [1623598567.764677717]: ROS Node Namespace: camera
[ INFO] [1623598567.764823735]: Device Name: Intel RealSense D435I
[ INFO] [1623598567.764981752]: Device Serial No: 048122071215
[ INFO] [1623598567.765065593]: Device physical port: 2-3.4-10
[ INFO] [1623598567.765152762]: Device FW version: 05.12.07.100
[ INFO] [1623598567.765230171]: Device Product ID: 0x0B3A
[ INFO] [1623598567.765353596]: Enable PointCloud: Off
[ INFO] [1623598567.765430653]: Align Depth: Off
[ INFO] [1623598567.765546270]: Sync Mode: Off
[ INFO] [1623598567.765771168]: Device Sensors: 
[ INFO] [1623598567.799993865]: Stereo Module was found.
[ INFO] [1623598567.836576137]: RGB Camera was found.
[ INFO] [1623598567.838506492]: Motion Module was found.
[ INFO] [1623598567.838787262]: (Confidence, 0) sensor isn't supported by current device! -- Skipping...
[ INFO] [1623598567.839208802]: num_filters: 0
[ INFO] [1623598567.839535750]: Setting Dynamic reconfig parameters.
 13/06 15:36:08,196 WARNING [547314573696] (messenger-libusb.cpp:42) control_transfer returned error, index: 300, error: No data available, number: 3d
 13/06 15:36:08,270 WARNING [547314573696] (messenger-libusb.cpp:42) control_transfer returned error, index: 300, error: No data available, number: 3d
[ INFO] [1623598568.552403696]: Done Setting Dynamic reconfig parameters.
[ INFO] [1623598568.554752871]: depth stream is enabled - width: 848, height: 480, fps: 30, Format: Z16
[ INFO] [1623598568.559463860]: color stream is enabled - width: 1280, height: 720, fps: 30, Format: RGB8
[ INFO] [1623598568.559720663]: setupPublishers...
[ INFO] [1623598568.571149157]: Expected frequency for depth = 30.00000
[ INFO] [1623598568.583971904]: Expected frequency for color = 30.00000
[ INFO] [1623598568.594969834]: setupStreams...
[ INFO] [1623598568.607990727]: insert Depth to Stereo Module
[ INFO] [1623598568.608201929]: insert Color to RGB Camera
[ INFO] [1623598568.710608131]: SELECTED BASE:Depth, 0
[ INFO] [1623598568.762268724]: RealSense Node Is Up!
[ WARN] [1623598568.841414125]: 
 13/06 15:36:08,843 WARNING [547314573696] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 13/06 15:36:08,893 WARNING [547314573696] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 13/06 15:36:09,045 WARNING [547314573696] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 13/06 15:36:09,117 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 13/06 15:36:09,130 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 13/06 15:36:09,161 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 13/06 15:36:09,194 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 13/06 15:36:09,197 WARNING [547314573696] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 13/06 15:36:09,228 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 13/06 15:36:09,247 WARNING [547314573696] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 13/06 15:36:09,261 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 13/06 15:36:09,294 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 13/06 15:36:09,328 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 13/06 15:36:09,361 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 13/06 15:36:09,395 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 13/06 15:36:09,399 WARNING [547314573696] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 13/06 15:36:09,428 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 13/06 15:36:09,449 WARNING [547314573696] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 13/06 15:36:09,461 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 13/06 15:36:09,494 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 13/06 15:36:09,528 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 13/06 15:36:09,561 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 13/06 15:36:09,594 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 13/06 15:36:09,628 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
^C 13/06 15:36:09,661 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
[camera/realsense2_camera-3] killing on exit
[camera/realsense2_camera_manager-2] killing on exit
 13/06 15:36:09,722 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 13/06 15:36:09,781 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 13/06 15:36:09,802 WARNING [547314573696] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
^C 13/06 15:36:09,842 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 13/06 15:36:09,903 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 13/06 15:36:09,962 ERROR [546770452864] (synthetic-stream.cpp:48) Exception was thrown during user processing callback:

MartyG-RealSense commented 3 years ago

Hi @mzahana

@AndreV84 has a greater knowledge of Docker than I do.

However, please try adding initial_reset:=true to your roslaunch instruction to see whether it makes a positive difference. For example:

roslaunch realsense2_camera rs_camera.launch initial_reset:=true

mzahana commented 3 years ago

@MartyG-RealSense thanks for your reply. Unfortunately , init_reset has no effect in my case.

AndreV84 commented 3 years ago

@mzahana what exactly is the command line you are executing the docker image with? what are the exact steps to reproduce the errors? you may try executing with --privileged , also you may mount video devices to the container somehow to see if it works do the published topics show any content? empty?

mzahana commented 3 years ago

Here is the docker run command

docker run --runtime nvidia -it --network host -e DISPLAY=$DISPLAY --restart always \    --user=${CONTAINER_USER_NAME} \  
  -v /tmp/.X11-unix/:/tmp/.X11-unix \   
 -v /dev:/dev \   
 --group-add=dialout \   
 --group-add=tty \  
  --tty=true \   
 --device=/dev/ttyUSB0 \   
 --device=/dev/ttyTHS0 \  
 -v ${WORKSPACE_DIR}:/home/${CONTAINER_USER_NAME}/shared_volume \   
 --workdir="/home/${CONTAINER_USER_NAME}" \    
--name=${CONTAINER_NAME} \  
  --privileged \   
 ${DOCKER_REPO} \   
 bash -c "${CMD}"

AndreV84 commented 3 years ago

which ROS distribution are you using? ROS1? Melodic? Noetic? OS which version is used? 18.04 ubuntu 20.04? within the container? outside of the container? You may also try ROS2 Regarding the execution of jetson command you may like to add to /etc/docker/daemon.json the content as follows

 to your /etc/docker/daemon.json configuration file before attempting to build the containers:

{
    "runtimes": {
        "nvidia": {
            "path": "nvidia-container-runtime",
            "runtimeArgs": []
        }
    },

    "default-runtime": "nvidia"
}

normally on Nx realsense d435i works, but sometimes powered usb hub is handy, also it might conflict with other usb devices, also the latest firmware is better to use to avoid bugs etc. Generally dockerization works, but it is tricky to implement; If it doesn't work for you with ROS1 try switching to ROS2 I reccolect the issues might be presented in ROS1 that would prevent from normal working, but ROS2 wouldn't have these issues that would emerge from container environment

AndreV84 commented 3 years ago

latest firmware is 5.12.14_50 as I recollect e.g. it could be installed from 2.47 sdk

AndreV84 commented 3 years ago

@mzahana what is the usb speed value returned by rs-fw-update -l is it 2.0 usb speed or 3.2 usb speed? we have had same issue with depth steram showing but color rgb not showing, but resolved it eventually for docker implemetnation I recollect we solved major issue by adding swap/ turning OFF zram


Turn off zram:

cd /etc/systemd
sudo mv nvzramconfig.sh nvzramconfig.sh.orig
sudo reboot
 Add swap file on nvme and verify:

sudo fallocate -l 32G /home/nvidia/nvme/swapfile
sudo chmod 600 /home/nvidia/nvme/swapfile
sudo mkswap /home/nvidia/nvme/swapfile
sudo swapon /home/nvidia/nvme/swapfile
sudo swapon -s
Add the line below to /etc/fstab so swap file will be automounted going forward:

/home/nvidia/nvme/swapfile swap swap defaults 0 0

mzahana commented 3 years ago

@AndreV84

The /etc/docker/daemon.json is the same as the one you mentioned

here is the output of rs-fw-update -l

connected devices:
1) Name: Intel RealSense D435I, serial number: 048122071215, update serial number: 046523051844, firmware version: 05.12.07.100, USB type: 3.2

I have a swap of 4Gb, on an SD card. I don't have an SSD

AndreV84 commented 3 years ago

@mzahana swap 4gb might not be sufficient for dockerized ros node with realsense sdk also you might need to install systemwide realsense sdk which you probably have done try assigning at least 8-12 gb of swap - it could fit on sdcard

AndreV84 commented 3 years ago

@mzahana is the rs-fw-update -l output from docker or from systemwide in the message above?

AndreV84 commented 3 years ago

does the ros node work from systemwide install on the same device?

mzahana commented 3 years ago

@AndreV84 the rs-fw-update -l is run system wide. I have not tried to use ROS system wide on Xavier NX I already have everything else I need running inside a container, except realsense ros

AndreV84 commented 3 years ago

@mzahana what is the output of rs-fw-update -l from the container?

mzahana commented 3 years ago

Now, I am trying to install the same librealsense system wide from source (same version as in the container); it was installed from apt-get. I am not sure if that is an issue.

AndreV84 commented 3 years ago

@mzahana typically we would try both from source but also from apt;for the container use we used from source as it would allow to reduce amount of issues; Could you share the output of

rs-fw-update -l

if executing it from inside the container?

AndreV84 commented 3 years ago

have you had a chance to try ros2 container? with realsense node? it could be 18.04 or upgraded to 20.04 container

mzahana commented 3 years ago

@AndreV84 Sorry for the late reply. The Xavier NX stopped working and I had to re-flash it. This time I am using Jetpack 4.4 instead of the latest. I read somewhere that things worked better with Jetpack 4.4

The output of rs-fw-update -l inside the container gives

connected devices:
1) Name: Intel RealSense D435I, serial number: 048122071215, update serial number: 046523051844, firmware version: 05.12.07.100, USB type: 3.2

The output of rs-fw-update -l outside the container gives

connected devices:
1) Name: Intel RealSense D435I, serial number: 048122071215, update serial number: 046523051844, firmware version: 05.12.07.100, USB type: 3.2

I don't use ROS 2 and I don' have it.

I installed the same librealsense version (2.45.0) outside/insde the container, and I still get the same error

UMMARY
========

PARAMETERS
 * /camera/realsense2_camera/accel_fps: -1
 * /camera/realsense2_camera/accel_frame_id: camera_accel_frame
 * /camera/realsense2_camera/accel_optical_frame_id: camera_accel_opti...
 * /camera/realsense2_camera/align_depth: False
 * /camera/realsense2_camera/aligned_depth_to_color_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_fisheye1_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_fisheye2_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_fisheye_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_infra1_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_infra2_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/allow_no_texture_points: False
 * /camera/realsense2_camera/base_frame_id: camera_link
 * /camera/realsense2_camera/calib_odom_file: 
 * /camera/realsense2_camera/clip_distance: -2.0
 * /camera/realsense2_camera/color_fps: -1
 * /camera/realsense2_camera/color_frame_id: camera_color_frame
 * /camera/realsense2_camera/color_height: -1
 * /camera/realsense2_camera/color_optical_frame_id: camera_color_opti...
 * /camera/realsense2_camera/color_width: -1
 * /camera/realsense2_camera/confidence_fps: -1
 * /camera/realsense2_camera/confidence_height: -1
 * /camera/realsense2_camera/confidence_width: -1
 * /camera/realsense2_camera/depth_fps: -1
 * /camera/realsense2_camera/depth_frame_id: camera_depth_frame
 * /camera/realsense2_camera/depth_height: -1
 * /camera/realsense2_camera/depth_optical_frame_id: camera_depth_opti...
 * /camera/realsense2_camera/depth_width: -1
 * /camera/realsense2_camera/device_type: 
 * /camera/realsense2_camera/enable_accel: False
 * /camera/realsense2_camera/enable_color: True
 * /camera/realsense2_camera/enable_confidence: True
 * /camera/realsense2_camera/enable_depth: True
 * /camera/realsense2_camera/enable_fisheye1: False
 * /camera/realsense2_camera/enable_fisheye2: False
 * /camera/realsense2_camera/enable_fisheye: False
 * /camera/realsense2_camera/enable_gyro: False
 * /camera/realsense2_camera/enable_infra1: False
 * /camera/realsense2_camera/enable_infra2: False
 * /camera/realsense2_camera/enable_infra: False
 * /camera/realsense2_camera/enable_pointcloud: False
 * /camera/realsense2_camera/enable_pose: False
 * /camera/realsense2_camera/enable_sync: False
 * /camera/realsense2_camera/filters: 
 * /camera/realsense2_camera/fisheye1_frame_id: camera_fisheye1_f...
 * /camera/realsense2_camera/fisheye1_optical_frame_id: camera_fisheye1_o...
 * /camera/realsense2_camera/fisheye2_frame_id: camera_fisheye2_f...
 * /camera/realsense2_camera/fisheye2_optical_frame_id: camera_fisheye2_o...
 * /camera/realsense2_camera/fisheye_fps: -1
 * /camera/realsense2_camera/fisheye_frame_id: camera_fisheye_frame
 * /camera/realsense2_camera/fisheye_height: -1
 * /camera/realsense2_camera/fisheye_optical_frame_id: camera_fisheye_op...
 * /camera/realsense2_camera/fisheye_width: -1
 * /camera/realsense2_camera/gyro_fps: -1
 * /camera/realsense2_camera/gyro_frame_id: camera_gyro_frame
 * /camera/realsense2_camera/gyro_optical_frame_id: camera_gyro_optic...
 * /camera/realsense2_camera/imu_optical_frame_id: camera_imu_optica...
 * /camera/realsense2_camera/infra1_frame_id: camera_infra1_frame
 * /camera/realsense2_camera/infra1_optical_frame_id: camera_infra1_opt...
 * /camera/realsense2_camera/infra2_frame_id: camera_infra2_frame
 * /camera/realsense2_camera/infra2_optical_frame_id: camera_infra2_opt...
 * /camera/realsense2_camera/infra_fps: 30
 * /camera/realsense2_camera/infra_height: 480
 * /camera/realsense2_camera/infra_rgb: False
 * /camera/realsense2_camera/infra_width: 848
 * /camera/realsense2_camera/initial_reset: False
 * /camera/realsense2_camera/json_file_path: 
 * /camera/realsense2_camera/linear_accel_cov: 0.01
 * /camera/realsense2_camera/odom_frame_id: camera_odom_frame
 * /camera/realsense2_camera/ordered_pc: False
 * /camera/realsense2_camera/pointcloud_texture_index: 0
 * /camera/realsense2_camera/pointcloud_texture_stream: RS2_STREAM_COLOR
 * /camera/realsense2_camera/pose_frame_id: camera_pose_frame
 * /camera/realsense2_camera/pose_optical_frame_id: camera_pose_optic...
 * /camera/realsense2_camera/publish_odom_tf: True
 * /camera/realsense2_camera/publish_tf: True
 * /camera/realsense2_camera/rosbag_filename: 
 * /camera/realsense2_camera/serial_no: 
 * /camera/realsense2_camera/stereo_module/exposure/1: 7500
 * /camera/realsense2_camera/stereo_module/exposure/2: 1
 * /camera/realsense2_camera/stereo_module/gain/1: 16
 * /camera/realsense2_camera/stereo_module/gain/2: 16
 * /camera/realsense2_camera/tf_publish_rate: 0.0
 * /camera/realsense2_camera/topic_odom_in: odom_in
 * /camera/realsense2_camera/unite_imu_method: 
 * /camera/realsense2_camera/usb_port_id: 
 * /rosdistro: melodic
 * /rosversion: 1.14.11

NODES
  /camera/
    realsense2_camera (nodelet/nodelet)
    realsense2_camera_manager (nodelet/nodelet)

auto-starting new master
process[master]: started with pid [391]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to b9d44daa-d1e7-11eb-ae13-204ef6c606d3
process[rosout-1]: started with pid [404]
started core service [/rosout]
process[camera/realsense2_camera_manager-2]: started with pid [421]
[ INFO] [1624207879.843564440]: Initializing nodelet with 6 worker threads.
process[camera/realsense2_camera-3]: started with pid [422]
[ INFO] [1624207880.790743308]: RealSense ROS v2.3.0
[ INFO] [1624207880.790917004]: Built with LibRealSense v2.45.0
[ INFO] [1624207880.791032556]: Running with LibRealSense v2.45.0
[ INFO] [1624207880.863347985]:  
[ INFO] [1624207881.162811939]: Device with serial number 048122071215 was found.

[ INFO] [1624207881.163363332]: Device with physical ID 2-3.4-5 was found.
[ INFO] [1624207881.163498917]: Device with name Intel RealSense D435I was found.
[ INFO] [1624207881.165752043]: Device with port number 2-3.4 was found.
[ INFO] [1624207881.165869707]: Device USB type: 3.2
[ INFO] [1624207881.174286498]: getParameters...
[ INFO] [1624207881.556073846]: setupDevice...
[ INFO] [1624207881.556284438]: JSON file is not provided
[ INFO] [1624207881.556380918]: ROS Node Namespace: camera
[ INFO] [1624207881.556466327]: Device Name: Intel RealSense D435I
[ INFO] [1624207881.556541847]: Device Serial No: 048122071215
[ INFO] [1624207881.556650423]: Device physical port: 2-3.4-5
[ INFO] [1624207881.557081368]: Device FW version: 05.12.07.100
[ INFO] [1624207881.557179673]: Device Product ID: 0x0B3A
[ INFO] [1624207881.557273561]: Enable PointCloud: Off
[ INFO] [1624207881.557341657]: Align Depth: Off
[ INFO] [1624207881.557437881]: Sync Mode: Off
[ INFO] [1624207881.557822682]: Device Sensors: 
[ INFO] [1624207881.595640386]: Stereo Module was found.
[ INFO] [1624207881.637386900]: RGB Camera was found.
[ INFO] [1624207881.638489367]: Motion Module was found.
[ INFO] [1624207881.638717912]: (Confidence, 0) sensor isn't supported by current device! -- Skipping...
[ INFO] [1624207881.638836696]: num_filters: 0
[ INFO] [1624207881.638903096]: Setting Dynamic reconfig parameters.
[ INFO] [1624207882.224429978]: Done Setting Dynamic reconfig parameters.
[ INFO] [1624207882.226686240]: depth stream is enabled - width: 848, height: 480, fps: 30, Format: Z16
[ INFO] [1624207882.230841260]: color stream is enabled - width: 1280, height: 720, fps: 30, Format: RGB8
[ INFO] [1624207882.231010284]: setupPublishers...
[ INFO] [1624207882.242717004]: Expected frequency for depth = 30.00000
[ INFO] [1624207882.257253396]: Expected frequency for color = 30.00000
[ INFO] [1624207882.269622454]: setupStreams...
[ INFO] [1624207882.282501113]: insert Depth to Stereo Module
[ INFO] [1624207882.282662458]: insert Color to RGB Camera
[ INFO] [1624207882.384443313]: SELECTED BASE:Depth, 0
[ INFO] [1624207882.443877012]: RealSense Node Is Up!
[ WARN] [1624207882.504123641]: 
 20/06 16:51:22,505 WARNING [547264389504] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 16:51:22,556 WARNING [547264389504] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 16:51:22,607 WARNING [547264389504] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 16:51:22,792 ERROR [546859889024] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 20/06 16:51:22,805 ERROR [546859889024] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 20/06 16:51:22,836 ERROR [546859889024] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 20/06 16:51:22,869 ERROR [546859889024] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 20/06 16:51:22,903 ERROR [546859889024] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 20/06 16:51:22,936 ERROR [546859889024] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 20/06 16:51:22,960 WARNING [547264389504] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 16:51:22,969 ERROR [546859889024] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 20/06 16:51:23,003 ERROR [546859889024] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 20/06 16:51:23,010 WARNING [547264389504] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 16:51:23,036 ERROR [546859889024] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 20/06 16:51:23,070 ERROR [546859889024] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 20/06 16:51:23,103 ERROR [546859889024] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 20/06 16:51:23,137 ERROR [546859889024] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 20/06 16:51:23,161 WARNING [547264389504] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 16:51:23,169 ERROR [546859889024] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 20/06 16:51:23,203 ERROR [546859889024] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 20/06 16:51:23,212 WARNING [547264389504] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 16:51:23,237 ERROR [546859889024] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 20/06 16:51:23,270 ERROR [546859889024] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 20/06 16:51:23,303 ERROR [546859889024] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 20/06 16:51:23,337 ERROR [546859889024] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 20/06 16:51:23,363 WARNING [547264389504] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 16:51:23,396 ERROR [546859889024] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 20/06 16:51:23,414 WARNING [547264389504] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 16:51:23,456 ERROR [546859889024] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
^C[camera/realsense2_camera-3] killing on exit
[camera/realsense2_camera_manager-2] killing on exit
 20/06 16:51:23,516 ERROR [546859889024] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
 20/06 16:51:23,566 WARNING [547264389504] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 16:51:23,577 ERROR [546859889024] (synthetic-stream.cpp:48) Exception was thrown during user processing callback: 
^C 20/06 16:51:23,617 WARNING [547264389504] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61

I am not sure what to try next. I still need to run D435i with realsense-ros inside the container, as everything else is working fine.

Any hints?

Thanks.

AndreV84 commented 3 years ago

@mzahana Did you add the swap at least somewhat 15GB? In Dockerfile you could just update lines to install ROS 2 instead of ROS1, then setup corresponding packages

AndreV84 commented 3 years ago

does realsense viewer run fine from withtin the docker container?

mzahana commented 3 years ago

@AndreV84 I am using a 16GB SD card for boot and 250GB SSD. Here is the output of free

              total        used        free      shared  buff/cache   available
Mem:        7949852     1521752     3949068       39924     2479032     6213464
Swap:       3974904           0     3974904

I guess the swap is 4GB. I followed this nice video from JetsonHacks to use the SSD Now, I am not sure if the swap is on the SD card or the SSD. How to increase it in this case?

Now, I am ssh-ing into the Xavier, so realsense-viewer won't likely work remotely. I can try this tomorrow, though.

AndreV84 commented 3 years ago

@mzahana it seems that you got the OS onto the ssd drive you may do these steps

Turn off zram:

cd /etc/systemd
sudo mv nvzramconfig.sh nvzramconfig.sh.orig
sudo reboot
 Add swap file on nvme and verify:

sudo fallocate -l 32G /home/nvidia/nvme/swapfile
sudo chmod 600 /home/nvidia/nvme/swapfile
sudo mkswap /home/nvidia/nvme/swapfile
sudo swapon /home/nvidia/nvme/swapfile
sudo swapon -s
Add the line below to /etc/fstab so swap file will be automounted going forward:

/home/nvidia/nvme/swapfile swap swap defaults 0 0

for remote realsense-viewer you may install turboVNC + VirtualGL, or else. even if jetson just has some display atatched it wopuld be possible to use realsense-viewer redirecting it to the display while over ssh, but it would require to add nomachine or vino to see also the output. While withotu conencted HDMI display - VirtuaoGL can be used

mzahana commented 3 years ago

@AndreV84 There is no nvme inside my home directory. Do I just create it?

AndreV84 commented 3 years ago

@mzahana you may either just create folder following the steps exactly or adjust paths as per your convinience the only limitation is that there should be enough space as for now

mzahana commented 3 years ago

@AndreV84 I just increased the swap size to 32GB, and I still get the same error :(

mzahana commented 3 years ago

@AndreV84 Do I need to do any kernel patching?

AndreV84 commented 3 years ago

@mzahana you may try ros2 realsnse foxy wrapper within a container https://github.com/IntelRealSense/realsense-ros/releases/tag/3.2.0

mzahana commented 3 years ago

@AndreV84 The point is I am not going to use ROS 2, even if it worked, as everything else is working fine with ROS 1 in the container. I am going to try to use a powered USB hub to see if it's a power issue.

AndreV84 commented 3 years ago

we did the kernel patching but it was only required for d455 on NX due to specific software compatibility typically RSUSB baqckend is used that doesn't require patching; @mzahana Does systemwide ROS1 rtealsense node work if running fropm outside of the container jusy for brief test?

mzahana commented 3 years ago

@AndreV84 I will need to install ROS 1 system wide and test.

mzahana commented 3 years ago

@AndreV84 So, I installed ROS1 system wide and realsense-ros pkg. I get the following Warning/ERROR in the beginning, then it stops appearing. Otherwise I am now able to get 30Hz of both color and depth topics

WARNING [546786234752] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 21:12:30,828 WARNING [546786234752] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 21:12:30,980 WARNING [546786234752] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 21:12:31,232 WARNING [546786234752] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 21:12:31,283 WARNING [546786234752] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 21:12:31,434 WARNING [546786234752] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 21:12:31,485 WARNING [546786234752] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 21:12:31,636 WARNING [546786234752] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 21:12:31,687 WARNING [546786234752] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 21:12:31,838 WARNING [546786234752] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 21:12:31,889 WARNING [546786234752] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 21:12:31,920 ERROR [546391974272] (uvc-streamer.cpp:106) uvc streamer watchdog triggered on endpoint: 132
 20/06 21:12:32,040 WARNING [546786234752] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 21:12:32,091 WARNING [546786234752] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 21:12:32,142 WARNING [546786234752] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 21:12:32,194 WARNING [546786234752] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 21:12:32,244 WARNING [546786234752] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 21:12:32,295 WARNING [546786234752] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61
 20/06 21:12:32,346 WARNING [546786234752] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61

What could be missing inside the container then, if it works fine outside it?

AndreV84 commented 3 years ago

@mzahana is the rgb topic still empty in the docker container? you could also try to install in the container realsense sdk from apt repository

mzahana commented 3 years ago

@AndreV84 I installed both realsense SDK and realsense-ros from apt repos, and I was able to get color/depth topics inside the container. However, I am not able to get pointcloud inside the container, but I get it outside the container.

roslaunch realsense2_camera rs_camera.launch filters:="pointcloud"

I get,

[ INFO] [1624217370.319602053]: publishing unordered pointcloud.
[ WARN] [1624217370.792778998]: No stream match for pointcloud chosen texture Process - Color

That was the main reason that I switched to realsense sDK and ros pkg from source instead from apt repos. If I can get pointcloud inside the container, then problem solved.

AndreV84 commented 3 years ago

@mzahana

try different usb ports of the NX ; some ports will do better than others [ it is the case with zed camera]
try using the source code realsense sdk. could you confirm it also got the rgb topics published inside the container?
try with/ without powered/ unpowered udb hubs with various cables

IntelRealSense / realsense-ros

(synthetic-stream.cpp:48) Exception was thrown during user processing callback: Out of frame resources! #1604