L515 stops working when moved in z-direction

[shervinkoushan]

We have multiple L515 cameras that all drop out (stops publishing data) when the camera is moved up and down in the z-direction. See https://youtube.com/shorts/xxIAm4CA2pE?feature=share for a demo

The same movement does not cause any harm when it is performed in x or y direction. Could this be related to the IMU?

We see that the IR light stops shining for a second when the camera drops out.

This was with realsense2_camera/rs_camera.launch

Error messages:

10/10 11:40:47,173 WARNING [140140175984384] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 10/10 11:41:24,108 ERROR [140139581912832] (uvc-streamer.cpp:106) uvc streamer watchdog triggered on endpoint: 130
 10/10 11:41:24,109 ERROR [140139347048192] (uvc-streamer.cpp:106) uvc streamer watchdog triggered on endpoint: 131
 10/10 11:41:24,256 WARNING [140140175984384] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[ WARN] [/camera/realsense2_camera_manager] [1696930884.988854669]: Hardware Notification:Fatal error occurred (15),1.69693e+12,Error,Hardware Error

Same issue with realsense viewer

[shervinkoushan]

Output when launching:

Details

``` PARAMETERS * /camera/realsense2_camera/accel_fps: -1 * /camera/realsense2_camera/accel_frame_id: camera_accel_frame * /camera/realsense2_camera/accel_optical_frame_id: camera_accel_opti... * /camera/realsense2_camera/align_depth: False * /camera/realsense2_camera/aligned_depth_to_color_frame_id: camera_aligned_de... * /camera/realsense2_camera/aligned_depth_to_fisheye1_frame_id: camera_aligned_de... * /camera/realsense2_camera/aligned_depth_to_fisheye2_frame_id: camera_aligned_de... * /camera/realsense2_camera/aligned_depth_to_fisheye_frame_id: camera_aligned_de... * /camera/realsense2_camera/aligned_depth_to_infra1_frame_id: camera_aligned_de... * /camera/realsense2_camera/aligned_depth_to_infra2_frame_id: camera_aligned_de... * /camera/realsense2_camera/allow_no_texture_points: False * /camera/realsense2_camera/base_frame_id: camera_link * /camera/realsense2_camera/calib_odom_file: * /camera/realsense2_camera/clip_distance: -2.0 * /camera/realsense2_camera/color_fps: -1 * /camera/realsense2_camera/color_frame_id: camera_color_frame * /camera/realsense2_camera/color_height: -1 * /camera/realsense2_camera/color_optical_frame_id: camera_color_opti... * /camera/realsense2_camera/color_width: -1 * /camera/realsense2_camera/confidence_fps: -1 * /camera/realsense2_camera/confidence_height: -1 * /camera/realsense2_camera/confidence_width: -1 * /camera/realsense2_camera/depth_fps: -1 * /camera/realsense2_camera/depth_frame_id: camera_depth_frame * /camera/realsense2_camera/depth_height: -1 * /camera/realsense2_camera/depth_optical_frame_id: camera_depth_opti... * /camera/realsense2_camera/depth_width: -1 * /camera/realsense2_camera/device_type: * /camera/realsense2_camera/enable_accel: False * /camera/realsense2_camera/enable_color: True * /camera/realsense2_camera/enable_confidence: True * /camera/realsense2_camera/enable_depth: True * /camera/realsense2_camera/enable_fisheye1: False * /camera/realsense2_camera/enable_fisheye2: False * /camera/realsense2_camera/enable_fisheye: False * /camera/realsense2_camera/enable_gyro: False * /camera/realsense2_camera/enable_infra1: False * /camera/realsense2_camera/enable_infra2: False * /camera/realsense2_camera/enable_infra: False * /camera/realsense2_camera/enable_pointcloud: False * /camera/realsense2_camera/enable_pose: False * /camera/realsense2_camera/enable_sync: False * /camera/realsense2_camera/filters: pointcloud * /camera/realsense2_camera/fisheye1_frame_id: camera_fisheye1_f... * /camera/realsense2_camera/fisheye1_optical_frame_id: camera_fisheye1_o... * /camera/realsense2_camera/fisheye2_frame_id: camera_fisheye2_f... * /camera/realsense2_camera/fisheye2_optical_frame_id: camera_fisheye2_o... * /camera/realsense2_camera/fisheye_fps: -1 * /camera/realsense2_camera/fisheye_frame_id: camera_fisheye_frame * /camera/realsense2_camera/fisheye_height: -1 * /camera/realsense2_camera/fisheye_optical_frame_id: camera_fisheye_op... * /camera/realsense2_camera/fisheye_width: -1 * /camera/realsense2_camera/gyro_fps: -1 * /camera/realsense2_camera/gyro_frame_id: camera_gyro_frame * /camera/realsense2_camera/gyro_optical_frame_id: camera_gyro_optic... * /camera/realsense2_camera/imu_optical_frame_id: camera_imu_optica... * /camera/realsense2_camera/infra1_frame_id: camera_infra1_frame * /camera/realsense2_camera/infra1_optical_frame_id: camera_infra1_opt... * /camera/realsense2_camera/infra2_frame_id: camera_infra2_frame * /camera/realsense2_camera/infra2_optical_frame_id: camera_infra2_opt... * /camera/realsense2_camera/infra_fps: 30 * /camera/realsense2_camera/infra_height: 480 * /camera/realsense2_camera/infra_rgb: False * /camera/realsense2_camera/infra_width: 848 * /camera/realsense2_camera/initial_reset: False * /camera/realsense2_camera/json_file_path: /home/shervin/wor... * /camera/realsense2_camera/linear_accel_cov: 0.01 * /camera/realsense2_camera/odom_frame_id: camera_odom_frame * /camera/realsense2_camera/ordered_pc: False * /camera/realsense2_camera/pointcloud_texture_index: 0 * /camera/realsense2_camera/pointcloud_texture_stream: RS2_STREAM_ANY * /camera/realsense2_camera/pose_frame_id: camera_pose_frame * /camera/realsense2_camera/pose_optical_frame_id: camera_pose_optic... * /camera/realsense2_camera/publish_odom_tf: True * /camera/realsense2_camera/publish_tf: True * /camera/realsense2_camera/reconnect_timeout: 6.0 * /camera/realsense2_camera/rosbag_filename: * /camera/realsense2_camera/serial_no: * /camera/realsense2_camera/stereo_module/exposure/1: 7500 * /camera/realsense2_camera/stereo_module/exposure/2: 1 * /camera/realsense2_camera/stereo_module/gain/1: 16 * /camera/realsense2_camera/stereo_module/gain/2: 16 * /camera/realsense2_camera/tf_publish_rate: 0.0 * /camera/realsense2_camera/topic_odom_in: odom_in * /camera/realsense2_camera/unite_imu_method: * /camera/realsense2_camera/usb_port_id: * /camera/realsense2_camera/wait_for_device_timeout: -1.0 * /rosdistro: melodic * /rosversion: 1.14.13 NODES /camera/ realsense2_camera (nodelet/nodelet) realsense2_camera_manager (nodelet/nodelet) auto-starting new master process[master]: started with pid [15340] ROS_MASTER_URI=http://localhost:11311 setting /run_id to e8c4dbd6-6766-11ee-8852-d45d64079e37 process[rosout-1]: started with pid [15359] started core service [/rosout] process[camera/realsense2_camera_manager-2]: started with pid [15366] process[camera/realsense2_camera-3]: started with pid [15367] [ INFO] [/camera/realsense2_camera_manager] [1696940218.994824255]: Initializing nodelet with 8 worker threads. [ INFO] [/camera/realsense2_camera_manager] [1696940219.065197367]: RealSense ROS v2.3.2 [ INFO] [/camera/realsense2_camera_manager] [1696940219.065219939]: Built with LibRealSense v2.50.0 [ INFO] [/camera/realsense2_camera_manager] [1696940219.065228941]: Running with LibRealSense v2.50.0 [ INFO] [/camera/realsense2_camera_manager] [1696940219.083547518]: 10/10 14:16:59,416 WARNING [139922736469760] (messenger-libusb.cpp:42) control_transfer returned error, index: 300, error: Success, number: 0 10/10 14:16:59,532 WARNING [139922736469760] (messenger-libusb.cpp:42) control_transfer returned error, index: 300, error: Success, number: 0 10/10 14:16:59,587 WARNING [139922736469760] (messenger-libusb.cpp:42) control_transfer returned error, index: 300, error: Success, number: 0 [ INFO] [/camera/realsense2_camera_manager] [1696940219.690898849]: Device with serial number f0221119 was found. [ INFO] [/camera/realsense2_camera_manager] [1696940219.691011944]: Device with physical ID 2-6-3 was found. [ INFO] [/camera/realsense2_camera_manager] [1696940219.691058514]: Device with name Intel RealSense L515 was found. [ INFO] [/camera/realsense2_camera_manager] [1696940219.692318269]: Device with port number 2-6 was found. [ INFO] [/camera/realsense2_camera_manager] [1696940219.692397298]: Device USB type: 3.2 [ INFO] [/camera/realsense2_camera_manager] [1696940219.696803013]: getParameters... [ INFO] [/camera/realsense2_camera_manager] [1696940219.794546414]: setupDevice... 10/10 14:16:59,853 WARNING [139922736469760] (messenger-libusb.cpp:42) control_transfer returned error, index: 300, error: Success, number: 0 10/10 14:16:59,908 WARNING [139922736469760] (messenger-libusb.cpp:42) control_transfer returned error, index: 300, error: Success, number: 0 [ INFO] [/camera/realsense2_camera_manager] [1696940219.982165566]: JSON file is loaded! [ INFO] [/camera/realsense2_camera_manager] [1696940219.982315513]: ROS Node Namespace: camera [ INFO] [/camera/realsense2_camera_manager] [1696940219.982379426]: Device Name: Intel RealSense L515 [ INFO] [/camera/realsense2_camera_manager] [1696940219.982421834]: Device Serial No: f0221119 [ INFO] [/camera/realsense2_camera_manager] [1696940219.982463325]: Device physical port: 2-6-3 [ INFO] [/camera/realsense2_camera_manager] [1696940219.982494580]: Device FW version: 01.05.08.01 [ INFO] [/camera/realsense2_camera_manager] [1696940219.982526125]: Device Product ID: 0x0B64 [ INFO] [/camera/realsense2_camera_manager] [1696940219.982556398]: Enable PointCloud: On [ INFO] [/camera/realsense2_camera_manager] [1696940219.982586070]: Align Depth: Off [ INFO] [/camera/realsense2_camera_manager] [1696940219.982615210]: Sync Mode: On [ INFO] [/camera/realsense2_camera_manager] [1696940219.982725020]: Device Sensors: [ INFO] [/camera/realsense2_camera_manager] [1696940220.083713262]: L500 Depth Sensor was found. [ INFO] [/camera/realsense2_camera_manager] [1696940220.094641606]: RGB Camera was found. [ INFO] [/camera/realsense2_camera_manager] [1696940220.094935655]: Motion Module was found. [ INFO] [/camera/realsense2_camera_manager] [1696940220.094992720]: Add Filter: pointcloud [ INFO] [/camera/realsense2_camera_manager] [1696940220.095780876]: num_filters: 1 [ INFO] [/camera/realsense2_camera_manager] [1696940220.095816792]: Setting Dynamic reconfig parameters. [ INFO] [/camera/realsense2_camera_manager] [1696940221.410853022]: Done Setting Dynamic reconfig parameters. [ INFO] [/camera/realsense2_camera_manager] [1696940221.411104215]: depth stream is enabled - width: 640, height: 480, fps: 30, Format: Z16 [ INFO] [/camera/realsense2_camera_manager] [1696940221.411801620]: color stream is enabled - width: 1280, height: 720, fps: 30, Format: RGB8 [ INFO] [/camera/realsense2_camera_manager] [1696940221.411826839]: setupPublishers... [ INFO] [/camera/realsense2_camera_manager] [1696940221.413155970]: Expected frequency for depth = 30.00000 [ INFO] [/camera/realsense2_camera_manager] [1696940221.415449600]: Expected frequency for color = 30.00000 [ INFO] [/camera/realsense2_camera_manager] [1696940221.416968827]: setupStreams... 10/10 14:17:01,644 WARNING [139922736469760] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11 [ INFO] [/camera/realsense2_camera_manager] [1696940221.658165217]: SELECTED BASE:Depth, 0 10/10 14:17:01,695 WARNING [139922736469760] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11 10/10 14:17:01,746 WARNING [139922736469760] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11 [ INFO] [/camera/realsense2_camera_manager] [1696940221.798072896]: RealSense Node Is Up! [ WARN] [/camera/realsense2_camera_manager] [1696940221.811315118]: [ WARN] [/camera/realsense2_camera_manager] [1696940221.811349884]: frame's time domain is HARDWARE_CLOCK. Timestamps may reset periodically. 10/10 14:17:02,348 WARNING [139922736469760] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11 10/10 14:17:02,697 WARNING [139922736469760] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11 10/10 14:17:03,949 WARNING [139922736469760] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11 10/10 14:17:05,200 WARNING [139922736469760] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11 10/10 14:17:06,452 WARNING [139922736469760] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11 10/10 14:17:07,703 WARNING [139922736469760] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11 ```

[MartyG-RealSense]

Hi @shervinkoushan I used your side-on shaking movement with an L515 but was not able to repeat the problem that you experienced. The official trailer video for L515 shows that it is capable of being used during high-speed travel.

https://www.youtube.com/watch?v=Rx-75ZEAMeg&t=36s

Could you first try unplugging the micro-sized end of the USB cable from the base of the L515 camera, reversing the connector's orientation and plugging it back into the camera, please (USB-C cables are two-way insertion at the micro-sized end). Does doing so make any difference?

The RealSense IMU can lose its tracking when the camera is turned sharply, though it looks as though you are using the camera/depth/color/points topic, which would not involve the IMU.

[shervinkoushan]

Hi @shervinkoushan I used your side-on shaking movement with an L515 but was not able to repeat the problem that you experienced. The official trailer video for L515 shows that it is capable of being used during high-speed travel.

Thank you for the prompt response. Note that the issue only occurs when the camera is abruptly moved in the z-direction.

Could you first try unplugging the micro-sized end of the USB cable from the base of the L515 camera, reversing the connector's orientation and plugging it back into the camera, please (USB-C cables are two-way insertion at the micro-sized end). Does doing so make any difference?

Unfortunately, the problem persists with the other side of the cable as well.

[MartyG-RealSense]

Does it occur if you move the camera quickly only in one direction and not alternating in two directions? For example, holding the camera high above your head and then plunging it down.

[shervinkoushan]

Does it occur if you move the camera quickly only in one direction and not alternating in two directions? For example, holding the camera high above your head and then plunging it down.

I would say it is easiest to reproduce by moving steadily up and down. I tried plunging it from both directions but that did not cause any issues.

[MartyG-RealSense]

If your real-world use of the camera will not involve the camera rapidly moving up and down then it may be worth waiting to see whether a problem occurs with fast travel in one direction, as the issue may not ever occur during normal use.

[shervinkoushan]

If your real-world use of the camera will not involve the camera rapidly moving up and down then it may be worth waiting to see whether a problem occurs with fast travel in one direction, as the issue may not ever occur during normal use.

Unfortunately it occurs quite often as the camera is attached to a drone

[MartyG-RealSense]

Does it still occur if you move the camera up-down at a slower speed, about half the speed that it is shaken in the video?

Will the camera be in that orientation in real-world use, as it is being held side-on (normal upright orientation is with the tripod on the bottom facing downwards)

[shervinkoushan]

Does it still occur if you move the camera up-down at a slower speed, about half the speed that it is shaken in the video?

Yes, also occurs at slightly lower speeds.

Will the camera be in that orientation in real-world use, as it is being held side-on (normal upright orientation is with the tripod on the bottom facing downwards)

It will not be sideways, but tilted at a 45 degree angle. Like this, but more tilted forward:

It is attached under the drone, pointing downwards at 45 degrees.

Any idea if this is a software or firmware issue? I have sometimes had to force shut down my PC after this happens.

[MartyG-RealSense]

I have repeated the shake test in the same orientation with two different L515 cameras on two different PCs and was unable to replicate your issue.

When the camera is being shaken, please look at the large USB connector in your computer to see if the connector is bouncing up and down in the USB port during shaking. If it is not rigidly locked in the computer's port then the up-down movement of the connector may be causing problems. You could try firmly holding the connector at the computer end as you shake the camera to hold it steady and see if it makes a difference.

[shervinkoushan]

Here is a new video with another camera. If you have any other suggestions (firmware, software, building from source etc I would be happy to test it)

https://github.com/IntelRealSense/realsense-ros/assets/59822141/0e356352-1779-4dc7-931b-dc0dc37b8c8c

[MartyG-RealSense]

In the above video the firmware version is 1.5.4.1. There have been a couple of firmware updates since then and it is recommended to update to the latest available one, 1.5.8.1. You can use the 'Install' button offered in the pop-up message in the top corner of the Viewer to do so.

[shervinkoushan]

In the above video the firmware version is 1.5.4.1. There have been a couple of firmware updates since then and it is recommended to update to the latest available one, 1.5.8.1. You can use the 'Install' button offered in the pop-up message in the top corner of the Viewer to do so.

Thanks, but in the first video the version was 1.5.8.1 :/

[MartyG-RealSense]

If you left-click on a small arrow icon at the bottom corner of the Viewer window to expand open the Viewer's debug log then you can see which error messages appear at the moment that the image stops updating.

[shervinkoushan]

If you left-click on a small arrow icon at the bottom corner of the Viewer window to expand open the Viewer's debug log then you can see which error messages appear at the moment that the image stops updating.

This is the output:

[MartyG-RealSense]

This message will have a similar cause to the control_transfer returned warnings in ROS, indicating that the camera has not provided new image frames for a period of more than 5 seconds.

If the L515 camera is still listed in the side-panel of the Viewer during these log warnings then it suggests that the camera has not become disconnected by the rapid up-down motion.

In regard to the red Fatal error occurred error, the fact that the camera is restorable to normal operation after it has occurred during rapid up-down movement indicates that it is a temporary glitch rather than a permanent camera hardware fault.

During further tests, I was able to replicate the error in the Viewer and the accompanying depth freeze with some difficulty by wildly swinging the camera up and down and in circular movements for a period of about 10 seconds. Stopping the camera and re-enabling it cleared the problem and enabled the camera to continue functioning normally. The error could not be generated again after the restart when repeating the same motions.

Does the issue occur if you first go to the More option at the top of the Viewer side-panel and select Hardware Reset from its menu to reset the camera and then enable the stream afterwards and shake the camera?

[shervinkoushan]

Thanks for the info.

1. The camera is still listed in the side panel when the error occurs
2. The camera works for some time after restart, but when moving it up and down the same error occurs
3. Yes, still able to reproduce the issue after a Hardware Reset

[shervinkoushan]

I noticed this parameter that might be relevant: freefall_detection_enabled

[shervinkoushan]

I have not been able to reproduce the issue after setting it to false

[MartyG-RealSense]

It sounds logical that disabling freefall detection would affect how the camera deals with rapid motion. This option shuts the sensor down when a free-fall is detected. It is enabled by default.

[shervinkoushan]

We see a lot less dropouts when running on the drone as well now.

Still, sometimes we lose data (even when the drone is hovering, so another issue). I wonder if it is related to noise

[MartyG-RealSense]

If severe vibration is being transmitted through the drone's frame to the camera during flight then that could disrupt the connection. if that is the case then adding vibration damping pads to the camera mounting points could help with this.

[MartyG-RealSense]

Hi @shervinkoushan Do you require further assistance with this case, please? Thanks!

[shervinkoushan]

Hi @shervinkoushan Do you require further assistance with this case, please? Thanks!

We will try to solve the remaining issues by adding more damping. Thanks for the help!

[MartyG-RealSense]

You are very welcome. Thanks very much for the update!