Lag in camera feed

[MAVProxyUser]

Ensure connected to USB3 hub:

Note there are both USB 2 & 3 hubs on the Jetson boards.

ubuntu@ubuntu:~/yolov7$ sudo lshw -short | grep Hub
/1/2                            bus            USB2.0 Hub
/2/3                            bus            USB3.0 Hub
ubuntu@ubuntu:~/yolov7$ lsusb | grep 0b5c
Bus 002 Device 006: ID 8086:0b5c Intel Corp. 

You can see above our Realsense cam (USB ID 0b5c) is on bus 2, which is the USB 3.0 hub.

[MAVProxyUser]

Note the speed difference. The USB3 hub is listed as 5000Mbit/s and the USB2 is 480Mbit/s

ubuntu@ubuntu:~/yolov7$ sudo lshw -C bus
...
  *-usbhost:0
       product: xHCI Host Controller
       vendor: Linux 5.10.104-tegra xhci-hcd
       physical id: 1
       bus info: usb@1
       logical name: usb1
       version: 5.10
       capabilities: usb-2.00
       configuration: driver=hub slots=4 speed=480Mbit/s
     *-usb:0
          description: USB hub
          product: USB2.0 Hub
          vendor: VIA Labs, Inc.
          physical id: 2
          bus info: usb@1:2
          version: 90.23
          capabilities: usb-2.10
          configuration: driver=hub slots=4 speed=480Mbit/s
  *-usbhost:1
       product: xHCI Host Controller
       vendor: Linux 5.10.104-tegra xhci-hcd
       physical id: 2
       bus info: usb@2
       logical name: usb2
       version: 5.10
       capabilities: usb-3.10
       configuration: driver=hub slots=4 speed=10000Mbit/s
     *-usb
          description: USB hub
          product: USB3.0 Hub
          vendor: VIA Labs, Inc.
          physical id: 3
          bus info: usb@2:3
          version: 90.23
          capabilities: usb-3.10
          configuration: driver=hub slots=4 speed=5000Mbit/s

[MAVProxyUser]

We need to find the optimal camera settings as well

#!/usr/bin/python3.8
import pyrealsense2 as rs
import numpy as np
import cv2

pipeline = rs.pipeline()
config = rs.config()

profile = pipeline.start(config)

devices = profile.get_device().query_sensors()
# [<pyrealsense2.sensor: "Stereo Module">, <pyrealsense2.sensor: "RGB Camera">, <pyrealsense2.sensor: "Motion Module">]
print(devices)

# https://intelrealsense.github.io/librealsense/python_docs/_generated/pyrealsense2.options.html
depth_sensor = profile.get_device().first_depth_sensor()
print(depth_sensor.get_supported_options())

color_sensor = profile.get_device().first_color_sensor()
print(color_sensor.get_supported_options())

motion_sensor = profile.get_device().first_motion_sensor()
print(motion_sensor.get_supported_options())

My current settings:

[<pyrealsense2.sensor: "Stereo Module">, <pyrealsense2.sensor: "RGB Camera">, <pyrealsense2.sensor: "Motion Module">]
[<option.exposure: 3>, <option.gain: 4>, <option.enable_auto_exposure: 10>, <option.visual_preset: 12>, <option.laser_power: 13>, <option.emitter_enabled: 18>, <option.frames_queue_size: 19>, <option.asic_temperature: 23>, <option.error_polling_enabled: 24>, <option.projector_temperature: 25>, <option.output_trigger_enabled: 26>, <option.depth_units: 28>, <option.stereo_baseline: 40>, <option.inter_cam_sync_mode: 42>, <option.emitter_on_off: 46>, <option.global_time_enabled: 53>, <option.emitter_always_on: 71>, <option.thermal_compensation: 72>, <option.hdr_enabled: 76>, <option.sequence_name: 77>, <option.sequence_size: 78>, <option.sequence_id: 79>, <option.emitter_frequency: 93>]
[<option.backlight_compensation: 0>, <option.brightness: 1>, <option.contrast: 2>, <option.exposure: 3>, <option.gain: 4>, <option.gamma: 5>, <option.hue: 6>, <option.saturation: 7>, <option.sharpness: 8>, <option.white_balance: 9>, <option.enable_auto_exposure: 10>, <option.enable_auto_white_balance: 11>, <option.frames_queue_size: 19>, <option.power_line_frequency: 22>, <option.auto_exposure_priority: 30>, <option.global_time_enabled: 53>]
[<option.frames_queue_size: 19>, <option.enable_motion_correction: 29>, <option.global_time_enabled: 53>]

https://support.intelrealsense.com/hc/en-us/community/posts/1500001025701-Cannot-get-stable-framerate-on-RealSense-D455

[MAVProxyUser]

@cherbini this applies to Luxonis lag too. Make sure you are using USB 3.