[BUG] UVC demo app does not work with OAK-D-Lite

[zegkljan]

Describe the bug Running python3 demo_script.py --app uvc with OAK-D-Lite creates /dev/video device(s) only for a few seconds, the camera then disconnects and reconnects again as a non-uvc device, the /dev/video devices disappear.

To Reproduce (interlaced with console and dmesg outputs)

1. Connect OAK-D-Lite

dmesg:

[ 3283.174698] usb 1-1: new high-speed USB device number 36 using xhci_hcd
[ 3283.315643] usb 1-1: New USB device found, idVendor=03e7, idProduct=2485, bcdDevice= 0.01
[ 3283.315665] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 3283.315674] usb 1-1: Product: Movidius MyriadX
[ 3283.315681] usb 1-1: Manufacturer: Movidius Ltd.
[ 3283.315687] usb 1-1: SerialNumber: 03e72485

v4l2-ctl --list-device:

v4l2loopback (platform:v4l2loopback-000):
    /dev/video9

HD WebCam: HD WebCam (usb-0000:00:14.0-7):
    /dev/video0
    /dev/video1
    /dev/media0

2. Run python3 depthai_demo.py --app uvc

python3 depthai_demo.py --app uvc console:

Existing venv found.
Installing requirements...
Requirement already satisfied: pip in ./venv/lib/python3.10/site-packages (21.3.1)
Looking in indexes: https://pypi.org/simple, https://artifacts.luxonis.com/artifactory/luxonis-python-snapshot-local/
Requirement already satisfied: depthai==2.13.3.0.dev+f513b0f27b4f07a9940cd9dfe65d7fd6deeaf29d in ./venv/lib/python3.10/site-packages (from -r /home/zegkljan/tmp/depthai/apps/uvc/requirements.txt (line 2)) (2.13.3.0.dev0+f513b0f27b4f07a9940cd9dfe65d7fd6deeaf29d)
[18443010312C7C0F00] [89.076] [system] [info] PRINT:LeonCss: bss_init took 42.307 ms
BootloaderConfig.options1 checksum doesn't match. Is: 0x10304418 should be: 0x7018171A
BootloaderConfig.options2 checksum doesn't match. Is: 0x00214442 should be: 0x11FD80E1
GPIO boot mode 0x16, interface USBD
Setting aons(0..4) back to boot from flash (offset = 0)
====ENABLE WATCHDOG====1
--> brdInit ...
initial keepalive, countdown: 10
PLL0: 700000 AUX_IO0: 24000 AUX_IO1: 24000 MCFG: 24000 MECFG: 24000
Board init ret 3
brdInitAuxDevices: Error: SC = 27: io_initialize expander_cam_gpios_1 [OK]

spi_N25Q_init: Flash JEDEC ID: ff ff ff
eeprom_read_status 0!
Reading VERSION 6 --- ->
Closing EEPROm!
Is booted from flash by bootloader: 0
Networking not available...
Called by: LOS, controller: LOS
Enumerating on socket: Cam_A / RGB / Center
Probe failed I2C1 0x10 reg 300a: expected 9281, read 0000
Probe failed I2C1 0x10 reg 300a: expected 9281, read 0000
Probe failed I2C1 0x10 reg 300a: expected 9281, read 0000
Probe failed I2C1 0x10 reg 300a: expected 9281, read 0000
Probe failed I2C1 0x10 reg 300a: expected 9281, read 0000
Probe failed I2C1 0x10 reg 300a: expected 9281, read 0000
Probe failed I2C1 0x10 reg 300a: expected 9281, read 0000
Probe failed I2C1 0x10 reg 300a: expected 7750, read 0000
Probe failed I2C1 0x10 reg 300a: expected 7750, read 0000
Probe failed I2C1 0x10 reg 300a: expected 7750, read 0000
Probe failed I2C1 0x10 reg 300a: expected 7750, read 0000
  >> Registered camera IMX214R0 (imx214) as /dev/Camera_0
Enumerating on socket: Cam_B / Left
Probe failed I2C2 0x60 reg 300a: expected 9281, read 7750
Probe failed I2C2 0x60 reg 300a: expected 9281, read 7750
Probe failed I2C2 0x60 reg 300a: expected 9281, read 7750
Probe failed I2C2 0x60 reg 300a: expected 9281, read 7750
Probe failed I2C2 0x60 reg 300a: expected 9281, read 7750
Probe failed I2C2 0x60 reg 300a: expected 9281, read 7750
  >> Registered camera MV200 (ov7750) as /dev/Camera_1
Enumerating on socket: Cam_C / Right
Probe failed I2C2 0x62 reg 300a: expected 9281, read 7750
Probe failed I2C2 0x60 reg 300a: expected 9281, read 7750
Probe failed I2C2 0x60 reg 300a: expected 9281, read 7750
Probe failed I2C2 0x60 reg 300a: expected 9281, read 7750
Probe failed I2C2 0x60 reg 300a: expected 9281, read 7750
Probe failed I2C2 0x60 reg 300a: expected 9281, read 7750
Probe failed I2C2 0x60 reg 300a: expected 9281, read 7750
  >> Registered camera MV200 (ov7750) as /dev/Camera_2
Enumerating on socket: CAM_D
initial keepalive, countdown: 9
Initializing XLink...
UsbPumpVscAppI_Event: 5 VSC2_EVENT_ATTACH
UsbPumpVscAppI_Event: 2 VSC2_EVENT_SUSPEND
UsbPumpVscAppI_Event: 3 VSC2_EVENT_RESUME
UsbPumpVscAppI_Event: 4 VSC2_EVENT_RESET
initial keepalive, countdown: 8
UsbPumpVscAppI_Event: 4 VSC2_EVENT_RESET
UsbPumpVscAppI_Event: 4 VSC2_EVENT_RESET
UsbPumpVscAppI_Event: 4 VSC2_EVENT_RESET
UsbPumpVscAppI_Event: 2 VSC2_EVENT_SUSPEND
UsbPumpVscAppI_Event: 3 VSC2_EVENT_RESUME
UsbPumpVscAppI_Event: 0 VSC2_EVENT_INTERFACE_UP
UsbPumpVscAppI_Event: 2 VSC2_EVENT_SUSPEND
UsbPumpVscAppI_Event: 3 VSC2_EVENT_RESUME
UsbPumpVscAppI_Event: 2 VSC2_EVENT_SUSPEND
UsbPumpVscAppI_Event: 3 VSC2_EVENT_RESUME
Done!
Usb connection speed: High - USB 2.0
I: [Timesync] [   1797428] [main] startSync:116     Timesync | Callback not set
Temperature: Driver registered.
Temperature: Initialized driver.
Temperature: Sensor opened: CSS.
Temperature: Sensor opened: MSS.
Temperature: Sensor opened: UPA.
Temperature: Sensor opened: DSS.
[18443010312C7C0F00] [89.076] [system] [info] PRINT:LeonMss: Called by: LRT, controller: LOS

Device started, please keep this process running
and open an UVC viewer. Example on Linux:
    guvcview -d /dev/video0

To close: Ctrl+C

dmesg:

[ 3390.764840] usb 1-1: Found UVC 1.00 device Luxonis Device (03e7:f63b)
[ 3390.765830] usb 1-1: Failed to query (GET_INFO) UVC control 9 on unit 1: -32 (exp. 1).
[ 3393.705249] usb 1-1: USB disconnect, device number 37
[ 3394.011224] usb 1-1: new high-speed USB device number 38 using xhci_hcd
[ 3394.151918] usb 1-1: New USB device found, idVendor=03e7, idProduct=2485, bcdDevice= 0.01
[ 3394.151922] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 3394.151924] usb 1-1: Product: Movidius MyriadX
[ 3394.151926] usb 1-1: Manufacturer: Movidius Ltd.
[ 3394.151927] usb 1-1: SerialNumber: 03e72485

3. There are no new /dev/video* devices. Also, the app uses 100 % of one CPU core. v4l2-ctl --list-device:

   
   v4l2loopback (platform:v4l2loopback-000):
   /dev/video9

HD WebCam: HD WebCam (usb-0000:00:14.0-7): /dev/video0 /dev/video1 /dev/media0

Sometimes, however it takes some time (on the order of seconds, or lower 10s of secods) during which the `/dev/video*` devices are present:

v4l2loopback (platform:v4l2loopback-000): /dev/video9

Luxonis Device: UVC Video Contr (usb-0000:00:14.0-1): /dev/video2 /dev/video3 /dev/media1

HD WebCam: HD WebCam (usb-0000:00:14.0-7): /dev/video0 /dev/video1 /dev/media0

Sometimes I also get the following segfault from `depthai_demo.py` (and in that case it does not use up 100 % of CPU, but does not exit either, I still have to Ctrl-C it):

... Device started, please keep this process running and open an UVC viewer. Example on Linux: guvcview -d /dev/video0

To close: Ctrl+C Stack trace (most recent call last) in thread 11826:

2 Object "[", at 0, in nil

1 Object "/usr/lib/libc.so.6", at 0x7f420ec0ed9f, in

0 Object "/home/zegkljan/tmp/depthai/apps/uvc/venv/lib/python3.10/site-packages/depthai.cpython-310-x86_64-linux-gnu.so", at 0x7f420d6fb785, in backward::SignalHandling::sig_handler(int, siginfo_t, void)

Segmentation fault (Address not mapped to object [(nil)])

The two "sometimes" alternatives happen seemingly at random, I have not noticed any pattern.

**Expected behavior**
The UVC app should work, i.e. provide a working UVC device(s).

**System log**

{ "architecture": "64bit ELF", "machine": "x86_64", "platform": "Linux-5.16.1-arch1-1-x86_64-with-glibc2.33", "processor": "", "python_build": "main Dec 18 2021 23:53:45", "python_compiler": "GCC 11.1.0", "python_implementation": "CPython", "python_version": "3.10.1", "release": "5.16.1-arch1-1", "system": "Linux", "version": "#1 SMP PREEMPT Sun, 16 Jan 2022 11:39:23 +0000", "win32_ver": "", "packages": [ "appdirs==1.4.4", "Brlapi==0.8.3", "btrfsutil==5.16", "CacheControl==0.12.6", "cffi==1.15.0", "chardet==4.0.0", "chrome-gnome-shell==0.0.0", "colorama==0.4.4", "contextlib2==0.6.0.post1", "cryptography==36.0.1", "depthai==2.14.1.0", "distlib==0.3.4", "distro==1.6.0", "dnspython==2.1.0", "filelock==3.4.0", "guake==3.8.1", "html5lib==1.1", "idna==3.3", "importlib-metadata==4.8.1", "lensfun==0.3.95", "libfdt==1.6.1", "louis==3.20.0", "Markdown==3.3.6", "more-itertools==8.10.0", "msgpack==1.0.3", "netsnmp-python==1.0a1", "numpy==1.21.5", "opencv-contrib-python==4.5.5.62", "opencv-python==4.5.5.62", "ordered-set==4.0.2", "packaging==20.9", "pbr==5.8.0", "pep517==0.12.0", "pip==20.3.4", "platformdirs==2.4.1", "ply==3.11", "progress==1.6", "pwquality==1.4.4", "pybind11==2.9.0", "pycairo==1.20.1", "pycparser==2.21", "PyGObject==3.42.0", "pyOpenSSL==21.0.0", "pyparsing==2.4.7", "pyvirtualcam==0.9.0", "pyxdg==0.27", "requests==2.26.0", "resolvelib==0.5.5", "retrying==1.3.3", "setuptools==59.1.1", "six==1.16.0", "TBB==0.2", "team==1.0", "toml==0.10.2", "tomli==1.2.3", "urllib3==1.26.7", "virtualenv==20.11.0", "webencodings==0.5.1", "zipp==3.6.0" ], "usb": [ "NoLib" ], "uname": [ "Linux eddie 5.16.1-arch1-1 #1 SMP PREEMPT Sun, 16 Jan 2022 11:39:23 +0000 x86_64" ] }

**Additional context**
The OAK-D-Lite is connected via USB-C to USB-A cable, sadly only an USB 2 cable. I'm getting an USB 3.1 cable very soon.

[Luxonis-Brandon]

Thanks for the thorough details and sorry about the trouble here!

I just reproduced the problem using a USB2 cable on an M1 Max Macbook Pro with my OAK-D-Lite. And then using a USB3 cable it actually worked fine:

So this is something to do with using a USB2 cable and the device acting as a webcam (UVC).

So the USB3(.1) cable you are getting should resolve this. I will need @alex-luxonis insight on UVC over USB2. I tried with python3 depthai_demo.py -usbs usb2 -app uvc but this had no impact, which I think makes sense as I don't think the usbs flag would have any bearing on UVC, as it's a setting for the actual DepthAI communication, not UVC communication.

Thanks and sorry again about the trouble.

[VanDavv]

@Luxonis-Brandon I just enabled CLI arguments in apps and we can use -usbs usb2 - but it doesn't make UVC app work. Now, opened from PhotoBooth, I see the same artifacts I saw when testing on Jetson (and that's why this app is disabled on this platform). Gave it a try on Windows 10 now and didn't work with USB2 connection as well, so I disabled UVC app if it's detected (so only USB3 allowed) until we get this solved, PR here - https://github.com/luxonis/depthai/pull/618

CC @alex-luxonis on the screenshots below, I saw the same artifacts on Jetson, reproduceable using python3 depthai_demo.py --app uvc -usbs usb2 (but commenting out the SystemExit line)

[zegkljan]

I just want to quickly confirm that with a proper USB3 cable UVC works flawlessly. I don't think UVC not working over USB2 is necessarily a problem, it just should be advertised, or the UVC app should refuse to run with just USB2 (and explain why). But UVC over USB2 would certainly be nice.

[VanDavv]

Thanks @zegkljan for testing and happy it's resolved on your side. With #618 merged, the UVC app will exit gracefully with a message suggesting USB3 cable usage and informing that we're working on solving the issue, as you suggested - best what we can do until it's resolved

[Luxonis-Brandon]

Thanks, all! And agreed we should just properly make a warning that says USB2 is not supported via webcam mode as of yet.

[arafattehsin]

Hi folks! Since this is open and I am not super familiar with what is going on (or the background of it). I am trying to run it as a webcam. I can see that I am able tolaunch depthai_demo.py and when I click on this

It gives me this in console

But when I try to access the app through any Camera app (or even OBS), I get this

I am absolutely sure that my cable is USB C (with USB 3) and the port is also USB 3.

[Luxonis-Brandon]

Odd. So the system thinks it is connected to USB2 only here. To debug, when running the demo app gui could you do lsusb and share the output here?

[alex-luxonis]

Hi @arafattehsin, As it's a Windows machine, may be a little more complicated to check the USB speed in the system, but our demo should print it. Do you see UsbSpeed.HIGH (equivalent to USB2) as below when the demo runs in standard mode?

Using depthai module from:  C:\Users\user\AppData\Local\Programs\DepthAI\venv\lib\site-packages\depthai.cp39-win_amd64.pyd
Depthai version installed:  2.14.1.0.dev+27fa4519f289498e84768ab5229a1a45efb7e4df
Setting up demo...
Available devices:
[0] 18443010915E1DF500 [X_LINK_UNBOOTED]
USB Connection speed: UsbSpeed.HIGH
Enabling low-bandwidth mode due to low USB speed... (speed: UsbSpeed.HIGH)

If that's the case, what type-C cable are you using, are both ends type-C, or it's type-A to type-C? With the latter, there's a possibility the device could be enumerated in USB2 mode if type-A is plugged first (so it's powered) and then the type-C end is plugged into the device and it's done "slowly" (first the power pins inside the type-C making contact, then data pins). A power-cycle (cable reconnection) is needed to recover from this.

The console logs are fine for the first part, I'm observing the same on Windows, it doesn't advance to printing the status unless an UVC consumer (e.g. Camera app) is started.

[alex-luxonis]

Unrelated to the above, adding a general note for the differentiation between an USB2 and USB3 cable, as many type-C cables on the market are USB2-only. If the other end is type-A, the color usually should be blue, or it can be checked if a row of 5 extra pins exists deep inside the connector: (image source)

[arafattehsin]

@alex-luxonis and @Luxonis-Brandon - thank you so much for these details and very helpful responses.

I re-run again and saw this (I think I get your point here @alex-luxonis)

After noticing my cable, I think what I wrote is not correct and your explanation makes sense. The USB A connector is different than what you have posted here about USB 3.

Now, the question becomes, is it because of USB 3 cable that my OAK D Lite is not working as a webcam or is there anything else I have to do as well? Just curious. I am happy to get a new cable (with USB 3 for sure)

[VanDavv]

@arafattehsin replacing the cable with USB3 one should resolve this issue

[alex-luxonis]

We'll also fix the USB2 issue (hopefully within 1..2 weeks), but the performance with USB3 will be better for uncompressed streaming.

At the moment, the UVC configuration is 1920×1080 NV12 (1.5 bytes/pixel). (the camera is actually configured at THE_4_K resolution with 2x downscale - this gives sharper images than THE_1080_P with scaling/binning on-sensor)

The bandwidth required for 30fps is: 1920 * 1080 * 1.5 * 30 = around 94 MB/s. USB2 can't sustain this, so we'll probably lower the resolution to 720p with USB2. And further will look at enabling JPEG or H.264/H.265 (AVC/HEVC) encoding with UVC.

[arafattehsin]

@alex-luxonis - Please take your time but an option to choose the resolution (or launching it with the specific resolution option) will always be great, considering its 4K capabilities. 🙂

[arafattehsin]

@alex-luxonis @Luxonis-Brandon - folks! USB C (3.0) arrived and works great. Woohoo! I will open up an enhancement soon which I think would add some values to the Oak D Lite as a product offering.

[Luxonis-Brandon]

Awesome thanks for confirming! And sounds great!

[Suisse00]

Hi guys, So I'm the cheap guy running on a computer with only USB2 (and thus a USB2 cable) on Windows 10 (x64). I don't know if I'm the only one but your workaround to close (and prevent) using USB2 doesn't quite work here.

There is even something strange when looking at the console. Calling dai.Device(pipeline, usb2Mode=False) (even with usb2mode=True):

- Always end up blocking forever (so your warning/SystemExit(1) is never called)

• Log stop working. Sometimes, I have half of a line about temperature sensor opened before it stop, sometimes everything from temperature sensors (but still freezes), sometimes it ends with the timesync line, ...

This is just for your information.

Turn out, I was unlucky enough so part of my issue was my software wasn't detecting the UVC in the first place. I just figure out that your warning show up when you connect to the webcam (put a facepalm here)

[brmarkus]

Do you use an actively powered USB-hub? USB2 versus USB3 is also about how much power could be provided to the end device. Which OAK-device do you use? (there are OAK-devices allowing to add an external power-supply to)

[Suisse00]

(If needed we may create a new issue specifically for me, especially for the log part).

OAK-D-Lite.

I tried two ways to power it (because It was also one of my concerns) using the "Y" splitter (dedicated power port). The data line is always on that cheap USB-2 computer.

One using a USB 2 Type-A powered hub (Amazon Basic (yeah I know...)) using the dedicated "fast charge port" (1.5A) powered from an external power source. However, since OAK-D-Lite it is an USB-C I thought you may support USB PD only which lead me...

To test using a USB-C 3.1 on a always on port from a laptop. (I also tried on the Type-A USB 3.1 port)

In every case I didn't hear any USB disconnect from the host which could lead me to think the power going to the OAK-D-Lite was way too low. However, I know the OAK-D-Lite may get enough power to not disconnect from host but not enough to work properly :/ (If only wireshark USB could work on that damn Windows host!)

Warning: Keep in mind, when I use a Type-A USB port I'm using a Type-A to Type-C wire which is likely to not be USB 3 compliant!

[hg111]

Hey guys,

We recently (Dec. 2022) purchased an OAK-D PRO W and we are having the same issue of not being able to run the camera as a webcam on Linux aarch 64 Ubuntu 20.04.

The error when running v4l2-ctl --list-devices : Cannot open device /dev/video0, exiting.

and ls -lrt /dev/video returns: ls: cannot access '/dev/video*': No such file or directory

lsusb does not list the OAK either

The only port on the camera is a USB-C and the cable we use is fairly new USB-C cable which we plug into USB-3 Type-A (Blue) adaptor on the PC side. We tested and this cable works perfectly with a Webcam. is there something else we should try? a 'patch' or some other solution or workaround?

---

Note: perhaps related is the comment here: https://unix.stackexchange.com/questions/678358/logitech-c920e-v4l2-ctl-list-devices-cannot-open-device-dev-video0-exiti

"Since the driver does not seem to detect your camera, it's possible the camera presents itself to the USB bus in some non-standard way that will need to be handled specially by the uvcvideo driver maintainers."

and in the faq: https://www.ideasonboard.org/uvc/faq/#faq1

if our device is a uvc device. running the command: lsusb -d -v | grep "14 Video"

should see a number of lines like this:

bFunctionClass 14 Video bInterfaceClass 14 Video ...

so is it missing the correct uvc drive ?

[Erol444]

Hi @hg111 , Have ran the UVC pipeline? Docs here.

[hg111]

yes, running python3 depthai_demo.py --app uvc we get:

"This app is temporarily disabled on AARCH64 systems due to an issue with stream preview. We are working on resolving this issue"

On Tue, Jan 31, 2023 at 5:14 AM Erol444 @.***> wrote:

Hi @hg111 https://github.com/hg111 , Have ran the UVC pipeline? Docs here https://docs.luxonis.com/en/latest/pages/oak_webcam/.

— Reply to this email directly, view it on GitHub https://github.com/luxonis/depthai/issues/616#issuecomment-1410328297, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZE7VXYJSHLIYHGKF7UOMDWVEF4JANCNFSM5MKFVWKQ . You are receiving this because you were mentioned.Message ID: @.***>

[Erol444]

@hg111 that would explain the issue; perhaps open a feature request on depthai-core to support uvc on aarch64 systems as well. Thanks, Erik

[hg111]

Will do. Thanks!

On Wed, Feb 1, 2023 at 7:20 AM Erol444 @.***> wrote:

@hg111 https://github.com/hg111 that would explain the issue; perhaps open a feature request on depthai-core https://github.com/luxonis/depthai-core https://github.com/luxonis/[depthai-core](https://github.com/luxonis/depthai-core) to support uvc on aarch64 systems as well. Thanks, Erik

— Reply to this email directly, view it on GitHub https://github.com/luxonis/depthai/issues/616#issuecomment-1412237687, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZE7VRBEKZM6RVDYIUUCALWVJ5LJANCNFSM5MKFVWKQ . You are receiving this because you were mentioned.Message ID: @.***>