search

OpenKinect / libfreenect2

Open source drivers for the Kinect for Windows v2 device
2.08k stars 752 forks source link

RGB error - Linux #1137

Closed ranjithshegde closed 3 years ago

ranjithshegde commented 4 years ago

Overview Description:

Version, Platform, and Hardware Bug Found:

uname -a Linux StudioElectronica 5.8.14-zen1-1-zen #1 ZEN SMP PREEMPT Thu, 08 Oct 2020 00:01:25 +0000 x86_64 GNU/Linux

`lsusb -t` /:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 10000M
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 480M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/8p, 10000M
    |__ Port 1: Dev 6, If 0, Class=Hub, Driver=hub/1p, 5000M
        |__ Port 1: Dev 7, If 0, Class=Vendor Specific Class, Driver=, 5000M
        |__ Port 1: Dev 7, If 1, Class=Vendor Specific Class, Driver=, 5000M
        |__ Port 1: Dev 7, If 2, Class=Audio, Driver=snd-usb-audio, 5000M
        |__ Port 1: Dev 7, If 3, Class=Audio, Driver=snd-usb-audio, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/16p, 480M
    |__ Port 1: Dev 6, If 0, Class=Hub, Driver=hub/1p, 480M
    |__ Port 8: Dev 2, If 1, Class=Human Interface Device, Driver=usbhid, 12M
    |__ Port 8: Dev 2, If 2, Class=Human Interface Device, Driver=usbhid, 12M
    |__ Port 8: Dev 2, If 0, Class=Human Interface Device, Driver=usbhid, 12M
    |__ Port 14: Dev 3, If 0, Class=Wireless, Driver=, 12M
    |__ Port 14: Dev 3, If 1, Class=Wireless, Driver=, 12M
  1. lspci -nn : 00:00.0 Host bridge [0600]: Intel Corporation 8th Gen Core Processor Host Bridge/DRAM Registers [8086:3ec4] (rev 0d) 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 0d) 00:02.0 VGA compatible controller [0300]: Intel Corporation UHD Graphics 630 (Mobile) [8086:3e9b] (rev 02) 00:04.0 Signal processing controller [1180]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Thermal Subsystem [8086:1903] (rev 0d) 00:08.0 System peripheral [0880]: Intel Corporation Xeon E3-1200 v5/v6 / E3-1500 v5 / 6th/7th/8th Gen Core Processor Gaussian Mixture Model [8086:1911] 00:12.0 Signal processing controller [1180]: Intel Corporation Cannon Lake PCH Thermal Controller [8086:a379] (rev 10) 00:14.0 USB controller [0c03]: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller [8086:a36d] (rev 10) 00:14.2 RAM memory [0500]: Intel Corporation Cannon Lake PCH Shared SRAM [8086:a36f] (rev 10) 00:14.3 Network controller [0280]: Intel Corporation Wireless-AC 9560 [Jefferson Peak] [8086:a370] (rev 10) 00:15.0 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH Serial IO I2C Controller #0 [8086:a368] (rev 10) 00:15.1 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH Serial IO I2C Controller #1 [8086:a369] (rev 10) 00:16.0 Communication controller [0780]: Intel Corporation Cannon Lake PCH HECI Controller [8086:a360] (rev 10) 00:17.0 SATA controller [0106]: Intel Corporation Cannon Lake Mobile PCH SATA AHCI Controller [8086:a353] (rev 10) 00:1d.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #9 [8086:a330] (rev f0) 00:1f.0 ISA bridge [0601]: Intel Corporation HM470 Chipset LPC/eSPI Controller [8086:a30d] (rev 10) 00:1f.3 Audio device [0403]: Intel Corporation Cannon Lake PCH cAVS [8086:a348] (rev 10) 00:1f.4 SMBus [0c05]: Intel Corporation Cannon Lake PCH SMBus Controller [8086:a323] (rev 10) 00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH SPI Controller [8086:a324] (rev 10) 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU106M [GeForce RTX 2070 Mobile] [10de:1f10] (rev a1) 01:00.1 Audio device [0403]: NVIDIA Corporation TU106 High Definition Audio Controller [10de:10f9] (rev a1) 01:00.2 USB controller [0c03]: NVIDIA Corporation TU106 USB 3.1 Host Controller [10de:1ada] (rev a1) 01:00.3 Serial bus controller [0c80]: NVIDIA Corporation TU106 USB Type-C UCSI Controller [10de:1adb] (rev a1) 02:00.0 Non-Volatile memory controller [0108]: Intel Corporation SSD 660P Series [8086:f1a8] (rev 03)

Steps to Reproduce:

  1. Protonect

Actual Results:

00:00.0 Host bridge [0600]: Intel Corporation 8th Gen Core Processor Host Bridge/DRAM Registers [8086:3ec4] (rev 0d)
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 0d)
00:02.0 VGA compatible controller [0300]: Intel Corporation UHD Graphics 630 (Mobile) [8086:3e9b] (rev 02)
00:04.0 Signal processing controller [1180]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Thermal Subsystem [8086:1903] (rev 0d)
00:08.0 System peripheral [0880]: Intel Corporation Xeon E3-1200 v5/v6 / E3-1500 v5 / 6th/7th/8th Gen Core Processor Gaussian Mixture Model [8086:1911]
00:12.0 Signal processing controller [1180]: Intel Corporation Cannon Lake PCH Thermal Controller [8086:a379] (rev 10)
00:14.0 USB controller [0c03]: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller [8086:a36d] (rev 10)
00:14.2 RAM memory [0500]: Intel Corporation Cannon Lake PCH Shared SRAM [8086:a36f] (rev 10)
00:14.3 Network controller [0280]: Intel Corporation Wireless-AC 9560 [Jefferson Peak] [8086:a370] (rev 10)
00:15.0 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH Serial IO I2C Controller #0 [8086:a368] (rev 10)
00:15.1 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH Serial IO I2C Controller #1 [8086:a369] (rev 10)
00:16.0 Communication controller [0780]: Intel Corporation Cannon Lake PCH HECI Controller [8086:a360] (rev 10)
00:17.0 SATA controller [0106]: Intel Corporation Cannon Lake Mobile PCH SATA AHCI Controller [8086:a353] (rev 10)
00:1d.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #9 [8086:a330] (rev f0)
00:1f.0 ISA bridge [0601]: Intel Corporation HM470 Chipset LPC/eSPI Controller [8086:a30d] (rev 10)
00:1f.3 Audio device [0403]: Intel Corporation Cannon Lake PCH cAVS [8086:a348] (rev 10)
00:1f.4 SMBus [0c05]: Intel Corporation Cannon Lake PCH SMBus Controller [8086:a323] (rev 10)
00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH SPI Controller [8086:a324] (rev 10)
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU106M [GeForce RTX 2070 Mobile] [10de:1f10] (rev a1)
01:00.1 Audio device [0403]: NVIDIA Corporation TU106 High Definition Audio Controller [10de:10f9] (rev a1)
01:00.2 USB controller [0c03]: NVIDIA Corporation TU106 USB 3.1 Host Controller [10de:1ada]
(rev a1)
01:00.3 Serial bus controller [0c80]: NVIDIA Corporation TU106 USB Type-C UCSI Controller [10de:1adb] (rev a1)
02:00.0 Non-Volatile memory controller [0108]: Intel Corporation SSD 660P Series [8086:f1a8] (rev 03)

screen

Expected Results: Clean image

Additional information: No such errors when using SDK on windows - meaning its not the fault of the device.

Also, same error regardless of gl, cl cuda, cpu

Also, same error with our without building with Vaapi enabled/disabled

floe commented 4 years ago

Hmm, the JPEG decode for the RGB images is (optionally) handled by VAAPI, so that might be part of the issue here.

ranjithshegde commented 4 years ago

Thanks for your reply. I tried building with Vaapi disabled. same story.

floe commented 4 years ago

Is there any chance you could test with a different USB controller? I'm running out of ideas... :-/

ranjithshegde commented 4 years ago

I am not sure what you mean on a different USB controller. I tested it on two other linux machines, there was indeed vaapi error on all of them (only ran with --norgb) but after disabling vaapi it ran fine on them

I also tested the kinect 1 on the same laptop (linux) as the above mentioned issue and no errors.

floe commented 4 years ago

OK, if it works on other Linux machines for you, then the USB controller might actually be the one to blame. The Kinect v2 is notoriously picky about which USB hosts it works with properly.

ranjithshegde commented 4 years ago

Thank you for your response. Thats kind of sad. I have an optimus laptop (hybrid graphics - intel + nvidia), and lots of rules to prevent the dedicated card from powering on unless intended, could they be part of the problem in any way? Because lspci lists both intel and nvidia having a usb3.1 and xHCI controllers

floe commented 4 years ago

Sorry, don't have experience with Optimus...

ranjithshegde commented 4 years ago

Thank you. I am still wondering about the USB controller error. Since it works on windows on the same laptop, could it still be the USB (in which case the SDK somehow bypasses the USB error) or something else?

floe commented 4 years ago

That's still possible, perhaps the Windows driver for the host controller configures something differently than the Linux driver.

jfm535 commented 3 years ago

something i can add is it could be an intel driver issue try the suggestion i found in this issue https://github.com/OpenKinect/libfreenect2/issues/1026#issuecomment-428365774 specificly export LIBVA_DRIVER_NAME=i965

I dont have the same hardware as the poster as seen here 

00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v6/7th Gen Core Processor Host Bridge/DRAM Registers [8086:5914] (rev 08)
00:02.0 VGA compatible controller [0300]: Intel Corporation UHD Graphics 620 [8086:5917] (rev 07)
00:04.0 Signal processing controller [1180]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Thermal Subsystem [8086:1903] (rev 08)
00:08.0 System peripheral [0880]: Intel Corporation Xeon E3-1200 v5/v6 / E3-1500 v5 / 6th/7th/8th Gen Core Processor Gaussian Mixture Model [8086:1911]
00:14.0 USB controller [0c03]: Intel Corporation Sunrise Point-LP USB 3.0 xHCI Controller [8086:9d2f] (rev 21)
00:14.2 Signal processing controller [1180]: Intel Corporation Sunrise Point-LP Thermal subsystem [8086:9d31] (rev 21)
00:16.0 Communication controller [0780]: Intel Corporation Sunrise Point-LP CSME HECI #1 [8086:9d3a] (rev 21)
00:1c.0 PCI bridge [0604]: Intel Corporation Sunrise Point-LP PCI Express Root Port #1 [8086:9d10] (rev f1)
00:1c.6 PCI bridge [0604]: Intel Corporation Sunrise Point-LP PCI Express Root Port #7 [8086:9d16] (rev f1)
00:1d.0 PCI bridge [0604]: Intel Corporation Sunrise Point-LP PCI Express Root Port #9 [8086:9d18] (rev f1)
00:1d.2 PCI bridge [0604]: Intel Corporation Sunrise Point-LP PCI Express Root Port #11 [8086:9d1a] (rev f1)
00:1f.0 ISA bridge [0601]: Intel Corporation Sunrise Point LPC Controller/eSPI Controller [8086:9d4e] (rev 21)
00:1f.2 Memory controller [0580]: Intel Corporation Sunrise Point-LP PMC [8086:9d21] (rev 21)
00:1f.3 Audio device [0403]: Intel Corporation Sunrise Point-LP HD Audio [8086:9d71] (rev 21)
00:1f.4 SMBus [0c05]: Intel Corporation Sunrise Point-LP SMBus [8086:9d23] (rev 21)
00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (4) I219-LM [8086:15d7] (rev 21)
03:00.0 Network controller [0280]: Intel Corporation Wireless 8265 / 8275 [8086:24fd] (rev 78)
04:00.0 PCI bridge [0604]: Intel Corporation JHL6240 Thunderbolt 3 Bridge (Low Power) [Alpine Ridge LP 2016] [8086:15c0] (rev 01)
05:00.0 PCI bridge [0604]: Intel Corporation JHL6240 Thunderbolt 3 Bridge (Low Power) [Alpine Ridge LP 2016] [8086:15c0] (rev 01)
05:01.0 PCI bridge [0604]: Intel Corporation JHL6240 Thunderbolt 3 Bridge (Low Power) [Alpine Ridge LP 2016] [8086:15c0] (rev 01)
05:02.0 PCI bridge [0604]: Intel Corporation JHL6240 Thunderbolt 3 Bridge (Low Power) [Alpine Ridge LP 2016] [8086:15c0] (rev 01)
06:00.0 System peripheral [0880]: Intel Corporation JHL6240 Thunderbolt 3 NHI (Low Power) [Alpine Ridge LP 2016] [8086:15bf] (rev 01)
3c:00.0 USB controller [0c03]: Intel Corporation JHL6240 Thunderbolt 3 USB 3.1 Controller (Low Power) [Alpine Ridge LP 2016] [8086:15c1] (rev 01)
3d:00.0 Non-Volatile memory controller [0108]: Sandisk Corp WD Black 2018/PC SN720 NVMe SSD [15b7:5002]

but since they and I both run intel it might work.

ranjithshegde commented 3 years ago

something i can add is it could be an intel driver issue try the suggestion i found in this issue #1026 (comment) specificly export LIBVA_DRIVER_NAME=i965

You are a lifesaver! I tested with LIBVA_DRIVER_NAME=/usr/lib/dri/i965 (my actual driver is iHD) and it worked like a charm! Thank you so much

ranjithshegde commented 3 years ago

Just a quick question (not directly related to libfreenect2), how do I get ALL calls to prepend LIBVA_DRIVER_NAME? I can get my zsh to do so by adding in rc file. But if I am using openFrameworks through vim, or basically any package that calls this, how do I get them to use this workaround?

jfm535 commented 3 years ago

Well I am not sure how you are calling it but if you dont want to use the export method you can also try something like this LIBVA_DRIVER_NAME=i965 ./bin/Protonect

I will look into seeing how to set this via c++ or the like

jfm535 commented 3 years ago

so far the env variable is handled here in libva https://github.com/intel/libva/blob/cd543745a6bc20d4b69000d36d26683ee211afb5/va/va.c and libfreenect2 calls the intialization code here https://github.com/OpenKinect/libfreenect2/blob/master/src/vaapi_rgb_packet_processor.cpp#L276

jfm535 commented 3 years ago

yes you can tho it may require modifying libfreenect2 and inserting a call to VAStatus vaSetDriverName(VADisplay dpy,char *driver_name) along the lines of vaSetDriverName(display,"i965"); here https://github.com/OpenKinect/libfreenect2/blob/master/src/vaapi_rgb_packet_processor.cpp#L273

Also can confirm adding that line to the file does force the driver.

ranjithshegde commented 3 years ago

Thank you so much! just compiled it and it works great! Really appreciate your help

leahcornelius commented 1 year ago

Hi, i am also experiencing this issue. If I use i965 as my vaapi driver then it fails to initalise (I'm on a core I3 which doesn't support i965) which does "solve" the issue as vaapii is no longer used for the RGB decoding so output is normal but its not really a fix as i would quite like to use vaapii and figure out why this weird green artifact is caused?