Avoiding unplug-replug on Jetson TK1

[computer-whisperer]

Hi,

I am using libfreenect2 to drive a kinect on an FRC competition robot with a Jetson TK1. Both the Jetson and the Kinect are hard wired to the main power supply of the robot, and start simultaneously when the robot power is switched on. The problem I am currently struggling with is how to run libfreenect2 without performing the obligatory unplug-replug routine with the Kinect.

I had it working once. Something I did when initially configuring the Jetson caused me to never encounter this problem. I don't know what exactly I did, since I was more or less just messing around trying to get libfreenect2 working. Unfortunately, due to an accident involving rm, I was forced to re-flash the operating system. Since then, I have not been able to fix the problem.

I completely understand if nobody has a solution to this, as your wiki explicitly states If the Kinect is plugged in during boot, it may not work. Reconnecting or plugging in Kinect2 after the system boots. If anyone would happen to have an inkling of how to fix this, help would be very much appreciated.

Thanks!

Version, Platform, and Hardware Bug Found:

1. git log -1 --oneline

   619bd52 depends: Do not download libva debs for non-x86

2. uname -a

   Linux tegra-ubuntu 3.10.40-gdacac96 #1 SMP PREEMPT Thu Jun 25 15:25:11 PDT 2015 armv7l armv7l armv7l GNU/Linux

3. lsusb -t

   /: Bus 03.Port 1: Dev 1, Class=root_hub, Driver=tegra-ehci/1p, 480M /: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=tegra-xhci/2p, 5000M | Port 1: Dev 2, If 0, Class=Hub, Driver=hub/1p, 5000M | Port 1: Dev 3, If 0, Class=Vendor Specific Class, Driver=usbfs, 5000M | Port 1: Dev 3, If 1, Class=Vendor Specific Class, Driver=usbfs, 5000M | Port 1: Dev 3, If 2, Class=Audio, Driver=snd-usb-audio, 5000M |__ Port 1: Dev 3, If 3, Class=Audio, Driver=snd-usb-audio, 5000M /: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=tegra-xhci/6p, 480M |__ Port 3: Dev 2, If 0, Class=Hub, Driver=hub/1p, 480M

4. lspci -nn

   00:00.0 PCI bridge [0604]: NVIDIA Corporation TegraK1 PCIe x1 Bridge [10de:0e13](rev a1) 01:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168](rev 0c)

Steps to Reproduce:

1. Power on Jetson TK1 and Kinect
2. Reboot Jetson TK1
3. Run ./Protonect

Actual Results: No command-line output after device serial: 033903335147 device firmware: 4.0.3912.0

Expected Results: Healthy heartbeat messages along the lines of: [Info] [DepthPacketStreamParser] 4 packets were lost

Reproducibility: Every time

EDIT: Sorry, I forgot to include these: dmesg

[   14.187420] usb 2-1.1: new SuperSpeed USB device number 3 using tegra-xhci
[   14.198789] usb 2-1.1: Parent hub missing LPM exit latency info.  Power management will be impacted.
[   14.199979] usb 2-1.1: New USB device found, idVendor=045e, idProduct=02c4
[   14.199988] usb 2-1.1: New USB device strings: Mfr=1, Product=2, SerialNumber=4
[   14.199994] usb 2-1.1: Product: Xbox NUI Sensor
[   14.200000] usb 2-1.1: Manufacturer: Microsoft
[   14.200006] usb 2-1.1: SerialNumber: 033903335147
[   31.253218] init: failsafe main process (611) killed by TERM signal
[   32.199617] init: plymouth-upstart-bridge main process ended, respawning
[   35.497732] HDMI: No HDMI device connected
[  166.407416] usb 2-1.1: reset SuperSpeed USB device number 3 using tegra-xhci
[  166.419699] usb 2-1.1: Parent hub missing LPM exit latency info.  Power management will be impacted.
[  166.535179] tegra-xhci tegra-xhci: WARN Event TRB for slot 3 ep 2 with no TDs queued?
[  166.930941] tegra-xhci tegra-xhci: WARN Event TRB for slot 3 ep 2 with no TDs queued?
[  166.963353] tegra-xhci tegra-xhci: WARN Event TRB for slot 3 ep 2 with no TDs queued?
[  167.745728] mc-err: [mcerr] (vde) csr_vdebsevr: EMEM decode error on PDE or PTE entry
[  167.753558] mc-err: [mcerr]   status = 0x60000022; addr = 0x80ca7000
[  167.759728] mc-err: [mcerr]   secure: no, access-type: read, SMMU fault: nr-nw-s

LIBUSB_DEBUG=3 ./Protonect

Version: 0.2.0
Environment variables: LOGFILE=<protonect.log>
Usage: ./Protonect [-gpu=<id>] [gl | cl | cuda | cpu] [<device serial>]
        [-noviewer] [-norgb | -nodepth] [-help] [-version]
        [-frames <number of frames to process>]
To pause and unpause: pkill -USR1 Protonect
[Info] [Freenect2Impl] enumerating devices...
[Info] [Freenect2Impl] 6 usb devices connected
[Info] [Freenect2Impl] found valid Kinect v2 @2:3 with serial 033903335147
[Info] [Freenect2Impl] found 1 devices
[Info] [CudaDepthPacketProcessorImpl] device 0: GK20A @ 852MHz Memory 1892MB
[Info] [CudaDepthPacketProcessorImpl] selected device 0
[Info] [Freenect2DeviceImpl] opening...
[Info] [Freenect2DeviceImpl] transfer pool sizes rgb: 20*16384 ir: 60*8*33792
[Info] [Freenect2DeviceImpl] opened
[Info] [Freenect2DeviceImpl] starting...
[Debug] [Freenect2DeviceImpl] status 0x090000: 9729
[Debug] [Freenect2DeviceImpl] status 0x090000: 9731
[Info] [Freenect2DeviceImpl] submitting rgb transfers...
[Info] [Freenect2DeviceImpl] submitting depth transfers...
[Info] [Freenect2DeviceImpl] started
device serial: 033903335147
device firmware: 4.0.3912.0
[Debug] [RgbPacketStreamParser] skipping rgb packet!
[Debug] [RgbPacketStreamParser] skipping rgb packet!
[Debug] [RgbPacketStreamParser] skipping rgb packet!
[Debug] [RgbPacketStreamParser] skipping rgb packet!
[Debug] [RgbPacketStreamParser] skipping rgb packet!
[Debug] [RgbPacketStreamParser] skipping rgb packet!

[hanyazou]

I don't know if this works or not because I can't reproduce the problem. I don't have TX1. But sending shutdown command might recover the sensor.

To send the shutdown command to the sensor, please delete #ifdef APPLE and #endif in libfreenect2.cpp and enable C++11 extension by cmake -DENABLE_CXX11=ON .. (C++11 extension is required for 238c44868473e87612f1bdcde24e99ef87e68fd2)

diff --git a/src/libfreenect2.cpp b/src/libfreenect2.cpp index a7eaf10..4ce7bb4 100644 --- a/src/libfreenect2.cpp +++ b/src/libfreenect2.cpp @@ -858,7 +858,7 @@ bool Freenect2DeviceImpl::close() 　/* This command actually reboots the device and makes it disappear for 3 seconds. 　* Protonect can restart instantly without it. 　/ -#ifdef APPLE 　/* Kinect will disappear on Mac OS X regardless during close(). 　* Painstaking effort could not determine the root cause. 　* See https://github.com/OpenKinect/libfreenect2/issues/539 @@ -867,7 +867,7 @@ bool Freenect2DeviceImpl::close() 　/ 　commandtx.execute(ShutdownCommand(nextCommandSeq()), result); 　libfreenect2::this_thread::sleep_for(libfreenect2::chrono::milliseconds(4*1000)); -#endif

[computer-whisperer]

Ah, now we find another problem. Adding -DENABLE_CXX11=ON results in this error on build, whether or not I make the source changes.

In file included from /home/ubuntu/libfreenect2/include/internal/libfreenect2/usb/transfer_pool.h:34:0,
                 from /home/ubuntu/libfreenect2/src/transfer_pool.cpp:29:
/home/ubuntu/libfreenect2/include/internal/libfreenect2/threading.h:68:24: fatal error: tinythread.h: No such file or directory
 #include <tinythread.h>
                        ^
compilation terminated.

[hanyazou]

Please re-run cmake and provide cmake output.

$ rm -rf CMakeCache.txt CMakeFiles $ cmake -DENABLE_CXX11=ON ..

[computer-whisperer]

Output of 'cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/freenect2 -DENABLE_CXX11=ON' https://gist.github.com/computer-whisperer/2dc6ff28d2045cb1544f

Thanks!

[hanyazou]

Do you still have the problem? Is LIBFREENECT2_THREADING_TINYTHREAD defined in libfreenect2/build/libfreenect2/config.h ? LIBFREENECT2_THREADING_TINYTHREAD should be undefined in the config.h if cmake and config process works properly.

[computer-whisperer]

It appears that the undefine command is commented out.

Here is the relevant section:

#define LIBFREENECT2_THREADING_STDLIB

/* #undef LIBFREENECT2_THREADING_TINYTHREAD */

#define LIBFREENECT2_WITH_CXX11_SUPPORT

I tried un-commenting the line, but the change was reverted when I ran make

[hanyazou]

Sorry, I have no idea about the tinythread problem. Here is my build output on TK1.

typescript.txt

[computer-whisperer]

@hanyazou thanks for trying. I will have to poke around a bit more to figure this out.

[computer-whisperer]

I was able to get it to build with -DENABLE_CXX11=ON after some hacks to threading.h. @hanyazou I made the changes to libfreenect.cpp that you suggested, but it did not fix the original problem.

[dujiulun]

Hi @computer-whisperer , did you have any luck solving this problem? It also bothers me, and I have tried many software hacks (recompiling the kernel, resetting the USB device, unbinding the USB device) but none worked. Any help would be appreciated.

[renatojmsdh]

I was able to get it to build with -DENABLE_CXX11=ON after some hacks to threading.h.

Hi @computer-whisperer,I'm having the same issue when using c++11 compatibility...could you share the "hacks" you did to threading.h for compiling it?