Don't move Kinect Bridge

[bingyo]

I am referring to Ros2Bot. Thank you very much.

I was a setup using the procedure. However, kinect2_bridge did not work.

$ rosrun kinect2_bridge kinect2_bridge
・・・
・・・
[kinect2_bridge] depth processing: ~nanHz (nanms) publishing rate: ~0Hz
[kinect2_bridge] color processing: ~301.53Hz (3.31642ms) publishing rate: ~7.65098Hz
・・・
・・・

I was investigating the cause. kinect2_bridge did not work but Protonect can work. So I entered the following command.

$ ldd /catkin_ws/devel/lib/kinect2_bridge | grep usb
libusb-1.0.so.0 => /home/ubuntu/Downloads/old_libfreenect2/libfreenect2/depends/libusb/lib/libusb-1.0.so.0 (0xb5185000)

$ ldd  ~/libfreenect2/examples/protonect/bin/Protonect | grep usb
libusb-1.0.so.0 => /home/ubuntu/libfreenect2/examples/protonect/../../depends/libusb/lib/libusb-1.0.so.0 (0xb6a51000)

Please tell me if you know the cause.

[Soy-Robotics]

hi bingyo. I noticed that color processing is working fine for you but not depth. Presently Nvidia with kinect2 have some usb issues. You can overcome this issue , When ever you are going to turn on the system please remove the usb hub(which kinect2 is connected). once turned ON, then connect the usb hub. Run kinect2_bridge, depth processing will work fine. Further issues please let me know.

Note: within couple of days will upload ros2bot description files and other packages too.

[bingyo]

Thank you for your reply. Sorry, I don't know how to remove the usb hub. Please advise remove command(?).

I am looking forward to new ros2bot description.

The following, for reference information.

Kinect V2 connect to Mini PCI-Express Card(usb3.0).

I enter commanded

$ rosrun kinect2_bridge kinect2_bridge

next, another terminal windows

$ rosrun kinect2_viewer kinect2_viewer kinect2 sd cloud

and,"kinect2_bridge " terminal windows

$ rosrun kinect2_bridge kinect2_bridge
・・・
sensor: default
        fps_limit: -1
       calib_path: /home/ubuntu/catkin_ws/src/iai_kinect2/kinect2_bridge/data/
          use_png: false
     jpeg_quality: 90
        png_level: 1
     depth_method: opengl
     depth_device: -1
       reg_method: default
       reg_devive: -1
        max_depth: 12
        min_depth: 0.1
       queue_size: 2
 bilateral_filter: true
edge_aware_filter: true
       publish_tf: false
     base_name_tf: kinect2
   worker_threads: 4
[Info] [Freenect2Impl] enumerating devices...
[Info] [Freenect2Impl] 6 usb devices connected
[Info] [Freenect2Impl] found valid Kinect v2 @2:2 with serial 003516652747
[Info] [Freenect2Impl] found 1 devices
[ INFO] [Kinect2Bridge::initDevice] Kinect2 devices found:
[ INFO] [Kinect2Bridge::initDevice]   0: 003516652747 (selected)
[Info] [Freenect2DeviceImpl] opening...
[Info] [Freenect2DeviceImpl] opened
[ INFO] [Kinect2Bridge::initDevice] starting kinect2
[Info] [Freenect2DeviceImpl] starting...
[Info] [Freenect2DeviceImpl] enabling usb transfer submission...
[Info] [Freenect2DeviceImpl] submitting usb transfers...
[Info] [Freenect2DeviceImpl] started
[ INFO] [Kinect2Bridge::initDevice] device serial: 003516652747
[ INFO] [Kinect2Bridge::initDevice] device firmware: 4.0.3911.0
[Info] [Freenect2DeviceImpl] stopping...
[Info] [Freenect2DeviceImpl] disabling usb transfer submission...
[Info] [Freenect2DeviceImpl] canceling usb transfers...
[Info] [Freenect2DeviceImpl] stopped
[ WARN] [Kinect2Bridge::initCalibration] using sensor defaults for color intrinsic parameters.
[ WARN] [Kinect2Bridge::initCalibration] using sensor defaults for ir intrinsic parameters.
[ WARN] [Kinect2Bridge::initCalibration] using defaults for rotation and translation.
[ WARN] [Kinect2Bridge::initCalibration] using defaults for depth shift.
[ INFO] [DepthRegistration::New] Using CPU registration method!
[ INFO] [DepthRegistration::New] Using CPU registration method!
[ INFO] [Kinect2Bridge::main] waiting for clients to connect
[ INFO] [Kinect2Bridge::callbackStatus] client connected. starting device...
[Info] [Freenect2DeviceImpl] starting...
[Info] [Freenect2DeviceImpl] enabling usb transfer submission...
[Info] [Freenect2DeviceImpl] submitting usb transfers...
[ INFO] [Kinect2Bridge::main] depth processing: ~infms (~0Hz) publishing rate: ~0Hz
[ INFO] [Kinect2Bridge::main] color processing: ~infms (~0Hz) publishing rate: ~0Hz
[Info] [Freenect2DeviceImpl] started
[Error] [RgbPacketStreamParser] packetsize or sequence doesn't match!
[ INFO] [Kinect2Bridge::main] depth processing: ~nanms (~nanHz) publishing rate: ~0Hz
[ INFO] [Kinect2Bridge::main] color processing: ~nanms (~nanHz) publishing rate: ~0Hz
[ INFO] [Kinect2Bridge::main] depth processing: ~nanms (~nanHz) publishing rate: ~0Hz
[ INFO] [Kinect2Bridge::main] color processing: ~1.40524ms (~711.623Hz) publishing rate: ~0.328425Hz
・・・
[ INFO] [Kinect2Bridge::main] depth processing: ~nanms (~nanHz) publishing rate: ~0Hz
[ INFO] [Kinect2Bridge::main] color processing: ~5.73227ms (~174.451Hz) publishing rate: ~16.3235Hz
[Info] [TurboJpegRgbPacketProcessor] avg. time: 42.2135ms -> ~23.6891Hz
[ INFO] [Kinect2Bridge::main] depth processing: ~nanms (~nanHz) publishing rate: ~0Hz
[ INFO] [Kinect2Bridge::main] color processing: ~6.06281ms (~164.94Hz) publishing rate: ~15.6599Hz
[ INFO] [Kinect2Bridge::main] depth processing: ~nanms (~nanHz) publishing rate: ~0Hz
[ INFO] [Kinect2Bridge::main] color processing: ~6.92659ms (~144.371Hz) publishing rate: ~15.9858Hz
[Info] [TurboJpegRgbPacketProcessor] avg. time: 43.5181ms -> ~22.979Hz
[Info] [TurboJpegRgbPacketProcessor] avg. time: 44.8926ms -> ~22.2754Hz
・・・
[Info] [TurboJpegRgbPacketProcessor] avg. time: 47.5788ms -> ~21.0177H

Don't work Kinect Bridge and very slow GUI.

[Soy-Robotics]

Hello @bingyo You have to remove kinect2 usb cable manually (Not using command line). As you are using pcle card instead of usb hub, you can directly remove kinect2 usb cable. Unplug kinect2 usb cable, restart the jetson board. Once restarted, connect kinect2 usb cable. For better performance, you can improve the CPU and GPU efficiency(I will send commands for that).

[bingyo]

Hello @GaiTech-Robotics

Oh, I understood.

Try remove kinect2 usb cable when this problem. And restart the jetson board. I retry kinect_bridge. But it wasn't work.

This is the terminal when remove kinect2 usb cable.

ubuntu@tegra-ubuntu:~$ rosrun kinect2_bridge kinect2_bridge
[ INFO] [Kinect2Bridge::initialize] parameter:

・・・

[ INFO] [Kinect2Bridge::main] depth processing: ~nanms (~nanHz) publishing rate: ~0Hz
[ INFO] [Kinect2Bridge::main] color processing: ~6.0492ms (~165.311Hz) publishing rate: ~15.9502Hz
[Info] [TurboJpegRgbPacketProcessor] avg. time: 41.4431ms -> ~24.1295Hz
[Error] [usb::TransferPool] failed to submit transfer: LIBUSB_ERROR_NO_DEVICE No such device (it may have been disconnected)
[Error] [usb::TransferPool] failed to submit transfer: LIBUSB_ERROR_NO_DEVICE No such device (it may have been disconnected)

・・・

[ INFO] [Kinect2Bridge::main] depth processing: ~nanms (~nanHz) publishing rate: ~0Hz
[ INFO] [Kinect2Bridge::main] color processing: ~6.39934ms (~156.266Hz) publishing rate: ~10.9952Hz
[ INFO] [Kinect2Bridge::main] depth processing: ~nanms (~nanHz) publishing rate: ~0Hz
[ INFO] [Kinect2Bridge::main] color processing: ~nanms (~nanHz) publishing rate: ~0Hz

[Soy-Robotics]

Hello Bingyo... please follow these steps...

1. remove the kinect2 usb cable
2. turn ON the jetson board.
3. Once the jetson board is completely ON(You will see ubuntu desktop), Now plug the kinect2 usb cable.
4. Now open the terminal $ roslaunch kinect2_bridge kinect2_bridge You can see in kinect2_bridge for "waiting for the clients to connect"
5. Open another terminal and type $ rosrun kinect2_viewer kinect2_viewer you can see color , depth and ir images

[bingyo]

Hello @GaiTech-Robotics

Thank you for steps. I did the same steps. But I can't see color and ir image display. The figure below is my jetson.

[Soy-Robotics]

@bingyo

Kinect2 works only with USB 3.0. you must connect kinect2 usb cable to the USB 3.0 port of jetson board.

Here in green circle is the USB port that you need to connect. And follow the same steps which i have provided to you

[bingyo]

@GaiTech-Robotics

USB 3.0 port of jetson board don't support Kinect2. The Jetson does not use Renesas for it’s USB 3.0 chip. http://jetsonhacks.com/2015/02/23/kinect-v2-with-libfreenect2/

I try kinect_bridge.

no Kinect2 deices found.

So I use Mini PCI-Express Card with USB 3.0 Cable. Could you use USB 3.0 port of jetson board?

[Soy-Robotics]

I am using usb 3.0 port. Have you changed " usb-port owner inft=0" to usb-port owner info=2"??

$ sudo -s

$ sudo gedit /boot/extlinux/extlinux.conf

Now change "usb_port_owner_info=0" to "usb_port_owner_info=2" save it

Reboot the system and follow as i have mentioned the steps above.

[bingyo]

Yes, I have. extlinux.conf is " usb-port owner info=2" now.

[Soy-Robotics]

@bingyo , Are you able to see depth image now??

[bingyo]

I can't see depth and color image now.

I connect kinect2 usb cable to ・USB 3.0 port of jetson board

no Kinect2 deices found.

・Mini PCI-Express Card(usb3.0).

[ INFO] [Kinect2Bridge::main] depth processing: ~nanms (~nanHz) publishing rate: ~0Hz
[ INFO] [Kinect2Bridge::main] color processing: ~6.0492ms (~165.311Hz) publishing rate: ~15.9502Hz

There is something that I would like you to confirm. Please enter following command and let me know the result of the command.

$ ldd /catkin_ws/devel/lib/kinect2_bridge | grep usb
libusb-1.0.so.0 => /home/ubuntu/Downloads/old_libfreenect2/libfreenect2/depends/libusb/lib/libusb-1.0.so.0 (0xb5185000)

$ ldd  ~/libfreenect2/examples/protonect/bin/Protonect | grep usb
libusb-1.0.so.0 => /home/ubuntu/libfreenect2/examples/protonect/../../depends/libusb/lib/libusb-1.0.so.0 (0xb6a51000)

[Soy-Robotics]

@bingyo libusb of ldd kinect2_bridge and ldd /Downloads/old_libfreenect should be same

Please run ./Protonect and provide the output
cd ~/Downloads/libfreenect2/build/bin sudo ./Protonect

Check this..

[bingyo]

I tried "sudo ./Protonect". But Kinect did not work and very slow to respond.

[Info] [Freenect2Impl] enumerating devices...
[Info] [Freenect2Impl] 7 usb devices connected
[Info] [Freenect2Impl] found valid Kinect v2 @2:2 with serial 003516652747
[Info] [Freenect2Impl] found 1 devices
[Info] [Freenect2DeviceImpl] opening...
[Info] [Freenect2DeviceImpl] opened
[Info] [Freenect2DeviceImpl] starting...
[Debug] [Freenect2DeviceImpl] status 0x090000: 9729
[Debug] [Freenect2DeviceImpl] status 0x090000: 9731
[Info] [Freenect2DeviceImpl] enabling usb transfer submission...
[Info] [Freenect2DeviceImpl] submitting usb transfers...
[Info] [Freenect2DeviceImpl] started
device serial: 003516652747
device firmware: 4.0.3911.0
[Debug] [RgbPacketStreamParser] skipping rgb packet!
[Debug] [DepthPacketStreamParser] not all subsequences received 0
[Debug] [DepthPacketStreamParser] subpacket too large
[Debug] [RgbPacketStreamParser] skipping rgb packet!
[Debug] [RgbPacketStreamParser] skipping rgb packet!
[Debug] [DepthPacketStreamParser] image data too short!
・・・   (and loop)

Next check.

./libfreenect2/examples/protonect/bin/Protonect 

I can see depth and RGB image

Both I used Mini PCI-Express Card.

[Soy-Robotics]

may i know ldd output from kinect2_bridge and /Downloads/libfreenect2/build/bin/Protonect??

[bingyo]

I paste output.

ldd /Downloads/libfreenect2/build/bin/Protonect

libfreenect2.so => /home/ubuntu/Downloads/libfreenect2/build/lib/libfreenect2.so (0xb6f71000)
libglfw.so.3 => /usr/local/lib/libglfw.so.3 (0xb6f5c000)
libGL.so.1 => /usr/lib/arm-linux-gnueabihf/tegra/libGL.so.1 (0xb6e83000)
libstdc++.so.6 => /usr/lib/arm-linux-gnueabihf/libstdc++.so.6 (0xb6dd9000)
libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0xb6db8000)
libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0xb6cd0000)
/lib/ld-linux-armhf.so.3 (0xb6fa9000)
libusb-1.0.so.0 => /home/ubuntu/Downloads/libfreenect2/depends/libusb/lib/libusb-1.0.so.0 (0xb6cb8000)
libturbojpeg.so.0 => /usr/lib/arm-linux-gnueabihf/libturbojpeg.so.0 (0xb6c7e000)
libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0xb6c63000)
libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0xb6bf7000)
libX11.so.6 => /usr/lib/arm-linux-gnueabihf/libX11.so.6 (0xb6b10000)
librt.so.1 => /lib/arm-linux-gnueabihf/librt.so.1 (0xb6b02000)
libXrandr.so.2 => /usr/lib/arm-linux-gnueabihf/libXrandr.so.2 (0xb6af4000)
libXi.so.6 => /usr/lib/arm-linux-gnueabihf/libXi.so.6 (0xb6ae2000)
libXxf86vm.so.1 => /usr/lib/arm-linux-gnueabihf/libXxf86vm.so.1 (0xb6ad6000)
libnvidia-tls.so.21.3 => /usr/lib/arm-linux-gnueabihf/tegra/libnvidia-tls.so.21.3 (0xb6aca000)
libnvidia-glcore.so.21.3 => /usr/lib/arm-linux-gnueabihf/tegra/libnvidia-glcore.so.21.3 (0xb53a0000)
libXext.so.6 => /usr/lib/arm-linux-gnueabihf/libXext.so.6 (0xb538e000)
libdl.so.2 => /lib/arm-linux-gnueabihf/libdl.so.2 (0xb5383000)
libudev.so.1 => /lib/arm-linux-gnueabihf/libudev.so.1 (0xb536f000)
libxcb.so.1 => /usr/lib/arm-linux-gnueabihf/libxcb.so.1 (0xb5354000)
libXrender.so.1 => /usr/lib/arm-linux-gnueabihf/libXrender.so.1 (0xb5345000)
libnvidia-rmapi-tegra.so.21.3 => /usr/lib/arm-linux-gnueabihf/tegra/libnvidia-rmapi-tegra.so.21.3 (0xb5333000)
libcgmanager.so.0 => /lib/arm-linux-gnueabihf/libcgmanager.so.0 (0xb5316000)
libnih.so.1 => /lib/arm-linux-gnueabihf/libnih.so.1 (0xb52fd000)
libnih-dbus.so.1 => /lib/arm-linux-gnueabihf/libnih-dbus.so.1 (0xb52ec000)
libdbus-1.so.3 => /lib/arm-linux-gnueabihf/libdbus-1.so.3 (0xb52bb000)
libXau.so.6 => /usr/lib/arm-linux-gnueabihf/libXau.so.6 (0xb52b0000)
libXdmcp.so.6 => /usr/lib/arm-linux-gnueabihf/libXdmcp.so.6 (0xb52a4000)

ldd /catkin_ws/devel/lib/kinect2_bridge/kinect2_bridge

libnodeletlib.so => /opt/ros/indigo/lib/libnodeletlib.so (0xb6eac000)
libopencv_imgproc.so.2.4 => /usr/lib/arm-linux-gnueabihf/libopencv_imgproc.so.2.4 (0xb6cbc000)
libopencv_highgui.so.2.4 => /usr/lib/arm-linux-gnueabihf/libopencv_highgui.so.2.4 (0xb6c85000)
libopencv_core.so.2.4 => /usr/lib/arm-linux-gnueabihf/libopencv_core.so.2.4 (0xb6b0c000)
libclass_loader.so => /opt/ros/indigo/lib/libclass_loader.so (0xb6aee000)
libtf.so => /opt/ros/indigo/lib/libtf.so (0xb6ac4000)
libroscpp.so => /opt/ros/indigo/lib/libroscpp.so (0xb69ce000)
libroscpp_serialization.so => /opt/ros/indigo/lib/libroscpp_serialization.so (0xb69c3000)
librosconsole.so => /opt/ros/indigo/lib/librosconsole.so (0xb699e000)
librostime.so => /opt/ros/indigo/lib/librostime.so (0xb6978000)
libboost_system.so.1.54.0 => /usr/lib/arm-linux-gnueabihf/libboost_system.so.1.54.0 (0xb696c000)
libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0xb6951000)
libconsole_bridge.so.0.2 => /usr/lib/arm-linux-gnueabihf/libconsole_bridge.so.0.2 (0xb6942000)
libkinect2_registration.so => /home/ubuntu/catkin_ws/devel/lib/libkinect2_registration.so (0xb6935000)
libfreenect2.so => /usr/local/lib/libfreenect2.so (0xb6901000)
libstdc++.so.6 => /usr/lib/arm-linux-gnueabihf/libstdc++.so.6 (0xb6856000)
libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0xb6835000)
libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0xb674e000)
/lib/ld-linux-armhf.so.3 (0xb6ef5000)
libbondcpp.so => /opt/ros/indigo/lib/libbondcpp.so (0xb6723000)
libtinyxml.so.2.6.2 => /usr/lib/arm-linux-gnueabihf/libtinyxml.so.2.6.2 (0xb670d000)
libroslib.so => /opt/ros/indigo/lib/libroslib.so (0xb66f8000)
libboost_filesystem.so.1.54.0 => /usr/lib/arm-linux-gnueabihf/libboost_filesystem.so.1.54.0 (0xb66e0000)
libboost_thread.so.1.54.0 => /usr/lib/arm-linux-gnueabihf/libboost_thread.so.1.54.0 (0xb66c8000)
libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0xb665c000)
libjpeg.so.8 => /usr/lib/arm-linux-gnueabihf/libjpeg.so.8 (0xb661d000)
libpng12.so.0 => /lib/arm-linux-gnueabihf/libpng12.so.0 (0xb65fc000)
libtiff.so.5 => /usr/lib/arm-linux-gnueabihf/libtiff.so.5 (0xb65a1000)
libjasper.so.1 => /usr/lib/arm-linux-gnueabihf/libjasper.so.1 (0xb6560000)
libIlmImf.so.6 => /usr/lib/arm-linux-gnueabihf/libIlmImf.so.6 (0xb64d6000)
libHalf.so.6 => /usr/lib/arm-linux-gnueabihf/libHalf.so.6 (0xb648b000)
libgtk-x11-2.0.so.0 => /usr/lib/arm-linux-gnueabihf/libgtk-x11-2.0.so.0 (0xb61c3000)
libgdk-x11-2.0.so.0 => /usr/lib/arm-linux-gnueabihf/libgdk-x11-2.0.so.0 (0xb6149000)
libgobject-2.0.so.0 => /usr/lib/arm-linux-gnueabihf/libgobject-2.0.so.0 (0xb610c000)
libglib-2.0.so.0 => /lib/arm-linux-gnueabihf/libglib-2.0.so.0 (0xb6044000)
libdc1394.so.22 => /usr/lib/arm-linux-gnueabihf/libdc1394.so.22 (0xb5fd7000)
libv4l1.so.0 => /usr/lib/arm-linux-gnueabihf/libv4l1.so.0 (0xb5fc9000)
libavcodec.so.54 => /usr/lib/arm-linux-gnueabihf/neon/vfp/libavcodec.so.54 (0xb563c000)
libavformat.so.54 => /usr/lib/arm-linux-gnueabihf/neon/vfp/libavformat.so.54 (0xb5575000)
libavutil.so.52 => /usr/lib/arm-linux-gnueabihf/neon/vfp/libavutil.so.52 (0xb5553000)
libswscale.so.2 => /usr/lib/arm-linux-gnueabihf/neon/vfp/libswscale.so.2 (0xb552b000)
libz.so.1 => /lib/arm-linux-gnueabihf/libz.so.1 (0xb550f000)
librt.so.1 => /lib/arm-linux-gnueabihf/librt.so.1 (0xb5501000)
libPocoFoundation.so.9 => /usr/lib/libPocoFoundation.so.9 (0xb541f000)
libtf2_ros.so => /opt/ros/indigo/lib/libtf2_ros.so (0xb53a2000)
libtf2.so => /opt/ros/indigo/lib/libtf2.so (0xb5379000)
libxmlrpcpp.so => /opt/ros/indigo/lib/libxmlrpcpp.so (0xb535b000)
libcpp_common.so => /opt/ros/indigo/lib/libcpp_common.so (0xb534d000)
librosconsole_log4cxx.so => /opt/ros/indigo/lib/librosconsole_log4cxx.so (0xb5338000)
librosconsole_backend_interface.so => /opt/ros/indigo/lib/librosconsole_backend_interface.so (0xb532e000)
liblog4cxx.so.10 => /usr/lib/liblog4cxx.so.10 (0xb51f7000)
libboost_regex.so.1.54.0 => /usr/lib/arm-linux-gnueabihf/libboost_regex.so.1.54.0 (0xb5147000)
libgomp.so.1 => /usr/lib/arm-linux-gnueabihf/libgomp.so.1 (0xb5134000)
libusb-1.0.so.0 => /home/ubuntu/Downloads/libfreenect2/depends/libusb/lib/libusb-1.0.so.0 (0xb511c000)
libturbojpeg.so.0 => /usr/lib/arm-linux-gnueabihf/libturbojpeg.so.0 (0xb50e2000)
libglfw.so.3 => /usr/local/lib/libglfw.so.3 (0xb50cc000)
libGL.so.1 => /usr/lib/arm-linux-gnueabihf/tegra/libGL.so.1 (0xb5015000)
libuuid.so.1 => /lib/arm-linux-gnueabihf/libuuid.so.1 (0xb5009000)
librospack.so => /opt/ros/indigo/lib/librospack.so (0xb4fd4000)
liblzma.so.5 => /lib/arm-linux-gnueabihf/liblzma.so.5 (0xb4fb3000)
libjbig.so.0 => /usr/lib/arm-linux-gnueabihf/libjbig.so.0 (0xb4fa1000)
libIex.so.6 => /usr/lib/arm-linux-gnueabihf/libIex.so.6 (0xb4f85000)
libIlmThread.so.6 => /usr/lib/arm-linux-gnueabihf/libIlmThread.so.6 (0xb4f78000)
libgmodule-2.0.so.0 => /usr/lib/arm-linux-gnueabihf/libgmodule-2.0.so.0 (0xb4f6d000)
libpangocairo-1.0.so.0 => /usr/lib/arm-linux-gnueabihf/libpangocairo-1.0.so.0 (0xb4f5b000)
libX11.so.6 => /usr/lib/arm-linux-gnueabihf/libX11.so.6 (0xb4e75000)
libXfixes.so.3 => /usr/lib/arm-linux-gnueabihf/libXfixes.so.3 (0xb4e69000)
libatk-1.0.so.0 => /usr/lib/arm-linux-gnueabihf/libatk-1.0.so.0 (0xb4e4b000)
libcairo.so.2 => /usr/lib/arm-linux-gnueabihf/libcairo.so.2 (0xb4da1000)
libgdk_pixbuf-2.0.so.0 => /usr/lib/arm-linux-gnueabihf/libgdk_pixbuf-2.0.so.0 (0xb4d81000)
libgio-2.0.so.0 => /usr/lib/arm-linux-gnueabihf/libgio-2.0.so.0 (0xb4c96000)
libpangoft2-1.0.so.0 => /usr/lib/arm-linux-gnueabihf/libpangoft2-1.0.so.0 (0xb4c80000)
libpango-1.0.so.0 => /usr/lib/arm-linux-gnueabihf/libpango-1.0.so.0 (0xb4c42000)
libfontconfig.so.1 => /usr/lib/arm-linux-gnueabihf/libfontconfig.so.1 (0xb4c13000)
libXrender.so.1 => /usr/lib/arm-linux-gnueabihf/libXrender.so.1 (0xb4c03000)
libXinerama.so.1 => /usr/lib/arm-linux-gnueabihf/libXinerama.so.1 (0xb4bf9000)
libXi.so.6 => /usr/lib/arm-linux-gnueabihf/libXi.so.6 (0xb4be7000)
libXrandr.so.2 => /usr/lib/arm-linux-gnueabihf/libXrandr.so.2 (0xb4bd9000)
libXcursor.so.1 => /usr/lib/arm-linux-gnueabihf/libXcursor.so.1 (0xb4bca000)
libXcomposite.so.1 => /usr/lib/arm-linux-gnueabihf/libXcomposite.so.1 (0xb4bbe000)
libXdamage.so.1 => /usr/lib/arm-linux-gnueabihf/libXdamage.so.1 (0xb4bb4000)
libXext.so.6 => /usr/lib/arm-linux-gnueabihf/libXext.so.6 (0xb4ba2000)
libffi.so.6 => /usr/lib/arm-linux-gnueabihf/libffi.so.6 (0xb4b94000)
libpcre.so.3 => /lib/arm-linux-gnueabihf/libpcre.so.3 (0xb4b5d000)
libraw1394.so.11 => /usr/lib/arm-linux-gnueabihf/libraw1394.so.11 (0xb4b4b000)
libv4l2.so.0 => /usr/lib/arm-linux-gnueabihf/libv4l2.so.0 (0xb4b38000)
libxvidcore.so.4 => /usr/lib/arm-linux-gnueabihf/libxvidcore.so.4 (0xb4a5b000)
libx264.so.142 => /usr/lib/arm-linux-gnueabihf/libx264.so.142 (0xb4929000)
libvpx.so.1 => /usr/lib/arm-linux-gnueabihf/vfp/neon/libvpx.so.1 (0xb47a4000)
libvorbisenc.so.2 => /usr/lib/arm-linux-gnueabihf/libvorbisenc.so.2 (0xb4636000)
libvorbis.so.0 => /usr/lib/arm-linux-gnueabihf/libvorbis.so.0 (0xb460c000)
libtheoraenc.so.1 => /usr/lib/arm-linux-gnueabihf/libtheoraenc.so.1 (0xb45d6000)
libtheoradec.so.1 => /usr/lib/arm-linux-gnueabihf/libtheoradec.so.1 (0xb45bf000)
libspeex.so.1 => /usr/lib/arm-linux-gnueabihf/libspeex.so.1 (0xb45a6000)
libschroedinger-1.0.so.0 => /usr/lib/arm-linux-gnueabihf/libschroedinger-1.0.so.0 (0xb4514000)
libopus.so.0 => /usr/lib/arm-linux-gnueabihf/libopus.so.0 (0xb44e2000)
libopenjpeg.so.2 => /usr/lib/arm-linux-gnueabihf/libopenjpeg.so.2 (0xb44c5000)
libmp3lame.so.0 => /usr/lib/arm-linux-gnueabihf/libmp3lame.so.0 (0xb4458000)
libgsm.so.1 => /usr/lib/arm-linux-gnueabihf/libgsm.so.1 (0xb4447000)
libva.so.1 => /usr/lib/arm-linux-gnueabihf/libva.so.1 (0xb442e000)
librtmp.so.0 => /usr/lib/arm-linux-gnueabihf/librtmp.so.0 (0xb4414000)
libgnutls.so.26 => /usr/lib/arm-linux-gnueabihf/libgnutls.so.26 (0xb438e000)
libbz2.so.1.0 => /lib/arm-linux-gnueabihf/libbz2.so.1.0 (0xb437a000)
libdl.so.2 => /lib/arm-linux-gnueabihf/libdl.so.2 (0xb436f000)
libactionlib.so => /opt/ros/indigo/lib/libactionlib.so (0xb4351000)
libaprutil-1.so.0 => /usr/lib/arm-linux-gnueabihf/libaprutil-1.so.0 (0xb4330000)
libapr-1.so.0 => /usr/lib/arm-linux-gnueabihf/libapr-1.so.0 (0xb430c000)
libicuuc.so.52 => /usr/lib/arm-linux-gnueabihf/libicuuc.so.52 (0xb420e000)
libicui18n.so.52 => /usr/lib/arm-linux-gnueabihf/libicui18n.so.52 (0xb40b9000)
libudev.so.1 => /lib/arm-linux-gnueabihf/libudev.so.1 (0xb40a4000)
libXxf86vm.so.1 => /usr/lib/arm-linux-gnueabihf/libXxf86vm.so.1 (0xb4098000)
libnvidia-tls.so.21.3 => /usr/lib/arm-linux-gnueabihf/tegra/libnvidia-tls.so.21.3 (0xb408d000)
libnvidia-glcore.so.21.3 => /usr/lib/arm-linux-gnueabihf/tegra/libnvidia-glcore.so.21.3 (0xb2963000)
libboost_program_options.so.1.54.0 => /usr/lib/arm-linux-gnueabihf/libboost_program_options.so.1.54.0 (0xb290e000)
libpython2.7.so.1.0 => /usr/lib/arm-linux-gnueabihf/libpython2.7.so.1.0 (0xb26b0000)
libfreetype.so.6 => /usr/lib/arm-linux-gnueabihf/libfreetype.so.6 (0xb2649000)
libxcb.so.1 => /usr/lib/arm-linux-gnueabihf/libxcb.so.1 (0xb262f000)
libpixman-1.so.0 => /usr/lib/arm-linux-gnueabihf/libpixman-1.so.0 (0xb25b9000)
libxcb-shm.so.0 => /usr/lib/arm-linux-gnueabihf/libxcb-shm.so.0 (0xb25ad000)
libxcb-render.so.0 => /usr/lib/arm-linux-gnueabihf/libxcb-render.so.0 (0xb259f000)
libselinux.so.1 => /lib/arm-linux-gnueabihf/libselinux.so.1 (0xb2581000)
libresolv.so.2 => /lib/arm-linux-gnueabihf/libresolv.so.2 (0xb2568000)
libharfbuzz.so.0 => /usr/lib/arm-linux-gnueabihf/libharfbuzz.so.0 (0xb2527000)
libthai.so.0 => /usr/lib/arm-linux-gnueabihf/libthai.so.0 (0xb2518000)
libexpat.so.1 => /lib/arm-linux-gnueabihf/libexpat.so.1 (0xb24f8000)
libv4lconvert.so.0 => /usr/lib/arm-linux-gnueabihf/libv4lconvert.so.0 (0xb2482000)
libogg.so.0 => /usr/lib/arm-linux-gnueabihf/libogg.so.0 (0xb2475000)
liborc-0.4.so.0 => /usr/lib/arm-linux-gnueabihf/liborc-0.4.so.0 (0xb241c000)
libgcrypt.so.11 => /lib/arm-linux-gnueabihf/libgcrypt.so.11 (0xb23b8000)
libtasn1.so.6 => /usr/lib/arm-linux-gnueabihf/libtasn1.so.6 (0xb23a4000)
libp11-kit.so.0 => /usr/lib/arm-linux-gnueabihf/libp11-kit.so.0 (0xb2376000)
libcrypt.so.1 => /lib/arm-linux-gnueabihf/libcrypt.so.1 (0xb233e000)
libicudata.so.52 => /usr/lib/arm-linux-gnueabihf/libicudata.so.52 (0xb0cca000)
libcgmanager.so.0 => /lib/arm-linux-gnueabihf/libcgmanager.so.0 (0xb0cad000)
libnih.so.1 => /lib/arm-linux-gnueabihf/libnih.so.1 (0xb0c94000)
libnih-dbus.so.1 => /lib/arm-linux-gnueabihf/libnih-dbus.so.1 (0xb0c84000)
libdbus-1.so.3 => /lib/arm-linux-gnueabihf/libdbus-1.so.3 (0xb0c53000)
libnvidia-rmapi-tegra.so.21.3 => /usr/lib/arm-linux-gnueabihf/tegra/libnvidia-rmapi-tegra.so.21.3 (0xb0c40000)
libutil.so.1 => /lib/arm-linux-gnueabihf/libutil.so.1 (0xb0c35000)
libXau.so.6 => /usr/lib/arm-linux-gnueabihf/libXau.so.6 (0xb0c2a000)
libXdmcp.so.6 => /usr/lib/arm-linux-gnueabihf/libXdmcp.so.6 (0xb0c1e000)
libgraphite2.so.3 => /usr/lib/arm-linux-gnueabihf/libgraphite2.so.3 (0xb0c04000)
libdatrie.so.1 => /usr/lib/arm-linux-gnueabihf/libdatrie.so.1 (0xb0bf8000)
libgpg-error.so.0 => /lib/arm-linux-gnueabihf/libgpg-error.so.0 (0xb0bec000)

I am sorry to trouble you, but I appreciate your support.

[Soy-Robotics]

I found the libusb for libfreenect2 and kinect2_bridge is same. So you must not have any libusb issues. Can you please do these

1. remove usb from usb ports of jetson board
2. restart jetson sbc. After restarting, once you see ubuntu screen, connect kinect2 usb into jetson USB3.0 port which i have shown in the picture.
3. open the terminal and check lsusb $ lsusb you must see three microsoft tags in lsusb list
4. As jetson usb3.0 is not preactivated, you should suspend usb ports Open the terminal and type following $ sudo sh -c 'for dev in /sys/bus/usb/devices//power/autosuspend; do echo -1 >$dev; done' $ grep . /sys/bus/usb/devices//power/autosuspend
5. now try running kinect2_bridge

[bingyo]

I restart SBC setup ・2.2.3 Now update the kernal ・2.3 CUDA Installation

But I can't see microsoft tags in lsusb list. I don't forget to create a udev rule /etc/udev/rules.d/90-kinect2.rules.

[Soy-Robotics]

when you are running kinect2 bridge, in the next terminal try to run rostopic echo's $ rostopic list $ rostopic echo /kinect2/hd/......

Check whether you are able to see messages published by kinect2 of color, depth ir seperately

[bingyo]

When I was going to connect Kinect V2 to a Mini PCI-Express Card.

I try "rostopic echo /kinect2/hd/......".

I can see data of camera_info, image_color, image_mono. Because processing is working.

so I can not see data of image_depth_rect.

[Soy-Robotics]

by using mini PCI card, are you able to see color and depth images properly from ./Protonect?? If so please try ./Protonect by connecting to USB 3.0.. Let me explain my setup. I have a jetson board with a small USB 3.0 hub connected to USB 3.0 jetson board. To the usb hub i have connected kinect2, keyboard, mouse. The display connected to hdmi port. When ever i restart jetson, i remove usb hub. Once restarted i will be connecting usb hub. If i dont remove usb hub and restart jetson by keeping usb hub plugged in, i used to get same error as you are getting now. Kinect2_bridge publishes color image but not depth. So please use usb port of jetson board(For me its working well). You can connect mouse and keyboard with MINI PCI card, but when connecting kinect2, pls use usb 3.0 of jetson board. When ever you restarting jetson, please remove all the usb ports and once restarted connect the usb hubs. Once done allow usb devises connected by running $ sudo sh -c 'for dev in /sys/bus/usb/devices//power/autosuspend; do echo -1 >$dev; done' and then $ grep . /sys/bus/usb/devices//power/autosuspend check that all should be -1. And you must see three microsoft tags in lsusb $ lsusb If you see depth image by running kinect2_bridge and kinect2_viewer, i will let you know how to avoid typing all these stuff to allow usb ports in next step.

[Soy-Robotics]

is there any update?? can i close this issue @bingyo

[bingyo]

I am sorry for the delay in getting in touch with you.

by using mini PCI card, are you able to see color and depth images properly from ./Protonect??

Yes, I can see image by ./Protonect

I want to retry your setup. Of course, From now on I don't use mini PCI card. Please wait for a day.

[bingyo]

I retried your setup.But I can't find out the bus id and device id of Kinect 2. I don't understand why this happens...

I understand your jetson can use Kinect2.So I will keep doing the best that I can. Please close this issue. Thank you very much! and happy new year!

[Soy-Robotics]

@bingyo, can u please elaborate your kinect 2 peripherals? are you using kinect 2 with kinect adapter or some local made adapter? as i found from some local made adapters you can see only one microsoft tag when connected to pc. eventhough it still works with it. Happy new year and all the best

[bingyo]

I don't use adapter for kinect v2. run the power from a 12v Li-Po battery. https://www.youtube.com/watch?v=KZunnyw_RKQ

I see one microsoft tag when connected to my note pc. And it works kinect_bridge.

[bingyo]

I add comment. Sorry @GaiTech-Robotics

I checked https://github.com/code-iai/iai_kinect2/issues/4#issuecomment-144222923

May be I used source code is new version. I think @xlz already know. Please tell me you(GaiTech-Robotics and xlz) use commit version.

[xlz]

You said Protonect works, but kinect2_bridge does not work. The only possible problem is your libusb. You should remove all your libusb copies under those depends directories, and install libusb binary directly sudo apt-add-repository ppa:floe/libusb && sudo apt-get update && sudo apt-get install libusb-1.0-0-dev. And rebuild libfreenect2.

It seems you have multiple copies if libfreenect2 littered everywhere. You should remove them all, and do a clean build using the current libfreenect2 master.

There is a lot of irrelevant information here, hard to follow.

[bingyo]

Thank you for your reply.

Which libfreenect2 should I use? GaiTech-Robotics(xlz) and OpenKinect or OpenKinect only?

[bingyo]

@xlz I updated libusb binary. And I rebuild libfreenect2[GaiTech-Robotics(xlz)] and libfreenect2[OpenKinect]. But kinect2_bridge did not work. I saw a infHz for the first time.

[ INFO] [Kinect2Bridge::main] depth processing: ~nanms (~nanHz) publishing rate: ~0Hz
・・・
[ INFO] [Kinect2Bridge::main] depth processing: ~infms (~0Hz) publishing rate: ~0Hz

[xlz]

I can only help you with OpenKinect libfreenect2 and the results from Protonect.

[bingyo]

Thank you for your help. I did rebuild OpenKinect libfreenect2.(I don't forget to command "cmake .. -DENABLE_CXX11=ON") Protonect kinect2_bridge did not work...

When run Protonect

[Info] [Freenect2DeviceImpl] started
...
[Debug] [RgbPacketStreamParser] skipping rgb packet!
[Debug] [RgbPacketStreamParser] skipping rgb packet!
[Debug] [DepthPacketStreamParser] not all subsequences received 16
[Debug] [RgbPacketStreamParser] skipping rgb packet!
...
[Info] [TurboJpegRgbPacketProcessor] avg. time: 135.436ms -> ~7.38354Hz
[Debug] [RgbPacketStreamParser] skipping rgb packet!
[Debug] [DepthPacketStreamParser] not all subsequences received 32
...

When run kinect2_bridge

ubuntu@tegra-ubuntu:~/catkin_ws$ rosrun kinect2_bridge kinect2_bridge
...
[ INFO] [Kinect2Bridge::initDevice] starting kinect2
[Info] [Freenect2DeviceImpl] starting...
[Error] [protocol::CommandTransaction] bulk transfer failed: LIBUSB_ERROR_IO Input/Output Error
Segmentation fault

[xlz]

What does ldd kinect2_bridge | grep libusb say? If it shows /home/ubuntu/Downloads/libfreenect2/depends/libusb/..., you did not follow instructions.

If it shows /usr/lib/..., then post results of dmesg and LIBUSB_DEBUG=3 rosrun kinect2_bridge kinect2_bridge.

[bingyo]

Output files are very lengthy data because I don't know unnecessary information. Please check my output files.

ubuntu@tegra-ubuntu:~$ ldd catkin_ws/devel/lib/kinect2_bridge/kinect2_bri dge | grep libusb
    libusb-1.0.so.0 => /lib/arm-linux-gnueabihf/libusb-1.0.so.0 (0xb517c000)

Run "ls /usr/lib/" usr_lib_LIST.txt

Run "dmesg" dmesg.txt

Run LIBUSB_DEBUG=3 rosrun kinect2_bridge LIBUSB_DEBUG.txt

[xlz]

First, I don't believe the provided dmesg.txt contains the messages up to the point when you ran kinect2_bridge.

Second, your reported errors LIBUSB_ERROR_IO do not match those in LIBUSB_DEBUG.txt.

[xlz]

You should first run rosrun kinect2_bridge kinect2_bridge and after it crashes run dmesg. Only in this order can you gather information about the errors, not the other way around.

Also, LIBUSB_DEBUG=3 is not passed to kinect2_bridge. I think rosrun filtered it, and I don't know how to really pass it to catkin_ws/devel/lib/kinect2_bridge/kinect2_bridge. You can strace -f -etrace=execve -o/tmp/trace rosrun kinect2_bridge kinect2_bridge to find out the exact command line and environment variables that execute catkin_ws/devel/lib/kinect2_bridge/kinect2_bridge, and run that command with those environment variables with the additional LIBUSB_DEBUG=3.

[bingyo]

I couldn't run same like my reported https://github.com/GaiTech-Robotics/Ros2Bot/issues/1#issuecomment-169910795 I'm sorry.

Run "strace -f -etrace=execve -o/tmp/trace rosrun kinect2_bridge kinect2_bridge" strace_kinect_bridge.txt

Run "dmesg" dmesg.txt

[xlz]

Also, @bingyo, I did not ask you to provide the output of strace -f -etrace=execve -o/tmp/trace rosrun kinect2_bridge kinect2_bridge. I asked:

run that command with those environment variables with the additional LIBUSB_DEBUG=3

"That command" is determined by looking at /tmp/trace after strace -f -etrace=execve -o/tmp/trace rosrun kinect2_bridge kinect2_bridge. If you don't know what this means, I'm unable to help with setting up LIBUSB_DEBUG logging.

Your second dmesg.txt does show representative error messages:

[  393.439069] xhci_hcd 0000:01:00.0: ERROR Transfer event TRB DMA ptr not part of current TD

There is no way to determine whether this error happened during Protonect or kinect2_bridge. I hope you can compare relevant dmesg during Protonect and kinect2_bridge. And LIBUSB_DEBUG log is essential in determining the problem.

I also see from your first dmesg.txt that you are using 3.10.40-grinch-21.3.4 kernel instead of the official kernel. I have not tested the grinch kernel. I have successful test results using the official kernel. The grinch kernel might be the reason for the above dmesg error messages.

[bingyo]

@xlz Sorry I didn’t know about strace. I’ll get /tmp/trace data.

First, I’ll restart setup of Jetson. Can I use Jetson TK1 Development Pack 1.1 [L4T r21.3] ? Next, I’ll get strace, /usr/lib/, and dmesg data if don’t work kinect_bridge.

Please wait…

[bingyo]

I have finished setup of Jetson. Unfortunately don't work Protonect and kinect_bridge. I checked issues page of OpenKinect and iai_kinect2. But I didn't know why not work.

I added LIBUSB_DEBUG=3 in ~/bashrc Is this correct? I will attach /usr/lib/, strace, and dmesg data.

lib_list.txt

When run Protonect Protonect_trace.txt Protonect_dmesg.txt Protonect_output.txt

When run kinect_bridge bridge_trace.txt bridge_dmesg.txt kinect_bridge_output.txt

[cocoszhou]

I want use gmapping_kinect2_cpu.launch, but it seems need package "kinect2_bridge/launch/kinect2_cpu.launch"? where is "kinect2_cpu.launch"?

THX.

[Soy-Robotics]

@cocoszhou , I have updated the package. You can find them here https://github.com/GaiTech-Robotics/Ros2Bot/tree/master/kinect2-launch

[cocoszhou]

THX.. I See.

[cocoszhou]

THX.. I See.

[cocoszhou]

THX.. I See.

[cocoszhou]

THX.. I See.

[bingyo]

@xlz , My output file[https://github.com/GaiTech-Robotics/Ros2Bot/issues/1#issuecomment-170220232] is okay?

I have one more of the Jetson TK1. So I retry setup again by @GaiTech-Robotics . I don't use Mini PCI-Express Card. I saw three microsoft tags in lsusb list for the first time. Bugger me. But kinect_bridge don't work.I saw same output

[ INFO] [Kinect2Bridge::main] depth processing: ~nanms (~nanHz) publishing rate: ~0Hz

Does @GaiTech-Robotics use libusb-1.0-0.dev of version 1.0.17?

[Soy-Robotics]

@bingyo , is protonect working? May i have complete output form kinect2_bridge?

[bingyo]

@GaiTech-Robotics , Not working.

Run ./Downloads/libfreenect2/build/bin/Protonect (Repository of GaiTech) Console is

...
[RgbPacketStreamParser] skipping rgb packet!
...
[TurboJpegRgbPacketProcessor] avg. time: 89.4523ms -> ~11.179Hz

Run ./libfreenect2/examples/protonect/bin/Protonect (Repository of OpenKinect) Same console and didn't show Windows.

[Soy-Robotics]

have you changed usb port info from 0 to 2??

USB 3.0 must be enabled for full performance. Edit /boot/extlinux/extlinux.conf to change usb_port_owner_info=0 to usb_port_owner_info=2 to enable USB 3.0.

$ sudo -s

$ sudo gedit /boot/extlinux/extlinux.conf

Now change "usb_port_owner_info=0" to "usb_port_owner_info=2"

and did you add kinect2 rules ?

You must have read and write permissions on the USB devices of Kinect 2.

lsusb to find out the bus id and device id of Kinect 2 (there should be 3 devices). ls -l /dev/bus/usb/$BUS_ID/$DEVICE_ID and you should have rw permissions. To make it permanent, you can create a udev rule /etc/udev/rules.d/90-kinect2.rules. Add these lines in it. ATTR{product}=="Kinect2"

SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02c4", MODE="0666"

SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02d8", MODE="0666"

SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02d9", MODE="0666" -Now remove the kinec2 cable, restart the system After reboot check for kinect2 inputs by typing lsusb