Different errors when running Kinect v2 while Calibration with TK1s.

[tonigialanze]

Hi everyone,

Good morning from Malta.

At the moment I am doing the Intrinsic Calibration on the Kinect v2 from the master pc. Most times when running this command: roslaunch kinect2_bridge kinect2_bridge.launch, I get the readings below on the console.

When I plug in a different Kinect v2 it seems to work, but the thing is that yesterday there was one which was not working and today is working, and vice versa. So the one which was not working yesterday is working today.

I am confused, could someone explain what could be the issue please. Thanks.

P.S (I apologize for pasting all the code)

starting kinect2

[Freenect2DeviceImpl] starting...
[Freenect2DeviceImpl] ReadData0x14 response
92 bytes of raw data
0x0000:  00 00 12 00 00 00 00 00 01 00 00 00 43 c1 1f 41    ............C..A
0x0010:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
0x0020:  0a 21 33 55 20 00 17 ba 00 08 00 00 10 00 00 00    .!3U ...........
0x0030:  00 01 00 00 00 10 00 00 00 00 80 00 01 00 00 00    ................
0x0040:  31 33 00 00 00 02 11 01 47 48 52 32 38 35 2e 31    13......GHR285.1
0x0050:  58 00 00 00 00 00 00 00 07 00 00 00                X...........

[Freenect2DeviceImpl] ReadStatus0x090000 response
4 bytes of raw data
0x0000:  23 26 00 00                                        #&..

[Freenect2DeviceImpl] ReadStatus0x090000 response
4 bytes of raw data
0x0000:  23 26 00 00                                        #&..

[Freenect2DeviceImpl] enabling usb transfer submission...
[Freenect2DeviceImpl] submitting usb transfers...
[Freenect2DeviceImpl] started

device serial: 028194433547
device firmware: 4.1.3917.0.7

default ir camera parameters: 
fx 368.201, fy 368.201, cx 253.871, cy 214.62
k1 0.0871863, k2 -0.273632, p1 0, p2 0, k3 0.0984557

default color camera parameters: 
fx 1081.37, fy 1081.37, cx 959.5, cy 539.5

camera parameters used:
camera matrix color:
[5962.055617609727, 0, 779.7913688789874;
  0, 12556.5289317544, 543.8668843524923;
  0, 0, 1]
distortion coefficients color:
[-30.26037214777007, 1160.387991462479, 0.3517051675311071, -1.272760934364576, -10368.56622414078]
camera matrix ir:
[357.9977260641165, 0, 249.7364218377166;
  0, 370.1276591341381, 208.5578089682938;
  0, 0, 1]
distortion coefficients ir:
[-0.09766178474697561, -0.02722784118754196, -0.01406074628168491, 0.0276686733668838, 0.03489189570138515]
rotation:
[]
translation:
[]

[DepthPacketStreamParser::handleNewData] not all subsequences received 0
[DepthPacketStreamParser::handleNewData] not all subsequences received 896
[DepthRegistrationOpenCL::listDevice]  devices:
  0: GeForce 820M (GPU)[NVIDIA Corporation]
[DepthRegistrationOpenCL::init]  selected device: GeForce 820M (GPU)[NVIDIA Corporation]
[kinect2_head_kinect2_bridge-2] process has died [pid 4311, exit code -11, cmd /home/late_interactive/workspace/ros/catkin/devel/lib/kinect2_bridge/kinect2_bridge __name:=kinect2_head_kinect2_bridge __log:=/home/late_interactive/.ros/log/b7dc7794-5585-11e7-9569-0862660dea0c/kinect2_head_kinect2_bridge-2.log].
log file: /home/late_interactive/.ros/log/b7dc7794-5585-11e7-9569-0862660dea0c/kinect2_head_kinect2_bridge-2*.log
[kinect2_head_kinect2_bridge-2] restarting process
process[kinect2_head_kinect2_bridge-2]: started with pid [4347]
[DepthRegistration::New] Using OpenCL registration method!
[DepthRegistration::New] Using OpenCL registration method!
[OpenCLDepthPacketProcessor::listDevice]  devices:
  0: GeForce 820M (GPU)[NVIDIA Corporation]
[OpenCLDepthPacketProcessor::init]  selected device: GeForce 820M (GPU)[NVIDIA Corporation]
[Freenect2Impl] enumerating devices...
[Freenect2Impl] 10 usb devices connected
[Freenect2Impl] found valid Kinect v2 @3:5 with serial 028194433547
[Freenect2Impl] found 1 devices
Kinect2 devices found: 
  0: 028194433547 (selected)
[Freenect2DeviceImpl] opening...
[Freenect2DeviceImpl] opened

[tonigialanze]

I have read in other issues that the Intrinsic Calibration is not indispensable for the Kinect v2, but I am afraid that the same issues will happen when doing the network calibration.

[marketto89]

Hey @tonigialanze

I can tell from the output that kinect2_bridge is reading a very weird calibration for the sensor you are using. This causes the node to segfaults. In order to fix the problem you should remove the folder named 028194433547 inside your kinect2_bridge/data (or rename it with another name). You should do the same also for the folder named with the serial of the other camera, if you say that it crashes too.

[tonigialanze]

Hey Marketto grazzie per la risposta subita :)

I have deleted that files as you suggested now, but I've been having different errors and would like to know what's the causing them. I am trying to calibrate a network of 4 sensors using 3 Jetson TK1's and a laptop with a good graphic card, ram and cpu.

1)

Most time when running roslaunch opt_calibration sensor_<sensor_id>.launch on Jetson TK1's I get this error:

[DepthPacketStreamParser::handleNewData] subpacket incomplete (needed 298496 received 141408)
[DepthPacketStreamParser::handleNewData] subpacket incomplete (needed 298496 received 28160)

I don't know if this error is 'normal' or not but it is happening constantly on the TK1s. The imagers also do not show up in Rviz, unless I need to set them up manually (or does it come automatically?)

2)

Occasionally I also get this error when trying to run the same command:

[DepthRegistration::New] Using OpenCL registration method!
[DepthRegistration::New] Using OpenCL registration method!
[OpenCLDepthPacketProcessor::init] ERROR: clGetPlatformIDs(-1001)
terminate called after throwing an instance of 'cl::Error'
  what():  clGetPlatformIDs
[Kinect_Master_kinect2_bridge-5] process has died [pid 7527, exit code -6, cmd /home/late_interactive/workspace/ros/catkin/devel/lib/kinect2_bridge/kinect2_bridge __name:=Kinect_Master_kinect2_bridge __log:=/home/late_interactive/.ros/log/924f5b68-55da-11e7-a5f3-0862660dea0c/Kinect_Master_kinect2_bridge-5.log].
log file: /home/late_interactive/.ros/log/924f5b68-55da-11e7-a5f3-0862660dea0c/Kinect_Master_kinect2_bridge-5*.log
[Kinect_Master_kinect2_bridge-5] restarting process

3)

Sometimes I also get an error that the Kinect device was not found even though it is connected to the PC.

I am imagining that these three different errors cover a lot of area of information, but I would like to know more as there is nothing specifically about this in the Troubleshooting section.

Thanks again.

[marketto89]

Regarding 1) it is normal to see the errors you mention. RViz is meant to show the people results (tracking) or the checkerboards (network calibration). If you want to see the images, you should add them manually. Also, other methods to check the images are either rqt_image_view or rostopic hz /<image_topic_you_want_to_check>

Regarding 2) have a look at this issue

Regarding 3) Sometimes it happens. I can suggest you to unplug and plug back the Kinect and look dmesg for errors.

[tonigialanze]

Unfortunately I was having a lot of various types of errors so I couldn't even start doing the network calibration.

For starters when launching roslaunch opt_calibration opt_calibration_master.launch it was giving me this message which I do not think it's normal:

setting /run_id to 14a6207a-567a-11e7-ad0a-0862660dea0c
process[rosout-1]: started with pid [23156]
started core service [/rosout]
process[rviz-2]: started with pid [23163]
process[opt_calibration_status_plot-3]: started with pid [23174]
process[opt_calibration-4]: started with pid [23175]
[ INFO] [1498046799.934734893]: /Kinect_Master added.
[ INFO] [1498046800.022620665]: /Kinect_Node_1 added.
[ INFO] [1498046800.052406160]: /Kinect_Node_2 added.
[ INFO] [1498046800.074780129]: /Kinect_Node_3 added.
[ WARN] [1498046800.085064239]: Not all messages received. Waiting...
[ WARN] [1498046805.085086628]: Not all messages received. Waiting...
[ WARN] [1498046810.085094031]: Not all messages received. Waiting...

and it continued with the 'Not all messages received' error. I've seen a lot of other users' issues and seen that in this particular terminal they have a message like "All sensors connected".

In Rviz I never saw the sensors inside the area where there is the grid, and never managed to get the mono low res image from TK1s as described in the Calibration in Practice page, it kept saying no image received.

Apart from the errors above (in this as well as previous post), this is another error which I couldn't get the TK1s to work was:

ADVERTISING: /Kinect_Node_2/depth_ir/points
set process thread priority to: 19
set process thread priority to: 19
starting main loop
set process thread priority to: 19
set process thread priority to: 19

never got passed this message.

Error numbered 1 in my previous post was the most common and even though @marketto89 said it's normal, I couldn't get the image inside Rviz, so I guess there was something wrong.

Some more Information. (Updated)

I don't know if this was the reason for all the errors but I went ahead and changed the hostname and IP address of each TK1 so I can access them. But I am guessing that even if its not in the instructions it should be done. I also used the NTP Configuration 2 even though I was testing with 4 sensors at the time (should be 6 or more on Configuration 2). I also skipped the Intrinsic Calibration as some of you suggested that with the Kinects v2 is not needed since the default calibrations are good. One other thing is that my checkerboard does not have a white border, so maybe this is not helping with calibration also.

At the moment I am re-installing everything in the Jetson TK1s so I restart everything again and double checking the process. I finished this and tested with a small 2 system camera but still no luck. The image from the Kinect that is stuck to the master computer works, the other does not and sends this error repeatedly:

[DepthPacketStreamParser::handleNewData] subpacket incomplete (needed 298496 received 28160)

[marketto89]

The message Not all messages received. Waiting... is displayed whenever the calibration procedure does not receive any data from at least one sensor, so it is correlated to the second issue you describe:

set process thread priority to: 19 never got passed this message.

This error is related to the USB firmware of the TK1 which causes troubles with the Kinect v2.

In particular, I suggest to:

• Connect to the power the Kinect v2 before plugging it to the TK1
• Turn the TK1 on, wait 20-30 seconds and then plug the Kinect v2 USB

In our experiments, following these instructions should make the Kinect v2 work as expected.

Let us know if this works for you

[tonigialanze]

Thanks a lot Marketto. As you suggested I started to boot the TK1s and than plug the Kinects and managed to arrive to the calibration stage.

Quite a task with 6 cameras :o

Thanks again

[tonigialanze]

I apologise for posting again here since it's closed, I forgot that the TK1s won't be accessible after we put the installation structure in place, so I cannot plug and plug again every time. The the installation will be there for 15 days.

Is there a work-around? Or should I leave the system on (or maybe just the TK1s) for the whole two weeks?

[nanodust]

re: leaving system on for weeks - it should be fine to do so.

the requirement of (power + kinect) decoupled from (boot+cpu) , and connecting them only after boot, could be simulated with software - it should be possible to delay udev or otherwise simulate unplugging and re-plugging the kincect in. I've had to hack such things in past, they are ugly but work. I don't have a TX1 so I can't get specific here, sorry.