OpenNI2-FreenectDriver with OpenCV

[OrysB]

Hey there,

in my recent attempt to get OpenNI2 working with OpenCV 4.4 and the Kinect V1 I stumbled across this most unfortunate Problem:

https://github.com/opencv/opencv/issues/18226

Can the OpenNI2-FreenectDriver create an IR stream? When I look in libfreenect/OpenNI2-FreenectDriver/src/ there are no IRStream files. Please correct me in case I got something wrong, otherwise what would be the best way to approach this? Thanks for help & have a nice day! Orys

[piedar]

As you noted, libfreenect IR is a video mode so it's not possible to capture other video at the same time. I guess the options are

• Write a program which integrates libfreenect and OpenCV directly (more control, no support for other devices)
• Modify OpenCV to continue gracefully if the IR stream is missing (possible reduced functionality in that case)

[OrysB]

Hey piedar, the problem with the first approach is, that if I want to use NiTE I have to use the OpenNI2 driver. Since I can't use the device with two recall functions at the same time, integrating libfreenect directly into OpenCV leads to a device busy error from libusb.

As for the second suggestion: it would be quite an effort to alter the OpenNI2-videocapture-support of OpenCV to work with the Kinect360 (the whole thing seems to rely on a device with three sensors). Do you know someone who already did it? Maybe I will do it some other time, but for now, since NiTE is not at all the latest approach for skeleton recognition, I think about dropping the whole NiTE thing and spending my time on NeuralNetworks.

Thanks for your help though. I am an absolute beginner, but it seems that at least I got the problem right. So that's already a success to me! :D Have a nice day! Orys