search

OpenKinect / libfreenect2

Open source drivers for the Kinect for Windows v2 device
2.07k stars 746 forks source link

Azure Kinect DK discussion #1067

Open xlz opened 5 years ago

xlz commented 5 years ago

Now it seems Azure Kinect is the next-generation Kinect and Kinect v2 has more or less entered an EOL status.

Their doc says "preview support for Linux... will be open sourced and made available in GitHub" and it targets Ubuntu 18.04 (x64) with OpenGL v4.4 or later.

This is certainly good news to have official Linux support. The question is what level of open source it is. Usually this kind of things are released as an open source wrapper for a binary blob. But there isn't really a lot of secret sauce in the Sensor SDK itself, which merely involves various jobs of bit-banging, so there's also a good chance it's completely open source. The second question is what level of support it is, e.g. how much back and forth it is to add some custom thing I want into it, support for various miscellaneous platforms and hardware models, how community friendly it is, etc. At the very least we want to add CUDA and OpenCL support.

After the equivalent of libfreenect4 supposedly released by Microsoft, there will also have demand for an equivalent of iai_kinect4. We'll see how it turns out.

floe commented 5 years ago

Do you have a pointer to that doc? Couldn't find it... On a related note, did you already pre-order the Kinect Azure? Still not possible in Europe, sadly... :'-(

xlz commented 5 years ago

https://aka.ms/kinectdocs

in case it doesn't work

document.pdf

carlhuang070 commented 5 years ago

How to get Azure Kinect DK Body tracking SDK ?

survyvaish commented 5 years ago

Has anyone written a ROS driver for the new Azure Kinect yet?

floe commented 5 years ago

@nonlinear1 was so kind to send me an USB PCAP file of a Kinect Azure session, I will upload the file here if possible (I don't think it should contain any sensitive information). A few remarks from a quick look through the capture:

All in all, it looks like supporting Kinect Azure might possibly be easier than Kinect v2.

xlz commented 5 years ago

The official SDK is here: https://github.com/microsoft/Azure-Kinect-Sensor-SDK. It looks quite open already.

floe commented 5 years ago

Never realized that. Whoops :-)

nonlinear1 commented 5 years ago

Actually, official SDK is open except depth engine. the depth engine is responsible for depth camera. the depth engine is close. I have succefully compiled SDK in Jetson Xavier except depth engine!

nonlinear1 commented 5 years ago

I have asked Microsoft about depth engine part of SDK, they say this part is close.If we let freenect2 support for Kinect for azure, just need code depth engine part .

floe commented 5 years ago

Can only upload max. 10 MB here, external link for PCAP file: https://floe.butterbrot.org/external/KinectforAzure1.zip

nonlinear1 commented 5 years ago

@xlz Do you have a e-mail? then, maybe we can communicate with each other easily.

xlz commented 5 years ago

Ah yes, you are right.

https://github.com/microsoft/Azure-Kinect-Sensor-SDK/blob/develop/docs/depthengine.md

The Azure Kinect Sensor SDK open source project does not include the depth engine which converts raw sensor data in to a normalized depth map.

I poked around in the SDK and it seems so.

It kind of makes sense because the new Kinect's depth sensor probably added a lot of new secret sauce that Microsoft doesn't want publicized this time.

xlz commented 5 years ago

I think the easiest thing you can do now is ask them for an arm64 build, which will work for you on Jetson. They only provide amd64 builds now. An arm64 build shouldn't be too hard for them.

nonlinear1 commented 5 years ago

I have ask microsoft for arm64 depth engine .They say they have not done arm64 build depth engine. And they let me waitting. and They haven't decided to bulid an arm64 build.So maybe I mean to build arm64 depthengine by myself.

在 2019-07-25 22:05:52，"Lingzhu Xiang" notifications@github.com 写道：

I think the easiest thing you can do now is ask them for an arm64 build, which will work for you on Jetson. They only provide amd64 builds now. An arm64 build shouldn't be too hard for them.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.