UWP Driver

[Noemata]

Please create a UWP accesible driver that is either compatible with or extends on the existing Kinect driver for UWP.

It's a drag to see no UWP support for such a cool product.

[wes-b]

Thanks @Noemata! We are tracking these feature requests over at https://feedback.azure.com/forums/920053-azure-kinect-dk. If you have not already, please pile on to the UWP support topic https://feedback.azure.com/forums/920053-azure-kinect-dk/suggestions/37615828-uwp-support.

[axodox]

I looked into this, I think it would be possible to write an UWP driver based on the https://github.com/Microsoft/Windows-driver-samples/tree/master/general/SimpleMediaSource sample app and this page https://docs.microsoft.com/en-us/windows-hardware/drivers/stream/uvc-camera-implementation-guide.

• The frist presents a software based camera device which is UWP supported, also describes how to do sensor cams.
• The second page describes how to make sensor groups and supply camera intrinsics.

[Noemata]

@axodox, UWP already has a fairly capable Kinect driver for the older generation device. This driver could easily be extended to support newer generation of tech since we are only looking at higher data rates and better resolution as being changed, everything else is the same. In fact, the older Kinect has more features that are supported by the existing UWP driver.

It's baffling that Microsoft doesn't support it's own tech in this way. Self inflicted wounds that limit adoption. Writing a UWP C# app for the old Kinect is trivially easy once you get past the documentation desert. Not so much in C++.

[Noemata]

@wes-b , if Microsoft made the existing MediaFoundation driver source code available, this issue would go away. Eventually, governments will mandate that software companies open source technology that is no longer supported by a given company exactly because of the scenario we have here. It's behavior such as this, on the part of Microsoft, that makes the company look bad despite all the recent good will associated with Microsoft Open Source initiatives. Microsoft is at the one foot in, one foot out stage. Jump all the way in, it's not so bad.

We don't expect Microsoft to have to make all the investments in these things. But the company should not be poised to prevent adoption either, just because it can.

[axodox]

@Noemata Yes I find it unfortunate too that there is no UWP support, however I can understand that this device is intended more towards the enterprise and factories where they are unlikely to use UWP. Our case is very special since we use UWP since our app targets the HoloLens, but we have created an AR view with Intel RealSense with the UWP API. The app is written C++/WinRT and runs a custom DirectX 11 engine.

What I intend to do is to create a driver which provides the sensor data to UWP the same style, sensor group: color + depth (we do not use IR, but could be added). That being said it will only be open-source if I do it in my free time, otherwise the legal team likely say no. In any case I think based on the examples above it should be doable in a day or two (a basic level, with no settings and only supporting a fixed depth and color mode). They already add the color image, and depth only requires some media description changes, the sensor grouping can be achieved by editing the inf (there is example by MS). Anyway if I do it open-source I will let you know.

[Noemata]

@axodox, I may be interested in helping your effort along. I've done driver level work and see your approach as being feasible. Please let me know if you'd like to collaborate? I would be particularly interested if this became an officially registered driver. FYI, the company you work for purchased some of my 3D tooling a few years back.

[axodox]

@Noemata Collaboration would be definitely welcome on my part, although right now I still need to figure out how that could work in relation to my job. If I do this on the clock, it might be lengthy to get approval to opensource. If I do it at my free time I can open-source it, but then I might not have that much time to do it. In any case ideally I have this to work at a basic level in a few weeks for a demo, and by few weeks I mean at max 2-3 days of dev time as I have other stuff as well.

So right now I do not intend to build a fully featured driver with all bells and whistles, but rather I just need to get the app working with this new Kinect (previously we used an Intel RealSense camera with it). This means that as long I can get it to return color and depth image at a fixed resolution and with hardcoded settings, and fill out the camera intrinsics and some other metadata for processing it is good to go for me. So what I am willing to provide is this level. Obviously it would need some more development to support IR and settings, different resolutions or even skelatal models etc.

Today I started experimenting with the sample driver in my free time - I did a lot of Windows C++ development, a lot of low level stuff too, but not drivers so this is a new area for me. In any case I set up a VM, and deployed the sample driver to it. Then edited it so I can make it provide both depth and color frames (empty frames for now) which works with my UWP app with my existing code using the WinRT Media Capture APIs. Next one is to connect it with the Azure Kinect and make it return real data, then add the basic metadata, sign and package it and for me it is good enough.

[axodox]

FYI, the company you work for purchased some of my 3D tooling a few years back.

Really? It is a small world indeed ;) I definitely hope to visit Canada again, I have spent 6 months in Montreal at the HQ, but nowaday I do most of my work with Germany and have not visited it for a few years (and of course since corona very few travelling).

[Noemata]

@axodox , I never miss a chance to visit Montreal! Please let me know if you need a hand. Add me to the repo you're working on when you are ready. I've been looking for an opportunity to move some of the MF C++/CX bits I have over to C++/WinRT.

[axodox]

@Noemata Sure I will tell you if I have a proof of concept implementation working. C++/WinRT is quite good, we had using it for years now for a Hololens project, I will likely replace the WRL stuff in the driver sample too to the winrt base.h alternatives at some point.

[axodox]

I decided to do it in my free time, so I can open source this thing. This repo is here: https://github.com/axodox/Azure-Kinect-UWP-Adapter, however there is not much to see yet. After initial testing I started implementing the actual kinect support, and also moving the example from WRL to C++/WinRT and latest C++. I will tell you once I have the color and depth frames coming from the kinect - as I mentioned I already did a test with procedurally generated data, so it should only be a question of when (probably next weekend when I have more time again).

[Noemata]

@axodox, very nice!

[axodox]

I also tested driver installation on a regular machine, as I remembered that it could be a problem with custom drivers unless you buy a cert. Fortunately since this only requires a user mode driver so I can self sign it and if the certificate is installed it allows regular machines to use it without disabling secure boot etc.

[Noemata]

@axodox, that is correct.

[axodox]

I have not have much time this week yet, but I managed to finish the move from WRL to c++/winrt, and figured out the class activation architecture and now it loads properly. I also implemented framerate / format / resolution selection for depth and color streams, and tested them with the WinRT Windows.Media.Capture APIs from an UWP app. The next step is forwarding actual Kinect data streams to the MF APIs and the UWP app.