search

microsoft / Windows-universal-samples

API samples for the Universal Windows Platform.
MIT License
9.49k stars 7.98k forks source link

CameraStreamCorrelation unstable and slow on Xbox One #771

Closed erith835 closed 6 years ago

erith835 commented 6 years ago

Hello, I am evaluating the possibilities of porting a Win8.1 Kinect app to UWP so that it runs on Xbox One. The CameraStreamCorrelation sample works well on my developement PC but not at all as well on the Xbox:

If I turn off the "Depth Fading" I get occational flicker and "Unable to acquire frame" messages.

If I turn on the "Depth Fading" I get very low FPS and about 15 "Unable to acquire frame" messages per second. The video also frequenty freezes for longer periods.

Is there anything you or I can to to make the CameraStreamCorrelation sample to work better on Xbox One?

Thanks, Erik

I have compiled the sample in "Release" mode for "x64" with "Remote Machine" selected.

PC develoment platform: Windows 10 Pro, Version 10.0.15063 Visual Studio Community 2017, version 15.4.0 Target platform: 10.0.15063.0 Platform Toolset: Visual Studio 2017 (v141)

Xbox One (not Xbox One S, not Xbox one X): Rebooted in dev mode OS version 10.0.16299.3036

oldnewthing commented 6 years ago

I've asked the feature team to investigate.

yl-msft commented 6 years ago

Hi, erith835, The "DepthFading" mode requires DepthCorrelatedCoordinateMapper API, which doesn't yet perform well on XBoxOne UWP platform with Kinect due to lack of CPU resources. We are evaluating perf improvements for this API on wider range of CPU/GPU support in future Windows releases.

jbuschdev-msft commented 6 years ago

Hey @erith835

We've developed a new sample that demonstrates how you can use the existing Windows::Media::Capture APIs and utilize a GPU based algorithm to perform similar (and alternative) calculations. We've tested this sample on an original Xbox One and found it takes ~2ms to execute the GPU work. You can find the new sample over here: CameraStreamCoordinateMapper

The CameraStreamCoordinateMapper sample has 2 modes for rendering: a "fixed" camera perspective, similar to the CameraStreamCorrelation sample, which visualizes from the perspective of the target camera. And then it also has a new "free-roam" camera which let's you visualize the depth data from different perspectives. Here is where the sample determines what mode to use. If it's in "free-roam" mode, then it will render directly from the virtual camera's perspective, otherwise it will render from the perspective of the physical camera.

The core algorithm executes here. And we acquire the resulting image from the algorithm here.

Hope that helps.