search

clarkezone / anglehosting

Experimenting with different angle hosting mechanisms
9 stars 2 forks source link

Off topic but ... #3

Open Noemata opened 4 years ago

Noemata commented 4 years ago

Since you are among the best uwp rendering gurus at Microsoft, I was wondering what thoughts you might have about a somewhat similar rendering problem but in a very different context.

I've been toying with how best to render a XAML surface and have it displayed within a Unity texture. For now I'm mostly interested in the rendering side of things and not worrying about input just yet. What would be the best approach from a performance perspective?

I would like to leverage XAML for the many complex 2D panels I'll need in my Unity app. XAML is so much more efficient for 2D work and has such a rich API, it seems wasteful not to leverage what's there when running on a Mixed Reality device.

I'm also curious why XAML surfaces aren't a thing in any of the WinRT 3D API's currently available from Microsoft?

Thoughts, ideas, suggestions?

mediabuff commented 4 years ago

You may try Xaml islands

Noemata commented 4 years ago

Interesting. Not sure if this is a clue just yet. Are the XAML island underpinnings available on the HoloLens 1 and 2? If so, I can see that working nicely. Given the HoloLens 2 is running on Core, it seems less likely it would have those bits? I presume this would most certainly work with desktop MR?

mediabuff commented 4 years ago

It's desktop only - for hosting XAML content in Win32 Windows. You need a HWND.

Conversely, you may want to invert the problem. Make UWP/Xaml as your primary host and draw your game content onto SwapChainPanel.

Noemata commented 4 years ago

Sorry mediabuff, I thought you might have had some specific insight into under the hood plumbing. This is a tricky problem. I've already tried 3 different approaches that work, just that I'm not happy with the perf level as they all involve shuffling data across threads or processes and are not directly tied to the Unity texture. So I'm probably copying the same data around at least three times and likely more for two of the approaches I've tried.

Noemata commented 4 years ago

FYI, the best perf so far has been derived by leveraging this approach:

https://github.com/shweaver-MSFT/MediaPlayback

mediabuff commented 4 years ago

Thanks for sharing the links. Seems to me a glorified WRL wrapper for MediaPlayer winrt api.

Much of that has been superceded and can be simplified using cppwinrt.

Could you please share what other approaches you tried ? Any pointers.

Noemata commented 4 years ago

(1) UWP Screen Sharing APIs + DirectX. (2) MediaFoundation + Screen Sharing APIs, and (3) DirectX approache I can't mention publicly. There's no "modern" example out there of how to do this. There is a Unity app that uses the older DWM API, from what I recall. Anyone that has done it isn't sharing.