Godot OpenXR can't target Windows Mixed Reality OpenXR runtime because it relies on DirectX

[rcoppy]

Hi there, just wanted to say you've been doing fantastic work!

Fyi--this OpenXR addon doesn't work with the Windows Mixed Reality platform out of the box:

ERROR: OpenXRApi::initialiseInstance: OpenXR Runtime does not support OpenGL extension!

Because WMR only support DirectX 😢

But! Preliminary reading found that someone as part of Google Summer of Code made an OpenGL to DirectX compatibility layer for Blender's OpenXR loader, and it actually (mostly) worked:

https://devtalk.blender.org/t/gsoc-2019-vr-support-through-openxr-weekly-reports/7665 https://devtalk.blender.org/t/gsoc-2019-core-support-of-virtual-reality-headsets-through-openxr/7614/15

The project is, however, 2 years old and skimming his last update parts of the workaround seem to have broken because of API changes.

Nonetheless, if WMR compatibility is a priority, @julianeisel 's foundational work is likely helpful.

Cheers!

[BastiaanOlij]

Yup it's a sad sad thing, I so want to see Godot running on a hololens.

It's mostly a matter of people not having the time to spend on it. Godot can be compiled using Angle so it runs on DirectX but every since Microsoft abandoned Angle it's been a pain in the ..... It requires this in the UWP build so Godot can run on XBox, and incidentally also on Hololens :)

Google has now picked it up and there have been a few trials to get Godot running with that version but I don't know the state of that. In theory if you compile Godot with Angle and make the necessary changes to this plugin it should work.

But someone has to find the time to do it. I will get to it eventually but right now there are other things higher on the priority list.

[rcoppy]

Hi again @BastiaanOlij !

I just thought of something out of the blue that is sort of a workaround to this problem:

In Unity's OpenXR module, in preferences you can force the loader to pick the Steam runtime, even when the WMR one is also present on the system. If you have the SteamVR-to-WMR translation layer utility installed, you can use steamvr openXR on a WMR device.

Is it possible in the Godot openXR settings to force using steamVR over WMR? Is an extra step but would at least make godot usable for my test cases. Thank you!! -Alex

[BastiaanOlij]

Hey Alex,

Well we don't have control over whether people install the WMR component for Steam, though they likely will because thats the only way to play most of the SteamVR games when you have a WMR headset.

The official take from OpenXR is that it should be the user who sets the default OpenXR runtime but you can override it. If you enable the OpenXR plugin in your project settings you'll see a dropdown menu is added to the editor that allows you to select teh runtimes. As there is no way to detect which runtimes are installed this only works for the main runtime types and only when they are installed in their default locations.

The script for this dropdown is in the plugin so you can see how this works but basically its all about setting an environment variable before Godot starts.

[rcoppy]

Ahh makes sense. thanks for the info!! 🙂

[GeorgeS2019]

@BastiaanOlij FYI DirectX to Vulkan: is this relevant to bring Godot support for DirectX (Hololens OpenXR) ??

Discussion references: https://synappsis.wordpress.com/2018/03/22/dxvk/ https://steamcommunity.com/sharedfiles/filedetails/?id=2461019058

https://github.com/doitsujin/dxvk/

[BastiaanOlij]

Wrong way around I'm afraid @GeorgeS2019 , we need Vulkan to DirectX so we can run Godot 4 on systems that only support DirectX.

[GeorgeS2019]

Here is the link

Not all features of the Windows 10 compositor is available in OpenGL or Vulkan, specifically scRGB float 16 HDR and transparent windows.

It seems it is possible to use a DirectX swap-chain and use Vulkan-DirectX interopability to get Vulkan to render in a frame buffer and without cost 'copy' the frame buffer to the directX swap-chain (like a c++ move). Care must be taken to somehow copy back the directX swap-chain image to the vulkan frame buffer to allow for scissored rendering.

Here is an excerpt from a forum post I found:

P.S. If you want to extend HDR to work in Vulkan, the consensus among all of the AAA games I have reverse engineered to date is to use a DXGI SwapChain + D3D11 Immediate Context to perform Vk / DXGI interop (copy the Vk swpachain's backbuffer to DXGI for Presentation).

It sounds weird, but if you match up the SwapChain image format and dimensions there is actually no overhead introduced by doing this (under the hood it's just a shared surface regardless which API is touching it). Just make sure you have cleanly released all references to said buffer after Present -- there's one implicitly held by the DWM until VBLANK and the D3D11 Immediate Context may hold one unless you call ClearState (...) and Flush (...) prior to Present. You get finer grained control over pretty much everything (latency, scaling, colorspace, G-Sync, ...) if you use IDXGISwapChain*::Present (...) to present final frames for scanout, so this interop is a blessing in disguise for HDR users.

[BastiaanOlij]

Hopefully someone will start to look into this for Godot 4 but I won't have time for it myself. There has been talk about doing a proper DirectX backend for Godot 4 once 4.0 has been released but it will likely require either help from Microsoft or someone with sufficient DirectX experience to become a contributor.

The OpenXR plugin just handles the OpenXR layer, until Godot 4 works in some way, shape or form with DirectX there isn't much we can do for the plugin.

[mbucchia]

Hello,

I've written an OpenXR API layer that will let Vulkan apps run on top of Direct3D 12 support on WMR (PCVR):

https://github.com/mbucchia/OpenXR-Vk-D3D12

I've tested it with the Godot 4 Alpha 10 demo scene, and it worked well on my HP Reverb G2. This can be installed system-wide (the API layer will load automatically) or bundled with the engine (need to load the API layer with xrCreateInstance()).

This won't help you on HoloLens 2, because the GPU driver itself does not support Vulkan, so texture interop (which is the end of the line) will not help here.

[GeorgeS2019]

@mbucchia In your view, what will be needed to make Godot4 run in HoloLens 2? Do you use Dozen? Great Job!!

[mbucchia]

@mbucchia In your view, what will be needed to make Godot4 run in HoloLens 2? Do you use Dozen? Great Job!!

AFAIK the GPU driver itself on HoloLens 2 does not support Vulkan/OpenGL. The best solution is for the app to natively support Direct3D. Hacky way would be a wrapper Vulkan -> Direct3D or OpenGL -> Direct3D. As pointed out earlier, dxvk is the other way around, so no luck. There is OpenGLon12 (now part of Mesa I believe) that I looked into a while back, and it would be possible to modify OpenGLon12 to retrieve the underlying Direct3D 12 texture, and copy it the OpenXR swapchain. I have however no idea whether OpenGLon12 has sufficient OpenGL support for Godot.

[mbucchia]

I wasn't aware of this Dozen project, but it's the same concept as OpenGLon12. Looking briefly at the code, it's also possible to modify the implementation to "reach" into internals and grab the ID3D12Resource and pull it out. There are official Vulkan extensions that you can implement in order to do that (VK_KHR_external_memory) but this is going to be (much) more effort than just exporting a single function (something non standard) to retrieve the resource directly. But the latter might not be acceptable by the Mesa developers.

[mbucchia]

... exporting a single function (something non standard) to retrieve the resource directly. But the latter might not be acceptable by the Mesa developers.

Something like this in Dozen:

ID3D12Resource* getUnderlyingD3D12Resource(VkImage image) {
  return dzn_image_from_handle(image)->res;
}

This will then allow you to create a thin interop layer between Dozen and D3D12 for OpenXR, when you would just need to write the code to initialize OpenXR using D3D12, and substitute the D3D12 swapchain images returned by xrEnumerateSwapchainImages with Vulkan textures, and eventually perform a copy between the underlying D3D12 Dozen texture (retrieved with getUnderlyingD3D12Resource()) into the actual D3D12 swapchain images. Don't forget to also signal a fence to the D3D12 command queue after doing the copy.

There may also be a way to wrap a D3D12 swapchain image directly into a Dozen texture and avoid the need for a copy, but this requires more surgery into Dozen than just the one line of code.

[GeorgeS2019]

@mbucchia

Hardware accelerated Vulkan: Dozen now works in WSL2

VK_ICD_FILENAMES=/usr/local/share/vulkan/icd.d/dzn_icd.x86_64.json vulkaninfo | grep -i deviceName

WARNING: dzn is not a conformant Vulkan implementation, testing use only.
WARNING: dzn is not a conformant Vulkan implementation, testing use only.
        deviceName        = Microsoft Direct3D12 (NVIDIA GeForce xxxxx
        deviceName        = Microsoft Direct3D12 (Intel(R) UHD Graphics xxx)

[mbucchia]

Thanks. I gave some input on what can be implemented, but I don't have any intention to do it myself. I don't have spare cycles for this.

[rcoppy]

wow!! thanks so much for this, amazing work 🙂

[GeorgeS2019]

@mbucchia @BastiaanOlij

Get Godot Engine to work in Hololens through OpenXR supported by Dozen

From @mbucchia (June 29)

There are official Vulkan extensions that you can implement in order to do that (VK_KHR_external_memory) but this is going to be (much) more effort than just exporting a single function (something non standard) to retrieve the resource directly. But the latter might not be acceptable by the Mesa developers.

Please feedback: from Jesse Natalie (Aug 4)

VK_KHR_external_memory_win32 will be implemented and would allow importing/exporting D3D12 resources for interop with Dozen. I don't think it makes sense to implement custom extensions (or non-extension functions) that only Dozen could support at this time.

Discussion source

[GeorgeS2019]

@mbucchia @BastiaanOlij Godot4 Direct3D 12 Rendering Driver

[BastiaanOlij]

@GeorgeS2019 cool huh! There are still a few things to be solved but this is making WMR support and Hololens support within reach.

I'm talking with Pedro about the missing bits, I hope to have time in the near future to help him out and see if we can get WMR to work on Windows.