REQUEST: Some documentation for IVRVirtualDisplay/IVRDriverDirectModeComponent support in macOS

[TheDeveloperGuy]

Currently, according to @natbro comments in #551 IVRVirtualDisplay cannot be used to create virtual HMD in macOS, it needs to reflect an actual (supported) HMD. In order to provide (for example) wireless HMD drivers in macOS, it's necessary to create a virtual HMD that is not backed by an actual HDMI(?) headset connected to the computer.

UPDATE: it appears the mechanism is similar to Windows. It just needs a bit of explanation around how shared textures, mutexes and syncing is done.

[TheDeveloperGuy]

In order to facilitate implementing IVRVirtualDisplay on macOS, it would be useful to get some code sample/documentation on how the following (Windows driver) things are done on macOS OpenVR:

Dealing with texture handle passed to IVRVirtualDisplay::Present():

    ID3D11Texture2D *pTexture
    device->OpenSharedResource((HANDLE)backbufferTextureHandle, __uuidof(ID3D11Texture2D), (void**)&pTexture);

Locking mutex:

    // Get mutex from texture
    IDXGIKeyedMutex *pSyncMutex = NULL;
    pTexture->QueryInterface(__uuidof(IDXGIKeyedMutex), (void **)&pSyncMutex));

    // Lock mutex
    pSyncMutex->AcquireSync(0, 10));

Getting shared handle for texture created in IVRDriverDirectModeComponent::CreateSwapTextureSet:

    // Get IDXGIResource interface
    IDXGIResource* pOtherResource;
    texture->QueryInterface(__uuidof(IDXGIResource), (void**)&pOtherResource);

    // Get shared handle for texture
    HANDLE sharedHandle;
    pOtherResource->GetSharedHandle(&sharedHandle);

The texture format specified in:

    IVRDriverDirectModeComponent::CreateSwapTextureSet( uint32_t unPid, uint32_t unFormat, uint32_t unWidth, uint32_t unHeight, vr::SharedTextureHandle_t( *pSharedTextureHandles )[ 3 ] );

[TheDeveloperGuy]

@natbro @JoeLudwig Any way I can get this info? It's very frustrating trying to develop complex software without any documentation or samples (not to mention having a contributer's attention for a few moments and then losing it again for 6 months!).

Since macOS support is in beta, the texture allocation and submission process is still subject to change, as is how the texture is synchronized - we may need to have the application indicate which MTLCommandQueue they are working on to write to their MTLTexture in order to ensure read/write ordering.

Is not enough to go on (sole documentation/guidance I've managed to extract from @natbro so far). I understand everyone's busy, but what am I supposed to do here?

[TheDeveloperGuy]

@natbro @JoeLudwig Is there any reason why the requisite information for implementing a macOS SteamVR driver is being withheld?

[TheDeveloperGuy]

Only "supported" VR HMD on macOS is Vive. Is this secrecy to prevent others from making macOS-compatible headsets? If so, maybe the "Open" in "OpenVR" is not at all applicable on macOS?

[OskarGroth]

@TheDeveloperGuy I share your frustration. I feel like Mac support is almost abandoned. It's been 8 months since the launch at WWDC and @natbro's slide saying "sample code available soon". Yet here we are struggling to understand basic functionality because no sample code or documentation has been released since.

I don’t blame Nat, he’s been helpful and was kind enough to look at my code and give me some critical pointers. But I do blame Apple/Valve for making such a big deal out of supporting Mac and then dragging their feet for all this time. It seems neither party is really serious about VR on the Mac. At least not beyond enabling basic Unity/Unreal support for the Vive so they can do cool marketing.

For third party devs who want to create unique experiences from scratch (https://cindori.org/vrdesktop in my case), you're either stuck on basic implementation problems or wondering whether it's even viable to create a product that relies on SteamVR because it's such a horrible user experience on macOS. Personally I have to force quit ipcserver_vr every 3-4 launches and can't even run my VR app cold unless I first "warm up" by launching SteamVR from inside Steam...

It seems there is constantly issues happening that is just not worthy for a product/SDK of this magnitude. Right now it feels like there's 1-2 guys working on SteamVR about 10% of their time. And then one guy occasionally working on macOS support, struggling to even reach feature-parity with Windows 7 months in. There needs to be a serious increase of resources to SteamVR development to convince Mac and hardware developers to even bother.

[TheDeveloperGuy]

@OskarGroth It definitely feels likes it's being deliberately ignored for whatever reason (I know a lot of devs hate Apple for their own reasons, so maybe that's part of it). I feel like @natbro or whoever else is working on macOS driver support for OpenVR could spend 5 minutes answering my questions and I'd be on my way. Definitely going to make it a lot harder to figure it out without that, but as I'm getting increasing demand for a macOS version of my "virtual" HMD driver, I'm going to have to go ahead anyway without the information (and thereby increase the negative perception of macOS SteamVR when things don't work properly, but hey ho what to do?)

[echuber2]

Given limited resources, I'm sure it's a matter of there not being enough VR users on Mac at all. Is Valve hiring?

[TheDeveloperGuy]

@echuber2 Chicken and egg scenario. If you make it they will come etc.

[OskarGroth]

@echuber2 Whenever people say "Mac doesn't have enough VR users" or gamers or whatever, this is my go-to argument:

The Mac community has a small, but incredibly talented landscape of creative technology developers. I’m talking about people like Vade, David Lublin, Steve T-S or Brad Larson (who by the way created the first SceneKit-Oculus bridge). There is a thousand more and 5 minutes on Twitter will prove you so. People that spend their day writing code and shaders, pushing the creative buttons of the platform. Because beyond gamers there is a community of makers, dreamers, hackers, and tinkerers, that you need to cater to. These are the people that will get press. These are the people that will go viral. This is how you’ll get “A new way to see your heartbeat or whatever with the Oculus Rift” on Vice, Buzzfeed, and other popular outlets.

• http://blog.mousta.ch/post/121102629003

Enabling Mac support for SteamVR is just a win-win for everyone involved (developers, Valve, Apple, HTC, users) but delivering this broken and undocumented SDK is the exact opposite, just incredibly damaging. Developers pulling their hair, losing money, users being disappointed and turned away from HTC/SteamVR or VR completely, etc.

[OskarGroth]

I've created a Slack community for discussing VR on Apple platforms:

https://join.slack.com/t/applevr/shared_invite/enQtMzM0OTE3NDMxOTIzLWRjMTA5YzY2N2I0M2EwNjMwNDZkMmMyNmQzYzM3ODE1ZjY3NmU4MzMwYjA2Zjc4ZDNlOTZkMjNlOGNiNTg3ODI

I hope it can be useful in figuring out pitfalls and solutions that can be shared among developers when the documentation is lacking, and have a positive impact on the state of VR on the platform.

[natbro]

as i said in the slack channel, there is no deliberate ignoring of macOS SteamVR going on - it's purely a matter of prioritization and time. enabling SteamVR on macOS was a first step to seeing if there was developer interest and customer interest, and it also helped us understand the cross-platform requirements of the runtime, APIs, and rendering paths. developer interest has mostly been from Unity and Unreal developers, 360-video editing folks, and people interested in SteamVR tracking technology - supporting them has been a focus and they seem happier than this thread :) there is more to do - but there isn't really any market for developers to sell their game content or even their custom VR experiences into because there aren't adequate GPUs on the platform - that starts to change with eGPUs and it starts to change as new models come out in the future due to the nature of rising GPU performance, but nothing changes the fact that there aren't yet consumer customers on the platform, just us developers. i do want to unblock you in your product development and fix instability, but we don't need to have it perfect. try to accept a little instability right now because it is not the instability blocking acceptance, it is lack of customers and the lack of GPU power which leads to lack of content to get the cycle going. the cycle will eventually get going if consumers find eGPUs a workable solution.

yep, we're always hiring.

[TheDeveloperGuy]

@natbro "If you build it, they will come".

Is there any way to get a dialogue going that has a cadence of less than 2 months? It's impossibly frustrating to develop something where it takes 2 months to get a question (sort of) answered...