Hotrian / OpenVRDesktopDisplayPortal

Put your favorite Desktop Window directly into any OpenVR game!
Other
449 stars 96 forks source link

Full desktop support #52

Closed bgr closed 7 years ago

bgr commented 7 years ago

Hi, my only use case for this app is having my full desktop in VR, and that's the only option that's missing 😄 I have a kludgy workaround right now that works good enough, and I saw that you mentioned you've tried full desktop capture but it worked bad. I figured it might actually work better than my workaround, so, if you still have that, can you please push that to a separate branch?

My workaround is to open OBS and enlarge its Filters window to maximum size, and have OpenVRDesktopDisplayPortal capture that window. I run that at 1.5x supersampling and it works good enough for me (i7, GTX 1070).

Additionally, would adding an option to have the overlay be curved help with performance in this case (since it'd go through that fast code path mentioned in OpenVR docs)?

Hotrian commented 7 years ago

The reason full desktop capture performs poorly is because this is currently using the GDI API. The GDI API is ancient and very slow at larger resolutions. Technically though, the screen capture script already supports full Desktop Capture using the GDI API, though again, at larger resolutions the capture rate will be very low. You would have to rewrite some stuff to make it use the full desktop instead of targeting an application, but the underlying code is already there.

Additionally, would adding an option to have the overlay be curved help with performance

No, I don't believe so as the bottleneck is the GDI API and nothing to do with my application or SteamVR in specific. The GDI API is not GPU accelerated and thus is quite slow. The new API is very fast (150FPS+ with my 2560x1440 monitor, capturing the full desktop), but unfortunately Unity only supports .Net 2.0, with some .Net 3.5 thrown in, and I cannot access the new API from Unity's .Net version.

There are some other APIs I can use, but they are either too slow or do not play nice with Unity as far as my testing has shown. Once Unity finishes support for .Net 4.6 it will be very easy to add the new API support.

Edit: API name redacted ;]

bgr commented 7 years ago

Thanks for the quick response! I'll take a look at the new API and see if I can make a separate app quickly so I don't have to use OBS anymore at least 😄.

Edit: API name redacted ;] - @Hotrian

tjhowse commented 7 years ago

My reply to the email chain didn't seem to come through to here, so I'll post it again. Sorry if this double-posts.

https://forum.unity3d.com/threads/upgraded-mono-net-in-editor-on-5-5-0b4.433541/

It looks like 5.5.0b4 enables optional support for .NET 4.6. This bodes well!

Hotrian commented 7 years ago

@tjhowse, I linked the follow up to that post :P. The problem is, I will need .Net 4.6 plugin support, and I tested their beta build but it would only allow .Net 3.5 plugins and below :(. I have high hopes that they plan to add .Net 3.5+ plugin support in the future though. Right now you cannot use external libraries built above 3.5, even on their .Net 4.6 compatible beta build.

tjhowse commented 7 years ago

https://forum.unity3d.com/threads/upgraded-mono-net-in-editor-on-5-5-0b4.433541/

It looks like 5.5.0b4 enables optional support for .NET 4.6. This bodes well!

bgr commented 7 years ago

(removed, please don't hate me <3. Comment if you really need this back - @Hotrian)

Hotrian commented 7 years ago

Thanks for that link @bgr! I've been working all day and I think I've finally got my test project to a state where I can start implementing the new API into OVRDDP :o!

Quite a few changes have to be made to OVRDDP to fully implement it, but my test project can currently push up to 500FPS on my system cloning my 1080p monitor (I literally score a 0 in the SteamVR Performance Tester :P; FX-6300 & R7 265). The latency is low enough that the lip sync feels spot on. My test project also uses (on average) only 4% of my CPU. That doesn't account for any of OVRDDP's overhead, but most (almost all) of OVRDDP's overhead is due to GDI's ancient capturing.

Edit: API name redacted ;]

Hotrian commented 7 years ago

This is now in OVRdrop, which is the closed source Steam Release of OVRDDP. Since I will continue to incorporate feature requests from OVRDDP into OVRdrop, I am closing this issue now.

Currently, the Steam Release only allows for capturing one monitor at a time, but support for full desktop will come in a future release.

If you are interested, you can find OVRdrop on Steam here.