search

amwatson / CitraVR

Port of the leading 3DS emulator, Citra — designed for playing 3DS homebrew and personal game backups in 3D on the go with your Quest.
GNU General Public License v3.0
707 stars 61 forks source link

Super Immersive Mode #60

Closed DrBeef closed 7 months ago

DrBeef commented 7 months ago

So this is the full history that implements Super Immersive mode. Annoyingly the earlier commits do a bunch of stuff that was subsequently backed out, but I don't have the git-fu to make things a bit cleaner, and I couldn't squash as much as I wanted to, apologies. I hope you are able to do something with this PR! Worst case scenario I'll simply merge all the changes in a single commit by hand.

Super Immersive Mode Configuration

So there's some extra configuration around this mode, firstly because it is currently two additional immersive modes (two "profiles") the immersive mode config is an integer setting once again.

Profile 1 is only for Ocarina of Time and Majora's Mask - There may be other games that work with this that were developed using the same 3DS "engine" that these two games use; maybe someone will find them. Majora's Mask is probably the best example of this approach. Profile 2 is for pretty much anything else, provided you know the register offset (which is another bit of config explained below).

For MK7 and Super Mario 3D Land the register offset is 90. It will be different for other games (Asphalt is 8). People will no doubt find the register offsets for other games. It is possible to set (by editing the config file) the immersive mode to "92" or "93" and then by tapping the right tumb-rest it will cycle through all register offsets, so it is possible to identify the correct one yourself for a game, but at the moment that doesn't get displayed anywhere so without having a debugger attached people won't really be able to do this themselves (unless they count the number of times they clicked it). "92" will allow you to cycle through all the registers using the Profile 1 approach "93" will allow you to cycle through all the registers using the Profile 2 approach

I have also added a positional scaling factor and a game scaling factor as some games will need larger positional factor values than others so the pos factor goes from 0-40 and the game scaling factor is 1x, 10x, 100x, the two then multiply to give a very wide range of values. I expect the community will figure out good values for these on a game-by-game basis.

Using It..

As long as the right profile is set and the appropriate register offset for the game is set, it should just work, however there are a couple of things to note..

For regular immersive mode the lower panel is now hidden and you need to look downwards (not too far) for it to show, this is just to prevent it being a distraction. For Super Immersive the lower panel is permanently hidden, if you need to see the lower panel hold either controller near the HMD and it will temporarily switch back to standard two panel non-immersive mode. I have increased the 3D depth slider to allow up to 400%, because the screen is so much closer in SI mode it needs a greater depth range to convey the same 3D depth.. especially when in 1st person mode in games like MK7, so being able to go a lot higher is necessary. If it is possible to only allow this extended range when an immersive mode is enabled that might be better.

Happy to make any tweaks needed or answer any questions!

Final thing...

I just realised another change that I have not included removed the const from the GameSurfaceLayer::Frame method.. it can be safely put back in I believe.

amwatson commented 7 months ago

@DrBeef Is it ready for me to take a look?

DrBeef commented 7 months ago

@amwatson yes, I feel it is. I think any changes around the depth slider can be addressed separately.

amwatson commented 7 months ago

LGTM.

I think this is brilliant. Amazing work!

The quad-layer-approach is a great idea to make it uninvasive -- it's amazing that it could be done with so few changes to existing code.

I'd like to roadmap a single-threaded (or at least coupled) VR/non-VR renderer at some point. Two Questions:

  1. Right now, in this change, the texture view matrix update and the layer head pose update aren't guaranteed to be in-sync, right? So if we coalesced the render stages, would that improve the feature?
  2. If/when we did, would it be possible to use a projection layer instead of a quad layer? I'd love to add Timewarp.

I'm adding some in-game UI soon. I'll be sure to add some UI for the Super Immersive Mode as well.

Users are definitely going to love it -- thank you so much!

DrBeef commented 7 months ago

LGTM.

I think this is brilliant. Amazing work!

Thank-you!, very glad to hear you like the change :-)

The quad-layer-approach is a great idea to make it uninvasive -- it's amazing that it could be done with so few changes to existing code.

I'd like to roadmap a single-threaded (or at least coupled) VR/non-VR renderer at some point. Two Questions:

  1. Right now, in this change, the texture view matrix update and the layer head pose update aren't guaranteed to be in-sync, right? So if we coalesced the render stages, would that improve the feature?

Sounds like a good idea!, It would probably synchronize things a lot better for sure, however I think because many games play at lower FPS than the Quest there will always be some juddering of the image due to no timewarp being applied to the rendered game when using a face-strapped quad layer. Which leads nicely onto the next question..

  1. If/when we did, would it be possible to use a projection layer instead of a quad layer? I'd love to add Timewarp.

When I first started looking into this change I had planned to use the projection layer. However I discovered that because the swapchain is being created using: xrCreateSwapchainAndroidSurfaceKHR (for completely understandable reasons, since Citra needs the Android Surface) I was getting issues. It then transpired that you can't call xrAcquireSwapchainImage (and xrReleaseSwapchainImage) on a swapchain created with xrCreateSwapchainAndroidSurfaceKHR. Without releasing the swapchain image you get XR errors when submitting the projection layer using a swapchain image that hasn't been "released". At that point I gave and lumped for just ensuring the quad layer was fixed in front of you. However it might well be possible to use a projected layer some way, I just didn't sink any more time into that aspect of it. A projected layer would definitely improve things greatly; ATW smoothing things out a lot would make for a nicer "VR" experience.

I'm adding some in-game UI soon. I'll be sure to add some UI for the Super Immersive Mode as well.

Oh that would be fantastic, especially for users looking for elusive Register Offsets for Profile #2, or wanting to adjust stereo depth if they didn't get it right before they started the game.

Users are definitely going to love it -- thank you so much!

My pleasure!, thank-you for merging the PR and again for creating a project that I found a great deal of fun in tinkering with. Really looking forward to hearing how people get on with it too.