Add ability to access XRControllers as Gamepads in fullscreen mode.

[NathanaelA]

Is your feature request related to a problem? Please describe. The only way currently to access XRControllers gamepad information is to requestSession with immersive-vr / ar. This creates a complexity for moving 2d content that is driven by Gamepad input into the Wolvic browser.

Describe the solution you'd like When entering fullscreen, and the developer has used either "getGamepads" or added the event "gamepadconnected" a new extra button appears on the bottom of the navigation bar that can be clicked which switches the XRControllers into being traditional gamepad's. Using the menu button on the controller triggers the gamepaddisconnected event and puts the XRControllers back into being the normal mouse emulation.

Describe alternatives you've considered N/a

Additional context This would allow any 2d or web content that is driven by gamepads to work properly with the natural gamepads on the XR controllers that already have buttons and axis's.

[svillar]

I am a bit confused here, when you say fullscreen mode you mean while watching a video in immersive mode? Not sure about what you exactly mean.

I ask, because as you said, when we're in immersive mode, WebXR spec already defines how the mapping from the controllers to the Gamepad API should be, and both Gecko and Chromium do implement that correctly.

[svillar]

Ping reporter, see previous comment.

[NathanaelA]

I am a bit confused here, when you say fullscreen mode you mean while watching a video in immersive mode? Not sure about what you exactly mean.

Full screen is when you use, someElement.requestFullScreen() in JavaScript, typically it is done to a video or a canvas object. In my case I'm using a canvas object. Wolvic does full screen the canvas properly, but the issue is access to the touch controller gamepad & buttons, which I need access to and is the root issue.

I ask, because as you said, when we're in immersive mode, WebXR spec already defines how the mapping from the controllers to the Gamepad API should be, and both Gecko and Chromium do implement that correctly.

The issue is that yes, I can use a requestsession with immersive-vr to get access, but that is a very high bar to jump to just get access to the touch controller's buttons/gamepad when you don't need anything from the immersive-vr session beyond the controllers. This creates a massive amount of extra code to handle all the extra blitting and math from the source web canvas into the immersive session's canvas.

Actual use case:

If you go to my site that I'm in the process of building; https://gamers.center, you can see exactly what I'm trying to do. Running the games in a 2d canvas full screen is awesome, but without a way to get the built in VR controllers you can't use the site in VR. This is the Crux of the issue, standard web site, but I need access to the VR touch controllers.

I HAVE done the extra work required to allow it to work in VR. So, if you use an Oculus w/ the Oculus browser you can use the site and play any games I've currently spent the time creating the gamepad mapping for in VR, however for most developer doing something like what I'm had to build is massive overkill that this feature request would easily solve and would eliminate having to deal with any of the WebXR spec for these types of use cases.

Also at this point unfortunately, Wolvic is still broken even with all the extra WebXR work because Wolvic does not support layers, specifically QuadLayers ( #992 ). So even though I did come up with a way to get access to the controllers, Wolvic's other issues still make my site unusable on all VR devices using Wolvic currently.

Feature Request/Design:

After discussing the issue with both the web XR spec maintainers and several others in the WebXR discord channel, the basic consensus is that without changing the spec -- the recommendation is that the Web browser actually implement a NEW clickable button that is added/enabled on the browser UI when the browser has a requestFullScreen activated that will when clicked allow the touch controllers to be used as a gamepad, disabling ALL default behavior of the device except perhaps mouse tracking. [i.e. pointing on the screen allows me to get the current mouse position via the mouse move events], it would trigger the normal "gamepadconnected" event, and show up as a normal gamepad.

Tapping the Oculus or menu button would break out of the gamepad (act like a gamepad disconnect event) and return to the normal mouse emulation/default behavior. This eliminates any spec issues and allows a much lower bar to get access to the gamepads when you don't need anything from the "immersive-vr" session or anything from WebXR.