fluid-lab / gamepad-navigator

GSoC 2020 project
Other
6 stars 10 forks source link

Improve handling of full screen media. #142

Closed duhrer closed 4 months ago

duhrer commented 5 months ago

Media elements like <video/> content often provide full screen controls. See this example.

When a media element is full screen, there are two key issues that the gamepad navigator could handle better.

First, the focusable elements are not limited to the fullscreen element, so you can end up having to navigate quite a ways to get back to visible controls without any onscreen feedback as to your position. If we can detect that we're in full screen mode and which element we're within, that should become the temporary context when searching for tabbables until we exit fullscreen mode.

Second, since our modals are part of the larger page, when a media element is in fullscreen mode, our modals are no longer usable. One solution would be to relocate our modal container as needed, so that all controls are available in fullscreen mode.

Another approach would be to detect and break out of fullscreen mode as needed, including:

  1. When opening the action launcher.
  2. When editing a form field that uses a modal (select boxes, text fields, number inputs).
  3. When we are asked to send an escape key while in fullscreen mode.

Although the second approach seems easier to implement, it seems more jarring for the user.

duhrer commented 5 months ago

I ended up updating the mutation observer to track video elements on the page. When they are resized (which happens when entering/exiting fullscreen), a model variable is updated to keep track of whether we are currently in fullscreen.

The modal manager has a model listener that attaches itself to the fullscreen element if fullscreen is active, or to the document body if fullscreen is not.

I also updated the updateTabbables invoker to check whether we're in fullscreen and to limit "tabbable" searching to the fullscreen element if there is one.

There needs to be some testing of this in combination with a fullscreen window and the new commands to enter/exit fullscreen mode (see #141).

duhrer commented 5 months ago

If an element is being displayed fullscreen, both of the actions added in #141 perform as expected:

  1. Attempting to enter fullscreen (again) vibrates the controller.
  2. The user can exit fullscreen.