Transition between media tracks without losing media focus (Implicit Media Sessions)

[richtr]

This PR allows web apps to implement playlist-like functionality without losing media focus and without causing 'glitches' in media control interfaces during track transitions.

This PR sets out a strawman proposal for how a focused media element can maintain its media focus during a track transition and/or transfer its media focus to another focusable media element of its choosing in a seamless manner. Track transitions may happen as a result of previous or next events being fired toward the current focused media element. Track transitions may also occur when an ended event is fired toward the current focused media element. If a web app chooses not to handle these events then media focus can only be lost in the usual ways (i.e. when the current track ends or another media element forcefully takes over as the focused media element as explained in the README).

By introducing a simple check immediately after previous, next or ended events have been handled by the focused media element the current page can transition to the next or previous track of its choosing without losing its media focus and without terminating its current OS-level media session privileges.

[richtr]

I also wonder if the MediaSession proposal needs a similar mechanism. Particularly when it is used to do 'stateless' track transitions as shown in the code example in MediaSession.md.

Pinging the usual suspects (@jernoble @foolip @marcoscaceres) for words of wisdom.

[foolip]

What do Android and iOS do when a single track has played to completion? Do the lock screen controls go away and media keys stop doing anything? I think that an app can keep focus and e.g. play from the beginning in response to a play button, which is what HTMLMediaElement does when you play() while ended.

The requirement to call play() on a media element in the event handlers sounds potentially annoying. While the API should allow you to preload, it's not crazy to e.g. fetch the next track URL over XHR when needed.

I think this API would actually work in the cases I care about, but I'm a bit nervous about requiring synchronous handover of audio focus. My thinking with MediaSession was that a session would simply remain active indefinitely, but can be explicitly ended or, of course, inactivated when another session becomes active.

[richtr]

My thinking with MediaSession was that a session would simply remain active indefinitely, but can be explicitly ended or, of course, inactivated when another session becomes active.

If this is true then it suggests that MediaSession is relying on 'implicit media sessions' also (i.e. it's not concerned with defining transitions between in-page media elements or tracks). The same principle can apply equally to a solution built directly within HTMLMediaElement without needing this PR.

The requirement to call play() on a media element in the event handlers sounds potentially annoying.

'invoking play()' on a focusable media element may not have been the right terminology here.

It was simply intended to signal a 'bridge' in retaining media focus during a media track or media element transition. Based on your feedback I'm not sure a 'bridge' is at all necessary to have after all (see next section).

I think this API would actually work in the cases I care about, but I'm a bit nervous about requiring synchronous handover of audio focus.

I can understand that. If, a focused media element will retain its media focus after it has played to completion then I guess this kind of check is defunct. A track transition can happen when previous, next or ended events are fired (or at any other time) and any transitions between media content in media control interfaces is going to remain smooth. It remains up to each web page to decide if it wants to support those transitions by handling such events though.

While the API should allow you to preload, it's not crazy to e.g. fetch the next track URL over XHR when needed.

True.

What do Android and iOS do when a single track has played to completion? Do the lock screen controls go away and media keys stop doing anything?

On iOS, lock screen controls disappear if I actively stop the currently focused/playing media and I then lock the device. Interestingly, these media notification controls still work after the content has been stopped, the phone has been locked and unlocked and I access the notification controls again. That suggests some persistence to receive remote control events until something else grabs them or until the media content's related app 'dies' but that media focus is handled in a subtly different way between lock screen controls and notification controls. I'm not sure if we need to capture that in any standards proposal but it's interesting as an implementation strategy.

I think that an app can keep focus and e.g. play from the beginning in response to a play button, which is what HTMLMediaElement does when you play() while ended.

To confirm on iOS, if media content plays to completion then hitting the 'Play' button in the media notification controls will restart the content from the beginning (in the same way it works in-page).

[richtr]

I'm tempted to scrap this PR if we don't need to 'bridge' during media transitions in order to retain media focus.

Also, it feels like web pages can handle previous, next and ended events to transition between media tracks in a playlist-like way without having to add any additional bridging checks (as this PR was proposing).

Just, on handling a previous, next or ended event:

1. (optionally) set up your next/previous track's src within any focusable media element or seek within the current focused media element,
2. (optionally) set up some metadata like title and poster on that focusable media element and then,
3. start playing out that element using .play() or by having .autoplay set to true to grab the media focus, thus becoming the focused media element and thus at this point then reflecting any focused media element state changes toward all connected media control interface(s).

I doubt we really need anything more complicated than that.