Continuous playback does not work in background

[defagos]

In background, continuous playback does not chain medias automatically. This is probably due to the fact that timers are paused in background and do not fire, and probably is a consequence of the changes made to reduce player activity in background.

Issue type

Incorrect behavior

Environment information

• Library version: 3.0.0 (probably affecting 2.0.0 as well)
• iOS version: Any
• Device: Physical

Reproducibility

Always reproducible

Steps to reproduce

1. Open the demo.
2. Enable continuous playback in the settings.
3. Play SRF audio example, seek near the end.
4. Send the app to the background. Playback does not continue. If you wake up the app, the countdown looks frozen, but if you wait playback continues automatically.

[defagos]

I investigated this issue.

When playback ends, the app is suspended since there is no active audio anymore. This is why the continuous timer does not fire until the app is woken up again.

I tried using the background processing capability and the associated API to ask for some more time. The notification is fired correctly, but playback does not start and remains paused in background.

Actually, when AVPlayer is in background, calling -play sets the player rate to 1, but the system resets it automatically to 0, preventing playback from starting.

Using AVQueuePlayer seems to work and is probably the recommended way to fix this issue. This requires support to be implemented in SRG Media Player. We will probably have to extend the existing API to support item enqueueing, maybe add a playlist API.

[defagos]

Rewriting SRG Media Player to support item enqueuing with a single controller would be a huge task. This would require us to store the media to be played with its context (initial position, segments, etc.), but also to add dedicated APIs to mange the queue itself (e.g. cancel playback of the current item and all remaining ones).

I verified that the issue we have with continuous playback is due to the fact that we are recreating an AVPlayer instance after the current item has ended (AVPlayerItemDidPlayToEndTimeNotification). This is namely how SRGMediaPlayerController is implemented.

If we use an AVQueuePlayer instead and do not destroy the player, but only insert the new item after the first has ended (the same AVPlayerItemDidPlayToEndTimeNotification), playback will continue. Note that this only works if insertion is made directly, we cannot delay playback.

Note that using AVPlayer and its -replaceCurrentItemWithPlayerItem: to achieve a similar result does not work.

I therefore recommend the following strategy to ensure SRGMediaPlayerController can chain playback:

• Use AVQueuePlayer instead of AVPlayer in SRG Media Player. Make the necessary adjustments so that the behavior stays the same.
• In Letterbox, always immediately start continuous playback when in background. Document this new behavior.

Of course, this means we don't use AVQueuePlayer in the best possible way for gapless playback, but this is not really the purpose here.

[defagos]

As noted by @pyby, the above approach assumes we readily have the URL at the end of the first media. But with Letterbox, and if the controller does not fetch the next media in advance, the URL is not known directly, and can take some time to be retrieved with a network request.

Using AVQueuePlayer is probably the right way to go, but we definitely need a better strategy to chain medias.

[defagos]

A rough draft of what we could make:

• SRG Media Player must be implemented with AVQueuePlayer. The player must be created once and for all at initialisation time, then reused with player item insertion.
• An API must be added to prepare next items to be played directly after the current one. This means providing the full context (start position, user info, etc.).
• At Letterbox level, we could implement a queue approach as well, or have a playlist service retrieve and prepare next item playback. If needed, we can perform background tasks to extend the liftetime in background, though I doubt this will be actually required (we can retrieve next media resource information while playing the previous media).

Anyway, this is no small task (and the above idea might probably need some thoughts).

[defagos]

Of course chaining should not break PiP or AirPlay playback if active.

[defagos]

Our SRF fellow colleagues found a workaround by registering a dummy remote control center command enabled at all times. Seems the command must not be the first registered, so the code should be inserted somewhere after the first play command has been registered (in the current codebase).

We should:

1. Integrate this fix in the service.
2. Build a LB demo and ensure it works on iOS 15 and 16.
3. Play some content on the device, close the player and check the command is still there after closing the player.
4. Let the device stay asleep for several hours on battery and check if the battery system reporting tool mentions a high background activity or not.

If the trick works on all iOS versions and the last answer is negative we can then deliver this workaround officially in Letterbox, but likely as an opt-in. It is namely likely that if the app does some kind of periodic work in the background (which Letterbox demo doesn't), the fact it is kept alive will make this work continue while the app seems to be dormant, possibly leading to battery drain (e.g. if network interfaces are woken up periodically, especially on mobile networks).

Note that since Xcode 13 / iOS 15 the energy log tool is not available anymore. This makes it more complicated to evaluate the energy impact of the workaround.

[defagos]

Workaround works on iOS 16 beta 3.

[defagos]

This issue is fixed in Letterbox successor Pillarbox. I will keep it open as it was one of the major reasons why Pillarbox exists in the first place, but Letterbox itself will never be able to fix this behavior as this requires the whole implementation to be built on AVQueuePlayer, which is impractical.