iOS 15 lock screen playback and PiP regressions

[defagos]

iOS 15 broke several subtle behaviors with our lock screen playback and PiP implementations. This must be investigated and fixed if possible.

Issue type

Incorrect behavior

Environment information

• Library version: Tested with 6.1.4, but probably any recent tag is affected.
• iOS version: 15
• Device: Physical

Reproducibility

Always reproducible

Steps to reproduce

Scenario 1:

1. Open the Inline demo project
2. Enable Det. locked behavior.
3. Start playback
4. Lock the screen

iOS 14 (expected behavior): Playback continues iOS 15.0 / 15.1b2 (bug): Playback pauses

Scenario 2:

1. Open Letterbox demo project and play a video
2. Detach the Letterbox view
3. Leave the app

iOS 14 (expected behavior): Playback continues in background iOS 15.0 / 15.1b2 (bug): Playback switches to PiP

The second scenarios could probably be implemented in the inline demo as well.

[defagos]

I investigated the lock screen behavior:

• The player seems to be paused by the system (AVPlayerMultitaskSupport appearing in the call stack, probably new) when the app enters background. Previously we waited a bit to detect whether the phone was locked, but the pause made by the system occurs before we even get to our lock test code.
• We still need a delay to check whether the device lock status when the app did move to the background, so removing the delay is not an option.

I fear we might have to drop lock screen playback if this is not something that Apple fixes in a later iOS update.

I am not sure the PiP issue is related, but this needs further investigation.

In any case I recommend we wait a bit to see if something is fixed in later iOS 15.1 betas before we remove behavior which was working.

[defagos]

I saw this message logged to the console when setting the player layer to nil when entering background:

2021-10-11 08:50:36.109548+0200 SRGMediaPlayer-demo[1141:197212] [Common] -[PGPictureInPictureProxy (0x109a0af60) _updateAutoPIPSettingsAndNotifyRemoteObjectWithReason:] - Acquiring remote object proxy for connection <NSXPCConnection: 0x280228500> connection to service with pid 60 named com.apple.pegasus failed with error: Error Domain=NSCocoaErrorDomain Code=4099 "The connection to service with pid 60 named com.apple.pegasus was invalidated from this process." UserInfo={NSDebugDescription=The connection to service with pid 60 named com.apple.pegasus was invalidated from this process.}

This might be what explains the lost PiP overlay.

[defagos]

I could find a way to solve both issues.

Lock screen issue

Starting with iOS 15 the system seems to consider whether a layer is attached to the player at the time the application moves to the background to decide whether the player must be paused or not. Our code had to detach the layer after a while because we need to determine whether the entry is due to the screen being locked or the app sent to the background.

To solve this issue I inverted the operations we made by first detaching the layer from the player when entering background (thus providing us more time for the decision), then reattaching it the conditions require it. When reattaching the layer in background the system will automatically pause the player.

PiP issue

Starting with iOS 15 the player layer readyForDisplay property is set to NO when locking the screen while the PiP overlay is displayed. On iOS 14 this was the case when locking the screen with the player layer normally displayed by the app, but not when doing the same with only the PiP layer displayed. I guess this might be a bug fix which makes sense.

In such cases, though, we were destroying the picture in picture controller. By discarding the corresponding code we fix the issue, without any negative impact IMHO. What is important is that the PiP controller is created when the player layer is ready to play, but afterwards we can just keep it around. I also checked deallocation and there is no leak associated with this change.

The fix also requires not detaching the player layer when the app moves to the background.

Tests

I could verify that both these fixes work fine on an iOS 14.8 device, but I don't have devices running older OS versions at the moment. I am confident the fix should work on earlier iOS versions as well but this should be tested.

Remark

This sadly does not fix #106 since the PiP leak is related to AVPictureInPictureController allocation, which is not changed here in any way.

[defagos]

Available for review on feature/ios15-issues. A few remarks:

• The behavior was tested on iOS 9, 12, 14, and 15. The code was updated in a way that the lock screen code path is affected, which ensures that other cases are not changed. We can therefore be confident everything works as expected on iOS versions for which we don't have any test device.
• The code was also tested with the simple example on tvOS (by tweaking the controller behavior directly to check detached behavior).
• Removing player pause calls was considered as they seem redundant with the system behavior, but those are actually essential before iOS 12.3.