Playback does not correctly end for some content

[defagos]

Description of the problem

When reaching the end of some contents we may get an error. For these same contents the player also does not reach the end screen displaying a restart button, even when playing the content normally to its end.

Relevant stack trace or log output

No response

Reproducibility

Always

Steps to reproduce

1. Open the Pillarbox demo.
2. Play the Switzerland say sorry example.
3. Seek to the end while keeping the finger down until the very end of the slider. The player displays an error (-16040).

Alternatively:

1. Open the Pillarbox demo.
2. Play the Switzerland say sorry example.
3. Seek near the end and wait until playback ends. The restart button is never displayed and the player is stuck with a non-working play button.

Library version

0.5.0

Operating system

iOS 16 and 17 beta 4

Code sample

No response

Is there an existing issue for this?

• [x] I have searched existing issues and found no similar bug report.

[defagos]

This is an AVQueuePlayer issue which arises when seeking continuously (calling seek each time the slider is moved, e.g.) with tolerance before set to zero and tolerance after to infinity.

Here is a simple example that reproduces the issue with the system native player and UI. It also logs errors and seek targets to the console:

import AVKit
import Combine
import SwiftUI

class InspectorPlayer: AVQueuePlayer {
    override func seek(to time: CMTime, toleranceBefore: CMTime, toleranceAfter: CMTime, completionHandler: @escaping (Bool) -> Void) {
        print("--> t = \(time.seconds), b = \(toleranceBefore.seconds), a = \(toleranceAfter.seconds)")
        super.seek(to: time, toleranceBefore: toleranceBefore, toleranceAfter: toleranceAfter, completionHandler: completionHandler)
    }
}

// Behavior: h-exp, v-exp
struct VanillaPlayerView: View {
    let item: AVPlayerItem

    @State private var player = InspectorPlayer()

    var body: some View {
        VideoPlayer(player: player)
            .ignoresSafeArea()
            .overlay(alignment: .topLeading) {
                CloseButton()
            }
            .onAppear(perform: play)
            .onReceive(errorPublisher()) { error in
                print("--> error: \(error)")
            }
    }

    private func errorPublisher() -> AnyPublisher<Error, Never> {
        player.publisher(for: \.currentItem)
            .compactMap { item -> AnyPublisher<Error, Never>? in
                guard let item else { return nil }
                return NotificationCenter.default.publisher(for: .AVPlayerItemFailedToPlayToEndTime, object: item)
                    .compactMap { notification in
                        notification.userInfo?[AVPlayerItemFailedToPlayToEndTimeErrorKey] as? Error
                    }
                    .eraseToAnyPublisher()
            }
            .switchToLatest()
            .eraseToAnyPublisher()
    }

    private func play() {
        player.insert(item, after: nil)
        player.play()
    }
}

Setting both tolerances to zero or infinity fixes the issue, at the expense of a different behavior of course. The issue really seems to arise when one tolerance is zero and the other not zero (no matter which one).

We should definitely report this issue to Apple. @waliid also found that forcing tolerances to both zero (not both infinity) near the end of the stream seems to work, while preserving seeking behaviors that we currently have. This is similar to the mitigations (unsuccessful in this case) we attempted to ensure that playlist behavior is correct when seeking to the end of the current item. All these issues are likely related somehow.

[defagos]

If we replace AVQueuePlayer with AVPlayer in the above code (passing an item at construction time) the issue arises the same.

[defagos]

Setting tolerances near the end of the stream to .zero works, though the tolerance applied must be large. I tried several values but had to increase the tolerance to 18 seconds at least to get stable behavior on a wide variety of content.

This value does not seem to be related to the chunk size of the content being played (in my case I considered content with a chunk size of 10 seconds, e.g.). But 18 is still an interesting value. It corresponds to 3 times the recommended HLS chunk size. Maybe a coincidence, maybe not.

Using .zero for tolerances near the time range end also preserves our seek behavior, e.g. step-by-step seeks, though of course this disables trick play at the very end of a stream. This should be acceptable, though.

It is also very likely that this measure, combined with the fact we pause playback during seeks, probably mitigates issues we previously had with playlists, see #191.

[waliid]

A workaround is to enhance accuracy towards the end of playback by adjusting tolerance parameters.

func seek(
        to time: CMTime,
        toleranceBefore: CMTime = .positiveInfinity,
        toleranceAfter: CMTime = .positiveInfinity,
        smooth: Bool,
        completionHandler: @escaping (Bool) -> Void
    ) {
        ...
        let isVeryCloseToTheEnd = ((currentItem?.duration.seconds ?? 0) - 18)...(currentItem?.duration.seconds ?? 0) ~= time.seconds
        let seek = Seek(
            time: time,
            toleranceBefore: isVeryCloseToTheEnd ? .zero : toleranceBefore,
            toleranceAfter: isVeryCloseToTheEnd ? .zero : toleranceAfter,
            isSmooth: smooth,
            completionHandler: completionHandler
        )
        ...
    }

ℹ️ The smallest working value is ~3.7~

[defagos]

Interesting information about media playlist packaging and durations.

[defagos]

The issue has been identified as a bug, likely only affecting streams supporting trick mode (we can namely easily reproduce it with Apple 16:9 basic streams or our own DRM-protected streams).

Reported under AVPlayer: Playback fails with a CoreMedia -16040 error when seeking near the end of a stream (FB13070742).

[defagos]

We mitigated the issue only for on-demand streams near the end of the time range, i.e. based on the itemDuration and time range. The testSkipForDvrInPastConditions was namely flaky if we attempted to apply the same criteria in all cases, which was also superfluous for DVR streams.