Support long playlists

[defagos]

As a user I want the player to avoid loading data unnecessarily and to have good performance, even when many items are loaded.

Acceptance criteria

• Playlists with a large number of items are well supported (10^2, maybe even 10^3 items).
• Item loading is efficient and does not unnecessarily load content (metadata, playlists).
• Playback start time is not delayed in the process (especially when a single item is played) and gapless playback works as before.
• No performance issues or hangs are experienced. For example initial item playback, playlist mutation or navigation between items is fast (no hang when switching between items).

Tasks

• [x] Add player immutable property to control how many items are preloaded.
• [x] Trigger asset retrieval pipelines lazily in PlayerItem, depending on the current position and preloading length (only a few neighboring items after the current one).
• [x] Ensures artwork retrieval is made only for items that need it.
• [x] Check that the behavior is fine over AirPlay (hopefully even better than before). ~Check resource loader delegate support and if an id is still required for a stable result (RSI content can be used since a token is always retrieved, though unnecessarily).~ (still useful)
• [x] Fix existing bug preventing a failed URN-based item whose media composition was not loaded from being reloaded (all preloaded items must be reloaded so that gapless playback is guaranteed):
  • Disable the network.
  • Open a URN example. The player displays an error.
  • Restore the network and try to restart playback. This never works since the publisher has failed.
• [x] Correctly display errors after retry from failure:
  • Open unknown URN example.
  • Retry. The error message disappears.
• [x] Remove prefix used to truncate playlists generated from media lists in the demo.
• [x] Make sure that the playlist examples with errors work well.
• [x] Check if we can avoid the Control Center being briefly empty when navigating a playlist fast (since items are loaded on-the-fly). Maybe inserting a dummy now playing info during the loading phase might help.
• [x] Go through our list of known issues and cleanup those which might have been fixed thanks to changes made to support long playlists.
• [x] Preserve failed item index as currentIndex in case of an error? Reuse it when replaying?
• [x] Ensure no resources are leaked (e.g. PlayerItemTrackers are correctly deinit).
• [x] Check UI refreshes.
• [x] Check analytics.
• [x] Test on iOS 16.

[defagos]

Let us wait until we have a concrete need for an app to load a bunch of items. Closed for now.

[defagos]

We have a thread explosion issue when loading content. This occurs mostly with MP3 playlists which lead to more threads being created than M3U playlists. Threads created by AVPlayer add up to those we create through Combine, and to those created by 3rd party SDKs like comScore, which likely prevent background work from progressing correctly.

The issue mostly affects the simulator or running as iPad app on macOS, though of course these use cases should not be ignored. On real devices (iPhone 14 Pro, iPhone XS, iPad Pro and iPhone SE) the issue is far less visible. In comparison to raw AVQueuePlayer behavior when several items are loaded, though, our implementation has more overhead we still have to mitigate. Here is a sample project that can be run in the simulator or on device to test AVQueuePlayer behavior when loaded with several items.

We probably need to carefully use Combine and introduce a smart look-ahead centralized strategy to load content. Of course, since we can have multiple playlists loaded in multiple players at the same time, it is likely a good idea to introduce a common PlayerItemLoader to manage item preloading for all player instances. Instead of actually performing data retrieval and AVPlayerItem creation and loading into a player, it is likely possible to queue a pending request and to execute it only when required (that means to preload the next item or to load an item when large jumps are made into a playlist). We could also imagine having a default loader and the possibility to instantiate several and provide a loader to associate with a Player or PlayerItem.

Maybe using actors / Swift concurrency could help in keeping the number of threads small, leaving some breathing room for Combine and other threads we likely have less control on (e.g. comScore).

I'll reopen this issue.

[defagos]

A few learnings from early attempts:

• The comScore thread explosion can be mitigated pretty easily by instantiating the ComScore streaming analytics object lazily. This has been already implemented separately, see #743.
• There is likely no need for a common item loader, contrary to what was written above. We can namely postpone item asset delivery by inserting a Publishers.Publish(onOutputFrom:) in our PlayerItem publisher pipeline, which we can trigger when needed (e.g. when an item needs to be loaded lazily, or after a network failure). The number of threads stay small but we still have to figure out where triggering is made optimally so that everything works well (and how many items we should preload in advance, but 2 seems a pretty good number).
• AVPlayerItem creation is not free so using prefix to only produce 2 items seems a good approach as well.

With all of the above most behaviors seem to work (some UTs fail of course but probably because we are not loading items, which is what was tested). Performance with long playlists in the simulator and on device seems fine.

Of course, if the above strategy works, it won't ensure that several player instances each preload two items. But this is likely a corner case which does not justify introducing a complex shared preloading strategy.

[defagos]

Here are a few facts and learnings (WIP).

Expected behaviors

The following behaviors are expected to work:

• Playback of a single URL-based item. Restart must be possible at the end.
• Playback of a single URN-based item. Restart must be possible at the end.
• Playback of a single failing URL-based item. Restart must be possible.
• Playback of a single failing URN-based item (e.g. temporary network issue). Restart must be possible.
• Gapless playback of MP3s provided as URLs.
• Gapless playback of M3U provided as URLs.
• Gapless playback of MP3s delivered for URNs.
• Gapless playback of M3U delivered for URNs.
• Failure of an item in a playlist must exhaust buffered items but must not exhaust the whole playlist (which can be huge).

Where preloading / queue truncation should be made

Queue truncation / preloading should be made:

• When assets change (because stored items changed or because an asset changed).
• When advancing to the next item, returning to the previous item, or moving to some index.
• When the player reaches the end for an item. The best way is to use the associated notification, not to observe currentItem changes directly, since those are triggered when updating the queue.

Remark

We must not update the queue when an item fails, otherwise lengthy playlists made of only failing items (e.g. no network) will be consumed entirely for no reason. In such cases we should just let the player exhaust the existing queue.

Preloading items

AVQueuePlayer preloads items in a different way, depending on their type:

• MP3s are preloaded immediately when in the player queue, which is one of the reasons this task exists.
• M3Us are preloaded when the previous content is nearing its end.

For this mechanisms to work playable MP3 / M3U URLs must be supplied to the player.

Remark

We experimented metadata retrieval triggered by a resource loader delegate (thus based on the player itself triggering the resource loading). This approach is doomed to fail, as in this case MP3s cannot be preloaded early, leading to a gap in the playback experience.

[defagos]

All AirPlay issues we had were fixed in the process 🍾

[defagos]

About the number of preloaded items:

• We tested constants larger than 2. AirPlay stability is not affected.
• The bigger the number, the heavier preloading is when involving audio. For this reason, and until a need is identified, we decided to fix the number of preloaded items to 2. We will open it (with dedicated warnings about large values) at a later time is a need is identified.

[defagos]

Found an issue (#764) when checking analytics visually but this issue is not related to this task.