Media3 background player MediaItem nuances

[Manuiq]

I was amazed by the simplicity of the Media3 integration so I spent an evening trying to create a simple project with it . My setup was a MediaSessionService (with MediaSession and ExoPlayer) + Activity (that controls playback via MediaBrowser). 1) I run into a small hiccup with this part:

val player = ExoPlayer.Builder(context).build()
val mediaSession = MediaSession.Builder(context, player)
   .build()

I assumed this setup was enough at least for the case of adding MediaItems via MediaBrowser from your own Activity via mediaBroser.addMediaItems(items) for them to be passed on to the player. But turned out there it's not that easy and you need to override MediaSession.Callback interface's fun onAddMediaItems() as there is a really important comment in the source code on it:

"Called when a controller requested to add new media items to the playlist via one of the Player.addMediaItem(s) or Player.setMediaItem(s) methods. Note that the requested media items don't have a MediaItem.LocalConfiguration (for example, a URI) and need to be updated to make them playable by the underlying Player. Typically, this implementation should be able to identify the correct item by its MediaItem.mediaId and/or the MediaItem.requestMetadata."

So turns out you can't just pass the items to player because the MediaItem is not serialized in full across processes which seems reasonable when you look at the LocalConfiguration class structure and all the difficulties with it's bundling/unbundling but it's not mentioned in any way anywhere else in the doc. Maybe this is all trivial for anyone working with previous media libraries but for a newbie in media playback it wasn't that obvious. So maybe addressing such points in the developer docs/codelabs can help?

2) Also in the demos-session sample PlaybackService is described as MediaSessionService in the Manifest

    <service
        android:name=".PlaybackService"
        android:foregroundServiceType="mediaPlayback"
        android:exported="true">
      <intent-filter>
        <action android:name="androidx.media3.session.MediaSessionService"/>
        <action android:name="android.media.browse.MediaBrowserService"/>
      </intent-filter>
    </service>

but extends MediaLibraryService in the actual code class PlaybackService : MediaLibraryService() { which is a bit confusing also. Do they have the same handling intent-filter wise?

Still, the library is great and I admire the level of quality in your 1.0.0 release.

[marcbaechinger]

Thanks for your feedback! We are very happy to hear you are excited about Media3 and it's new media session integration! :)

Thanks for raising these two points where we should improve or clarify our documentation. I think a first step will be the migration guide that we have ready and that will be published as soon as we release the next beta release that is happening this week. It covers both points you are mentioning: the need for implementing Callback.onAddMediaItems() and also that adding both service interfaces to the intent filter in the manifest provides backwards compatibility with client apps using the legacy MediaBrowserCompat of androidx.media to connect against the MediaLibraryService.

I think besides the migrations guide, we should add the requirement of implementing Callback.onAddMediaItems() in this doc on android.developer.com.

So to address you points: I have added the 'documentation candidate' label to make sure we revisit how we document the need for implementing the callback. Can you give me a hint where you would have expected that documentation or better: where you looked first for answering your question about it? Would the documentation in DAC be a good place that you would have consulted/found?

Second, yes, adding the action in the intent filter is sufficient to make legacy MediaBrowserCompat clients find the service and connect to it. The MediaLibraryService will then receive legacy API calls, transform them to new API calls that are sent to the MediaLibrarService just to transform the response back to the legacy API to be sent back to the client. That happens under the hood and an app only needs to implement the new API which is the MediaLibrarySession.Callback.

[Manuiq]

Yeah, the docs in DAC seem like a great place for such points, thats the place I started with. And thanks for the quick response!

[nh7dev]

Hello,

I had the same problem and it took me forever to figure out what's wrong. The problem is that there's no mention at all in the guide that it's necessary to override onAddMediaItems. And I didn't receive a single exception or debug message telling me what the problem was. Player.Listener.onPlayerError wasn't called either.

Here's my question, code and log: https://stackoverflow.com/questions/74035158/android-media3-session-controller-playback-not-starting/

I think besides the migrations guide, we should add the requirement of implementing Callback.onAddMediaItems() in this doc on android.developer.com.

Yes that would be the best place to mention it imo.

I think it's important to fix this issue as soon as possible, it seems like a very crucial part of implementing Media3.

Thank you.

[galacticappster04]

I would like to raise this as well. I made a recent comment on this question on androidx/media:

How one will implement a lookupUriForMediaId()? Obviously, lookupUriForMediaId() could be a suspending function. This must take awhile as I need to query this from a source. The current approach, as suggested here, encourages us to load everything at once (I notice the sample project did this by building the content tree). Is this ideal? Let say the device contains 10k songs, is this even ideal to load and build the content tree this way?

If we have the content tree on a var tree : MediaItemNode, now it is easy to query as this is in memory already but I am a bit worried that it might cause some performance issue?

Also how is this/should be done in Media3?

It appears localConfiguration is being stripped during unmarshalling. When service receives the mediaitems it is missing cruicial information like the media Uri due to security/privacy reason.

So now my question is how do we recover the lost information in Media3 without building the content tree or querying the MediaSource? I have an abstraction for interacting with MediaSource and unfortunately and quite obviously it would be a suspending function but since the Player callbacks are not suspending and returns a Future this is not coroutines friendly.

[marcbaechinger]

You can implement Callback.onAddMediaItems() as described in a comment above.

[zt64]

Hi, I was also encountering this issue with being unable to set media items on my media controller. How could I implement it such that I'm able to play audio and video at the same time, if that's possible?

[marcbaechinger]

Thanks @zt64 Can you clarify a bit what didn't work and why the guidance in the comment above didn't help?

[zt64]

I found this solution which does work but only for a single URI. I wanna be able to play media using a MergingMediaSource so that I have audio and video. I'm not sure what else I can do

[marcbaechinger]

Sorry, I don't understand what you are asking I'm afraid. Being a bit more verbose can help in some of such cases.

The MediaController API is based on the Player API. The playlist API of the Player is based on MediaItems. So you can only do what the Player can do with a media item. If you want to support a MediaSource that is not supported by ExoPlayer's DefaultMediaSourceFactory out of the box, then you need to provide your custom MediaSourceFactory to convert a media item to a custom MediaSource which can be a MergingMediaSource:

1. Write a custom MediaSource.Factory that has the ability to create a MerginMediaSource out of a MediaItem.
2. Inject the new factory into the player when using ExoPlayer.Builder(context, mediaSourceFactory) .
3. Test whether this works without any MediaSession/MEdiaController integration.

Before thinking about MediaSession/MediaController, the above should work and be understood by the users that will later try to access this functionality with a MediaController.

4. Once this works with the player, move it to your media session app where you need to implement MediaSession.Callback.onAddMediaItems() accordingly to make sure your custom media source factory gets all the data in the media item that it needs.

If this doesn't help, please open a new issue, so we don't go off topic too much.

[chientrm]

For anyone coming in the future, when you run mediaController.setMediaItem(mediaItem), all information from that mediaItem is removed before getting passed to your MediaSessionService for security reasons. So that mediaItem.setUri will not work. Instead you need to use mediaItem.setMediaId. For ex:

val mediaItem = MediaItem
    .Builder()
    .setMediaId("bunny")
    .build()
mediaController.setMediaItem(mediaItem)
mediaController.prepare()
mediaController.play()

Next, you need to add a callback in your MediaSessionService to map that mediaId to a valid uri:

class PlaybackService : MediaSessionService(), MediaSession.Callback {
    ....
    override fun onAddMediaItems(
        mediaSession: MediaSession,
        controller: MediaSession.ControllerInfo,
        mediaItems: MutableList<MediaItem>
    ): ListenableFuture<MutableList<MediaItem>> {
        val updatedMediaItems = mediaItems.map {
            it
                .buildUpon()
                .setUri("https://example.com/${it.mediaId}.mp4")
                .build()
        }.toMutableList()
        return Futures.immediateFuture(updatedMediaItems)
    }
    ...
}

Make sure to register the callback when you create MediaSession:

class PlaybackService : MediaSessionService(), MediaSession.Callback {
  ...
    override fun onCreate() {
        super.onCreate()
        val player = ExoPlayer.Builder(this).build()
        mediaSession = MediaSession
            .Builder(this, player)
            .setCallback(this) // <- Set callback
            .build()
    }
  ...
}

[nhcodes]

It seems like this has been fixed in version 1.1.0.

Session: Add default implementation to MediaSession.Callback.onAddMediaItems to allow requested MediaItems to be passed onto Player if they have LocalConfiguration (e.g. URI) (#282).

[deeppandya]

you can implement onAddMediaItems like below,

override fun onAddMediaItems(
        mediaSession: MediaSession,
        controller: MediaSession.ControllerInfo,
        mediaItems: List<MediaItem>
    ): ListenableFuture<List<MediaItem>> {
        // We need to use URI from requestMetaData because of https://github.com/androidx/media/issues/282
        val updatedMediaItems: List<MediaItem> =
            mediaItems.map { mediaItem ->
                 MediaItem.Builder()
                    .setMediaId(mediaItem.mediaId)
                    .setMediaMetadata(mediaItem.mediaMetadata)
                    .setUri(mediaItem.requestMetadata.mediaUri)
                    .build()
            }
        return Futures.immediateFuture(updatedMediaItems)
    }

and when you setMediaItems, you need to add uri to request meta data.