Support mixing audio with other apps on Android

[asindhu]

What is the need and use case of this feature? RNTP currently supports the iOS audio mixing modes but does not provide such options for Android. The use case would be to allow feature parity for any app that uses IOSCategoryOptions.MixWithOthers or DuckOthers for scenarios where the app's audio may combine with that of other apps in iOS -- for example, in my case, playing background music from another app while my app plays narrated audio. Currently such uses cannot be supported on Android with RNTP.

Describe the ideal solution A flag in setupPlayer that indicates how audio from RNTP should mix with other apps on Android. This could mirror the Expo AV library interface, which has fewer options than for iOS, but still includes both a DoNotMix and DuckOthers option for Android.

Describe alternatives you've considered I've spent a lot of time trying workarounds for this. Prior to RNTP I was using Expo's AV module, and in its config options it has InterruptionModeAndroid.DuckOthers which provides me with the functionality I need. Unfortunately, I need my audio to continue playing with the app backgrounded, which Expo AV does not support, so I switched over to RNTP.

Now with RNTP, if I still pull in Expo AV and configure it with InterruptionModeAndroid.DuckOthers, I temporarily get the behavior I'm looking for -- the first RNTP track and the audio from the other app overlap and mix. However, as soon as the track in my RNTP app changes to a new one, the background audio from the other app stops playing, so this solution unfortunately isn't workable.

Additional context I've looked through all of the prior issues/questions for RNTP including this one which seems to be a similar request but was closed with the comment that no such support for mixing sources exists in Android. However, given that Expo AV exposes an option for audio mixing in Android as mentioned above (see this link for the docs), it seems to indeed be available.

Doing some research on this in the Android developer docs, it does seem that at the native level this is fully supported through a construct called "audio focus", which supports ducking and mixing, as explained here.

How I can Help Unfortunately I'm completely new to mobile development and especially Android. I have no knowledge of the layer below react-native JavaScript. However I am absolutely willing to help debug and/or test this feature if you are kind enough to address it. Thanks so much for your time and consideration, I really appreciate it.

[lovegaoshi]

isnt what ur describing autoHandleInterruptions? you will manage any your apps ducking and pausing behavior in event.remoteduck. its even off by fefault i think? https://rntp.dev/docs/next/api/objects/player-options though i hugely dislike manually managing event.remoteduck s ux.

[asindhu]

Thanks very much for the quick response! I just tried implementing this but I'm not sure how to get it working. So if I set autoHandleInterruptionsto true, the audio never mixes between apps. If I set autoHandleInterruptions to false, I still get the behavior I mentioned before -- the audio will mix for the first track, but as soon as the next track starts playing, the other app's audio stops.

If I listen for the RemoteDuck event, I do get an event when I start playing the audio from the other app, {"paused": true, "permanent": true}. So basically I think what is happening is with autoHandleInterruptions=true it's listening for that event for me and stopping my audio. With autoHandleInterruptions=false it's giving me the event and asking me to handle it. I think I understand all this.

The issue though is that I don't want to permanently stop my audio when another app is playing. I want it to keep going. So I can ignore the RemoteDuck event and the audio keeps playing, but only until the end of the track. Then, as the next track starts, the other app's audio stops and I don't believe I have any kind of mechanism in RNTP to control that, or do I?

Thanks again for your help!

[asindhu]

Didn't mean to close the issue, sorry.

[lovegaoshi]

that sounds like a bug.

i dont care enoguh for this feature but it sounds like rntp is incorrectly requesting audiofocus at every exoplayer.play; it shouldnt be. also see https://github.com/doublesymmetry/react-native-track-player/issues/2114

On Fri, Dec 29, 2023, 11:03 AM asindhu @.***> wrote:

Didn't mean to close the issue, sorry.

— Reply to this email directly, view it on GitHub https://github.com/doublesymmetry/react-native-track-player/issues/2229#issuecomment-1872281331, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZMOVVSXTWJ5Z7TSRENMDU3YL4HZDAVCNFSM6AAAAABBGI3QX2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZSGI4DCMZTGE . You are receiving this because you commented.Message ID: @.*** com>

[asindhu]

Got it, thanks for the clarification. So in that issue you referenced it's about whether the focus is in setup or on play, which I guess is related, but in my case I wouldn't want the other app to stop even when you hit play or pause in RNTP. Basically I want the background app sound completely unaffected by my app's playback.

While I don't have a deep understanding of any of this, I think there's likely a broader question of simply this: when RNTP requests focus, whether that mixes, ducks, or stops other apps. It does seem that the native Android environment supports that sort of configuration.

Anyway totally get it if this isn't a priority as it's sort of a niche feature. Would still appreciate it if someone is willing to address it down the road, thank you!

[lovegaoshi]

you need to disable requesting audiofocus on exoplayer becomes ready at https://github.com/doublesymmetry/KotlinAudio/blob/main/kotlin-audio/src/main/java/com/doublesymmetry/kotlinaudio/players/BaseAudioPlayer.kt#L110

[mihaibulic2]

1. @lovegaoshi, is your suggestion to do this:

   • wire in a setting in TrackPlayer.setUpPlayer like androidMode: 'primary' | 'ambient' or something
   • wire in a similar setting to kotlineaudio
   • if set to 'ambient', skip the request focus call in BAP.kt (also please lmk if you have thoughts on what those modes should be)

2. anything else that you can think of that would need to be done?

3. is this a good feature to make more broadly available or should I just for RNTP + KotlinAudio for myself?

4. I'm trying to test this out but I'm having some trouble convincing the rntp example app to use the local copy of kotlinaudio. I've followed the instructions and I see kotlin-audio in my local .m2 repo, but for some reason can't get the rntp example to find that local copy (I have mavenLocal at the top of my deps in build.gradle). I'll keep troubleshooting, but curious if there is any advice?

[lovegaoshi]

yeah u can do somrthing like that. personally i dont like every single rntp app that doesnt handle audiofocus automatically, but if it works for you go for it

i cant figure out using local maven for the longest time and resorted to spamming builds on jitpack, soeey cant gelp much.

On Tue, Jan 2, 2024, 10:22 AM mihaibulic2 @.***> wrote:

1. @lovegaoshi https://github.com/lovegaoshi, is your suggestion to do this:

   • wire in a setting in TrackPlayer.setUpPlayer like androidMode: 'primary' | 'ambient' or something
   • wire in a similar setting to kotlineaudio
   • if set to 'ambient', skip the request focus call in BAP.kt

2. anything else that you can think of that would need to be done?

3. is this a good feature to make more broadly available or should I just for RNTP + KotlinAudio for myself?

4. I'm trying to test this out but I'm having some trouble convincing the rntp example app to use the local copy of kotlinaudio. I've followed the instructions and I see kotlin-audio in my local .m2 repo, but for some reason can't get the rntp example to find that local copy (I have mavenLocal at the top of my deps in build.gradle). I'll keep troubleshooting, but curious if there is any advice?

— Reply to this email directly, view it on GitHub https://github.com/doublesymmetry/react-native-track-player/issues/2229#issuecomment-1874384103, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZMOVVW27NFQ5G2UAPEYEB3YMRF43AVCNFSM6AAAAABBGI3QX2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZUGM4DIMJQGM . You are receiving this because you were mentioned.Message ID: @.*** com>

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[liamjones]

I'm trying to test this out but I'm having some trouble convincing the rntp example app to use the local copy of kotlinaudio. I've followed the instructions and I see kotlin-audio in my local .m2 repo, but for some reason can't get the rntp example to find that local copy (I have mavenLocal at the top of my deps in build.gradle). I'll keep troubleshooting, but curious if there is any advice?

i cant figure out using local maven for the longest time and resorted to spamming builds on jitpack, soeey cant gelp much.

It's probably because, by default, Gradle caches deps for 24 hrs. You can override this in the build.gradle: https://docs.gradle.org/current/userguide/dynamic_versions.html#sec:controlling_dependency_caching_programmatically. Also make sure you're producing a -SNAPSHOT version build or setting { changing = true } on the dep so it knows it's a dynamic dependency version.

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 7 days with no activity.