Q: Background iOS audio troubleshooting

[drew-512]

Hi friends,

I'm troubleshooting background audio getting suspended on iOS when the app is backgrounded. Probably on my end, but the only lead I have is:

-> applicationWillResignActive()
2023-02-10 22:40:31.836599-0600 NodleMusicPlayer[88752:6613379] [Player] player rate changed to 0.000000
2023-02-10 22:40:31.836634-0600 NodleMusicPlayer[88752:6613379] [Player] player time control status changed to paused
-> applicationDidEnterBackground()

Is AVPro the one emitting [Player] messages? Hoping this indicates the issue.

Same results for an iOS 16 and 13.4 both OS 10.14 and 12.1.

Triple checked:

1. MediaPlayer instance Audio Mode set to System Direct
2. AVAudioSessionCategoryPlayback patch in UnityAppController.mm:

• (void)startUnity:(UIApplication*)application {

  UnityInitApplicationGraphics(); ...

  AVAudioSession* audioSession = [AVAudioSession sharedInstance];

  [audioSession setCategory: AVAudioSessionCategoryPlayback / PATCH / error: nil]; [audioSession setActive: YES error: nil]; [[UIApplication sharedApplication] beginReceivingRemoteControlEvents]; // uneeded? observed in the wild

  ..

  
  3. Proper iOS settings:

Going bonkers on what I could be missing.

Thanks, Drew

[MorrisRH]

The default player behaviour on iOS is to suspend video playback when the app is put into the background (that's the system, not us imposing this).

I'm aware of a work-around for keeping the audio playing but we've never had cause to implement it so as it stands there is nothing that can be done.

I am assuming that this is video playback and not just an audio file? Audio only files should respect the background modes, but as we're a video player it's not something that has been tested.

[drew-512]

Yes, just audio. Strangely, I throught I had it working months ago while evaluating AVPro (specifically for this scenario), and so I'm scratching my head -- I had switched back to use Unity output on AVPro while Direct with Capture was under construction.

And yes, well aware these nuances here are driven by iOS. My testing with with both local audio file playback and streaming audio are the same -- the above log messages appear when the app is backgrounded. From my experience, iOS will still suspend a backgrounded app if that permission is set and if no audio is playing (regardless of network activity) -- but that's only if no audio is playing.

Is AVPro is emitting the [Player] messages (above)? Trying to understand what's happening.

Trying to get a POC / prototype out the door with background audio working, so any help is appreciated.

[drew-512]

Repro:

1. Using Unity 2021.3.18 (OS 10.14), create fresh URP project
2. Add AVPro (2.7.2) from package manager
3. Open scene Demo_MediaPlayer
4. OnMediaPlayer: Platform Specific -> iOS -> Audio Mode -> System Direct
5. Delete AudioListener and AudioOutput from the MediaPlayer game object
6. (Optional): put a mp3 audio file into SteamingAssets, make a new media ref for it, and set the MediaPlayer's current item to that ref
7. In Player settings, for iOS, enable Enable Custom Background Behaviors and Audio, Airplay, PIP
8. Switch target to iOS, build Xcode project
9. Patch UnityAppController.mm, replacing AVAudioSessionCategoryAmbient with:

            [audioSession setCategory: AVAudioSessionCategoryPlayback error: nil];

10. Build and run on iOS device
11. Press play (audio should begin playing)
12. Background the iOS app

Expected: audio continues to play Actual: audio suspends, console output:

-> applicationWillResignActive()
2023-02-10 22:40:31.836599-0600 NodleMusicPlayer[88752:6613379] [Player] player rate changed to 0.000000
2023-02-10 22:40:31.836634-0600 NodleMusicPlayer[88752:6613379] [Player] player time control status changed to paused
-> applicationDidEnterBackground()

[MorrisRH]

After some playing around I've realised the problem has been introduced from our side in response to another issue (related to playback pausing when the control centre is open).

I'll think on it and see what can be done. We're planning another release later this week so hopefully I'll have something for you by then.

The [Player] ... output is from us in response to notifications being sent by the system player, basically so we can see what's going on under the hood.

[MorrisRH]

I've put in a fix specifically for audio only media, it will make it into the next release 2.7.3.

[drew-512]

Thanks for the update and attention on this Morris. I thought I was losing it and have spent so much time trying to crack this.

[Chris-RH]

The latest version has been released. Please let us know if the issue is not fixed.

[drew-512]

Success here!

Sincerely appreciate the quick turnaround, thank you.

For the archives, folks should note that UnityIsAudioManagerAvailableAndEnabled() in UnityAppController.mm is evaluating as true (for reasons unknown), so it should also be removed when patching. I'm guessing this evaluates as false when the entire Unity audio system is disabled, but we'd still like to use Unity for audio for effects and such when in the foreground.