fix: AudioSources seemingly become invalid after leaving standby (on Android)

[larssn]

Description

So we have a few Pixel phones, that, when the devices have been in standby for a while, stop playing cached sounds after waking up again.

Maybe we're doing it wrong.

We basically cache them like this:

AudioSource? click;
AudioSource? addItem;
AudioSource? delete;
AudioSource? purchase;
AudioSource? scan;

Future<AudioSource> getAudioSource(SoLoud instance, String soundName) async {
  const root = 'assets/sounds';
  if (soundName == 'click.wav') {
    click ??= await instance.loadAsset('$root/click.wav');
    return click!;
  } else if (soundName == 'add-item.wav') {
    addItem ??= await instance.loadAsset('$root/add-item.wav');
    return addItem!;
  } else if (soundName == 'delete.wav') {
    delete ??= await instance.loadAsset('$root/delete.wav');
    return delete!;
  } else if (soundName == 'purchase.wav') {
    purchase ??= await instance.loadAsset('$root/purchase.wav');
    return purchase!;
  } else if (soundName == 'scan.wav') {
    scan ??= await instance.loadAsset('$root/scan.wav');
    return scan!;
  } else {
    throw Exception('Unknown sound filename');
  }
}

And then the "cached" AudioSources are using like so:

Future<void> feedback([String? sound]) async {
  final sl = SoLoud.instance;
  if (!await sl.initialized) await sl.init();
  final source = await getAudioSource(sl, sound ?? 'click.wav');
  await sl.play(source);
}

We instantiate SoLoud in main.dart like so:

final sl = SoLoud.instance;
if (!await sl.initialized) await sl.init();

Steps To Reproduce

1. Use the above code
2. Verify that sounds play
3. Minimize the app in Android
4. Turn the device off for a while
5. Maximize the app again, and try to play the sound. Nothing should play at this point until the app is restarted.

Expected Behavior

I expect that assets stay loaded, even after leaving standby.

[alnitak]

You should think of handles as ephemeral things that become invalid just after the AudioSource, that owns it, finishes playing it.

So, when you call the loadAsset() it gives you back an AudioSource. And when you play() it you will get the playing handle. You can play it again and again and every time you get back a different handle. Handles become invalid when you manually stop or when they finish playing.

Also, if an audio has been already loaded, you should see a log warning:

Sound [...] was already loaded. Prefer loading only once, and reusing the loaded sound when playing.

So when you, for example, want to play a click, just check if AudioSource? click;. If it is null use loadAssets() else play() it.

If you have any doubts, please let me know.

[larssn]

Thanks for your reply.

So when you, for example, want to play a click, just check if AudioSource? click;. If it is null use loadAssets() else play() it.

If you have any doubts, please let me know.

Sorry, I called it "handles" when in reality I meant AudioSources: The AudioSources become invalid.

And I already checked If "click" etc is null, as you can see from the code above :)

The problem is the assets loaded via loadAssets stop working, apparently.

[alnitak]

I suspected you meant AudioSources, but I went ahead because your code looked right :) Anyway, I tried your code on a Samsung Galaxy Note S20 and on some emulators without having your issue.

I used this code

```dart import 'dart:developer' as dev; import 'package:flutter/foundation.dart'; import 'package:flutter/material.dart'; import 'package:flutter_soloud/flutter_soloud.dart'; import 'package:logging/logging.dart'; void main() async { // The `flutter_soloud` package logs everything // (from severe warnings to fine debug messages) // using the standard `package:logging`. // You can listen to the logs as shown below. Logger.root.level = kDebugMode ? Level.FINE : Level.INFO; Logger.root.onRecord.listen((record) { dev.log( record.message, time: record.time, level: record.level.value, name: record.loggerName, zone: record.zone, error: record.error, stackTrace: record.stackTrace, ); }); // WidgetsFlutterBinding.ensureInitialized(); /// Initialize the player. // await SoLoud.instance.init(); runApp( const MaterialApp( home: TestInvalid(), ), ); } /// Simple usecase of flutter_soloud plugin class TestInvalid extends StatefulWidget { const TestInvalid({super.key}); @override State createState() => _TestInvalidState(); } class _TestInvalidState extends State { AudioSource? click; AudioSource? addItem; AudioSource? delete; AudioSource? purchase; AudioSource? scan; @override void dispose() { SoLoud.instance.deinit(); super.dispose(); } Future getAudioSource(SoLoud instance, String soundName) async { if (soundName == 'click.wav') { click ??= await instance.loadAsset('assets/audio/explosion.mp3'); return click!; } else if (soundName == 'add-item.wav') { addItem ??= await instance.loadAsset('assets/audio/explosion.mp3'); return addItem!; } else if (soundName == 'delete.wav') { delete ??= await instance.loadAsset('assets/audio/explosion.mp3'); return delete!; } else if (soundName == 'purchase.wav') { purchase ??= await instance.loadAsset('assets/audio/explosion.mp3'); return purchase!; } else if (soundName == 'scan.wav') { scan ??= await instance.loadAsset('assets/audio/explosion.mp3'); return scan!; } else { throw Exception('Unknown sound filename'); } } Future feedback([String? sound]) async { final sl = SoLoud.instance; if (!await sl.initialized) await sl.init(); final source = await getAudioSource(sl, sound ?? 'click.wav'); final handle = await sl.play(source); // Attempted fix of handles dying, but it doesn't work. if (!sl.getIsValidVoiceHandle(handle)) { debugPrint('SoundHandle error'); click = null; addItem = null; delete = null; purchase = null; scan = null; await sl.disposeAllSources(); } } @override Widget build(BuildContext context) { return Scaffold( body: Center( child: Column( mainAxisSize: MainAxisSize.min, children: [ ElevatedButton( onPressed: () async { await feedback(); }, child: const Text('feedback'), ), ElevatedButton( onPressed: () async { debugPrint(click.toString()); }, child: const Text('print click'), ), ], ), ), ); } } ```

When I ran it I verified that the sound plays. Then I minimized the app and turned the device off for 5 and 10 min. The sound played normally after turning it on again and bringing the app to the foreground.

Please, can you confirm the above code gives you the same problem? Also, the output log could be useful to identify it.

[larssn]

I'll give it a go, but I think you need to turn the device off for a bit longer: it might need to enter doze mode.

[alnitak]

I'll give it a go, but I think you need to turn the device off for a bit longer: it might need to enter doze mode.

I hadn't thought about doze mode.

I tried to force entering into that idle mode following this guide. I still have not the problem and no clue about this issue!!

I am trying now to leave the phone turned off for a longer time.

[larssn]

I've created a simple project with the above code. I'll also leave it for a while, and see if that does anything.

The reason I expect some kind of memory cleanup, is because our main app is very big, so Android might be more inclined to kill off certain resources.

[larssn]

A few hours has passed, and my sound is now also dead in the tiny test project. I ran the project as a release on my Pixel 5 (physical device).

Can you verify? If not, I'll simply attach the project, and we'll keep digging. 😊

Thanks for the help

[alnitak]

I woke up my phone too. Not only the sound are gone, but also the UI!!! I can't see those 2 buttons and cannot verify anything. Also attaching the debugger to the app, didn't display anything in the console log!

I don't know what is happening here. I could think that the OS kills all the app resources of course and when the app resurrects, it think the audio engine is still alive but it is not. So here:

if (!await sl.initialized) await sl.init();

it thinks it is already initialized but it's not true. If this makes sense, we should deinit() the player when the OS goes into doze mode. But I don't know how to grab that event.

[larssn]

Not sure either, and I have zero experience with FFI tbh. Normally I'd recommend looking at Android lifecycle events, but I'm not even sure those are relevant in this context?

[alnitak]

Unfortunately, the Android lifecycle detects when the app goes in the background but not when it goes to idle after some time of inactivity.

What I still see, is that after sending the app in background and turning the phone off for an hour, when I bring the app in foreground I see a blank UI page (no buttons of the little example posted here).

I also tried sentry.io but no exceptions are thrown.

Also by using adb to force the doze mode with:

adb shell dumpsys battery unplug
adb shell dumpsys deviceidle force-idle

then check if mState=ACTIVE with:

adb shell dumpsys deviceidle

it doesn't give me any problem when restoring the app to the foreground (the UI is visible and the sound plays). To notice the issue I had to wait about an hour with the phone turned off.

This make me think that, if the adb command are working, could not be a doze mode (idle) problem?!

[larssn]

Yeah might not be doze mode. I'm surprised your Flutter Widget dies entirely.

Maybe try wrapping the dispose soloud call in try/catch, in case something crashes the widget there?

[KzumO36]

Same on iOS, its disappearing everywhere after a while. Sounds becoming null.

[AydinBK]

This plugin is by far the one with the best sound experience and the lowest latency, but this issue needs to be solved.

Has anybody been able to figure out what the course of the issue is?

[FluffyBunniesTasteTheBest]

Any update on this issue?

[alnitak]

@FluffyBunniesTasteTheBest I am awaiting some suggestions because I don't know where the problem comes from.

[FluffyBunniesTasteTheBest]

This seems to be more of an iOS than an Android issue: My App gets quite some broken audio reports from iOS users, but none from Android users, although only one third of its >20k monthly users are on iOS, which seems out of proportions...

Simply reloading all audio clips when the app was in background for longer than 20 minutes didn't solve the issue - i.e. some iOS users still complained that there was no audio output.