Playing in background for iOS

[wsievern]

Hey Chip,

Love this repo! Moved my flutter app using melty soundfont from raw_sound to this, and the difference in audio quality is huge. Also, feeding the samples only when the buffer is running low on samples is brilliant.

However, I've run into a problem I haven't been able to crack. Before, while using raw_sound, I was able to keep the audio playing in the background with the screen off/when the user closed the app. Now that I'm using flutter_pcm_sound, whenever I turn off the screen or navigate away from my app on iOS, the audio fades out after a second or two. It restarts again if I re-open the app.

Any idea what's going on here? I've poked around quite a bit in the library, and online, but can't figure out why this is happening. For what it's wort it's only happening when I'm testing on my physical iPhone, but not on the simulator.

Thx for putting this together! Appreciate it.

[chipweinberger]

thanks for the kind words =)

to be honest, I'm not sure.

It probably has something to do with the setup code. Or maybe permissions?

     else if ([@"setup" isEqualToString:call.method])
        {
            NSDictionary *args = (NSDictionary*)call.arguments;
            NSNumber *sampleRate  = args[@"sample_rate"];
            NSNumber *numChannels = args[@"num_channels"];

            self.mNumChannels = [numChannels intValue];

            // cleanup
            if (_mAudioUnit != nil) {
                [self cleanup];
            }

            // create
            AudioComponentDescription desc;
            desc.componentType = kAudioUnitType_Output;
#if TARGET_OS_IOS
            desc.componentSubType = kAudioUnitSubType_RemoteIO;
#else // MacOS
            desc.componentSubType = kAudioUnitSubType_DefaultOutput;
#endif
            desc.componentFlags = 0;
            desc.componentFlagsMask = 0;
            desc.componentManufacturer = kAudioUnitManufacturer_Apple;

            AudioComponent inputComponent = AudioComponentFindNext(NULL, &desc);
            OSStatus status = AudioComponentInstanceNew(inputComponent, &_mAudioUnit);
            if (status != noErr) {
                NSString* message = [NSString stringWithFormat:@"AudioComponentInstanceNew failed. OSStatus: %@", @(status)];
                result([FlutterError errorWithCode:@"AudioUnitError" message:message details:nil]);
                return;
            }

            // set stream format
            AudioStreamBasicDescription audioFormat;
            audioFormat.mSampleRate = [sampleRate intValue];
            audioFormat.mFormatID = kAudioFormatLinearPCM;
            audioFormat.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
            audioFormat.mFramesPerPacket = 1;
            audioFormat.mChannelsPerFrame = self.mNumChannels;
            audioFormat.mBitsPerChannel = 16;
            audioFormat.mBytesPerFrame = self.mNumChannels * (audioFormat.mBitsPerChannel / 8);
            audioFormat.mBytesPerPacket = audioFormat.mBytesPerFrame * audioFormat.mFramesPerPacket;

            status = AudioUnitSetProperty(_mAudioUnit,
                                    kAudioUnitProperty_StreamFormat,
                                    kAudioUnitScope_Input,
                                    kOutputBus,
                                    &audioFormat,
                                    sizeof(audioFormat));
            if (status != noErr) {
                NSString* message = [NSString stringWithFormat:@"AudioUnitSetProperty StreamFormat failed. OSStatus: %@", @(status)];
                result([FlutterError errorWithCode:@"AudioUnitError" message:message details:nil]);
                return;
            }

            // set callback
            AURenderCallbackStruct callback;
            callback.inputProc = RenderCallback;
            callback.inputProcRefCon = (__bridge void *)(self);

            status = AudioUnitSetProperty(_mAudioUnit,
                                kAudioUnitProperty_SetRenderCallback,
                                kAudioUnitScope_Global,
                                kOutputBus,
                                &callback,
                                sizeof(callback));
            if (status != noErr) {
                NSString* message = [NSString stringWithFormat:@"AudioUnitSetProperty SetRenderCallback failed. OSStatus: %@", @(status)];
                result([FlutterError errorWithCode:@"AudioUnitError" message:message details:nil]);
                return;
            }

            // initialize
            status = AudioUnitInitialize(_mAudioUnit);
            if (status != noErr) {
                NSString* message = [NSString stringWithFormat:@"AudioUnitInitialize failed. OSStatus: %@", @(status)];
                result([FlutterError errorWithCode:@"AudioUnitError" message:message details:nil]);
                return;
            }

            result(@(true));
        }

[wsievern]

Got it working! Edited the setup like this:

else if ([@"setup" isEqualToString:call.method])
        {
            NSDictionary *args = (NSDictionary*)call.arguments;
            NSNumber *sampleRate  = args[@"sample_rate"];
            NSNumber *numChannels = args[@"num_channels"];

            self.mNumChannels = [numChannels intValue];

            NSError *error = nil;
            [[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayback error:&error];
            if (error) {
                NSLog(@"Error setting AVAudioSessionCategoryPlayback: %@", error);

            }
            [[AVAudioSession sharedInstance] setActive:YES error:&error];
            if (error) {
                NSLog(@"Error activating AVAudioSession: %@", error);

            }

            // cleanup
            if (_mAudioUnit != nil) {
                [self cleanup];
            }

Getting some audio clicks when exiting the app/turning off the screen now... Onto debugging that.

[chipweinberger]

awesome. let me know if you fix the other issues =)

[chipweinberger]

1. did you fix the other issues
2. should we add more arguments to "setup" to fill all of the use cases?
3. can you open a PR? thanks.

btw, don't just log, return an error =)

[wsievern]

hi chip! so sorry this slipped by me, I didn't see it until now... I did fix the other issues, and audio is playing back perfectly in my iOS app now.

1) yes 2) i'm not sure, what do you think? it's working for me on iPhones and iPads. 3) yes

[wsievern]

btw, fixing the other issues was a matter of tweaking the buffer size and feed thresholds--once I made them sufficiently large playback stopped glitching while turning off the screen or navigating outside of the app. here is the code in my app that handles this:

MusicPlayer({ required this.userDataManager, }) { buffer = PcmArrayInt16.zeros( count: 7056); //double the value that produces no clicks on my iPhone 12 mini }); }

// ...

Future init() async { await FlutterPcmSound.setLogLevel(LogLevel.verbose); await FlutterPcmSound.setup(sampleRate: 44100, channelCount: 1); await FlutterPcmSound.setFeedThreshold(44100 ~/ 3); FlutterPcmSound.setFeedCallback(playMusic); }

[chipweinberger]

this was merged

https://github.com/chipweinberger/flutter_pcm_sound/pull/7

[chipweinberger]

this is now in v1.2.0