IOS Latency issue - There is a latency of 500ms when we record voice from microphone with background music

[shanalishams]

Hello,

We are facing a latency issue of 500ms when we are trying to record output from the microphone with the background music. I am adding relevant fragments of code to help you understand how we are initializing and doing the processing.

Following is the initialization code, here we are initializing the AdvanceAudioPlayer, Recorder, and the SuperpoweredIOSAudioIO with AVAudioSessionCategoryPlayAndRecord as audioCategory.

backingMusicPlayer = new Superpowered::AdvancedAudioPlayer((unsigned int) 44100, 0);
backingMusicPlayer->open([backingMusicPath UTF8String]);
backingMusicPlayer->syncMode = Superpowered::AdvancedAudioPlayer::SyncMode::SyncMode_TempoAndBeat;

recorder = new Superpowered::Recorder([recordingFileFinalPath UTF8String], false);

recorder->prepare([recordingFileFinalPath UTF8String], 44100, false, 1);

outputIO = [[SuperpoweredIOSAudioIO alloc] initWithDelegate:(id<SuperpoweredIOSAudioIODelegate>)self preferredBufferSize:12 preferredSamplerate:44100 audioSessionCategory:AVAudioSessionCategoryPlayAndRecord channels:2 audioProcessingCallback:audioProcessing clientdata:(__bridge void *)self];

 [outputIO start];

In the following code, we are doing the processing inside the audio callback where we are getting the input from a microphone and in output we are adding a background music using processStereo() and to record them together we are adding them using Add1()

In IOS we get Input and Output in separate variables so to record them together I am adding them using Add1() function and giving the result to the recorder is that correct? I think that is causing the latency. How can we merge the input and output in an efficient way in IOS? or is there any other setting that we can turn on to reduce the latency or other aspects we need to see in IOS?

We are using wired as well as Bluetooth headsets for recording voice.

static bool audioProcessing(void *clientdata, float *input, float *output, unsigned int numberOfFrames, unsigned int samplerate, uint64_t hostTime) {
  __unsafe_unretained RCTSingleFitModuleiOS *self = (__bridge RCTSingleFitModuleiOS *)clientdata;
  return [self audioProcessing:input output:output numFrames:numberOfFrames samplerate:samplerate];
}

- (bool)audioProcessing:(float *)input output:(float *)output numFrames:(unsigned int)numberOfFrames samplerate:(unsigned int)samplerate {

    backingMusicPlayer->outputSamplerate = samplerate;

    if (playing) {
        backingMusicPlayer->processStereo(output, false, numberOfFrames, backingMusicPlayerVolume);
        if (recording) {
          Superpowered::Add1(output, input, numberOfFrames * sizeof(float) * 2);
          recorder->recordInterleaved(input, numberOfFrames);
        }
    }
    return playing;
  }
  return  false;
}

Let me know if the code confuses you. I will add more details. We need to solve the latency problem as it is getting blocker for us. So any immediate reply will be a great help.

[ivannador]

Hello! What do you mean exactly by having 500ms latency?

I can see a couple problems in your code which might be related to the problem:

• I can't see the backingMusicPlayer toggled to play anywhere. Might be that it's started later thus causing latency?
• Superpowered::Add1 requires the length of the input in number of samples, so sizeof(float) is not needed.

[ivannador]

No further input from user, closing issue.