superpoweredSDK / Low-Latency-Android-iOS-Linux-Windows-tvOS-macOS-Interactive-Audio-Platform

🇸Superpowered Audio, Networking and Cryptographics SDKs. High performance and cross platform on Android, iOS, macOS, tvOS, Linux, Windows and modern web browsers.
https://superpowered.com
1.35k stars 285 forks source link

Crashes type EXC_BAD_ACCESS KERN_INVALID_ADDRESS, SuperpoweredIOSAudioIO incorrect data writing #787

Open AndrewRudyk opened 1 month ago

AndrewRudyk commented 1 month ago

Superpowered version: 2.6.6

Describe the bug My app uses Superpowered SDK to create different effects: AutomaticVocalPitchCorrection, ThreeBandEQ, TimeStretching, Bitcrusher, etc. Now I have a lot of reports from Firebase Crashlytics

image image

A lot of problems arise in the variable "samplerate" from class SuperpoweredIOSAudioIO that comes with the SDK.

Xcode shows a lot of Warnings in this class, for example "This code path does interprocess communication underneath which can cause non-deterministic delays. Investigate ways to do this work off the main thread

There are reports of this code path causing UI hangs. See Xcode Organizer for details . Look for the report that shows calls to -[AVAudioSession currentRoute] underneath"

I think that SuperpoweredIOSAudioIO writes incorrect data to the samplerate variable, which is then used by other SuperpoweredSDK objects.

Can you fix this?

Steps to Reproduce Due to the nature of the bug, the reproduction paths vary, and the crash does not happen predictably, making it challenging to outline specific steps to trigger the issue.

Device information Please list which devices have this bug.

ivannador commented 1 month ago

Could you please try with the latest update to the SuperpoweredIOSAudioIO class?

https://github.com/superpoweredSDK/Low-Latency-Android-iOS-Linux-Windows-tvOS-macOS-Interactive-Audio-Platform/commit/caf2102570ba86997cced5e47a6a9090031c166a

AndrewRudyk commented 1 month ago

Could you please try with the latest update to the SuperpoweredIOSAudioIO class?

caf2102

Hello! I'll check it and get back to you with the results. We will have more results after we release the app. Thank you.

AndrewRudyk commented 1 month ago

Hello, I updated Superpowered SDK to 2.2.8 and sent our app to AppStore. And I see current statistics in Firebase,

image image image image

But something still goes wrong:)

I saw comment in SuperpoweredIOSAudioIO

image

Perhaps this problem is still too common

gaborszanto commented 1 month ago

Some memory handling is wrong in your Superpowered.mm file, and therefore the Superpowered features you're using are crashing. Please check your buffer sizes.

AndrewRudyk commented 1 month ago

Hello, I took code for Superpowered.mm from your examples in repo and preferredBufferSize == 12 took from repo too. How can I change or check the preferredBufferSize?

I use effect objects (Recorder, Flanger, Filter, etc) from SuperpoweredSDK in chain.

- (void)startProcessing {
    if (audioIO == nil) {
        audioIO = [[SuperpoweredIOSAudioIO alloc] initWithDelegate:(id<SuperpoweredIOSAudioIODelegate>)self
                                               preferredBufferSize:12
                                               preferredSamplerate:mySamplerate
                                              audioSessionCategory:AVAudioSessionCategoryPlayAndRecord
                                                          channels:2
                                           audioProcessingCallback:audioProcessing
                                                        clientdata:(__bridge void *)self];
        [audioIO start];
    }
}

// ...

// This code crashes
// SuperpoweredIOSAudioIO calls this closure

static bool audioProcessing(void *clientdata, float *input, float *output, unsigned int numberOfFrames, unsigned int samplerate, uint64_t hostTime) {
    __unsafe_unretained SuperpoweredBridge *self = (__bridge SuperpoweredBridge *)clientdata;

    if (self->limiter) {
        self->limiter->samplerate = samplerate;
        self->limiter->process(input, input, numberOfFrames);
    }

    if (self->compressor) {
        self->compressor->samplerate = samplerate;
        self->compressor->process(input, input, numberOfFrames);
    }

    // preparing for recording
    self->playerA->outputSamplerate = samplerate;
    if (self->_recording && !self->recorderIsPrepared) {
        self->recorderIsPrepared = self->recorder->prepare(self->test2Url, samplerate, false, 0);
    }

    if (self->_recording) {
        self->recorder->recordInterleaved(input, numberOfFrames);
    }

    if (self->spVoicetune) {
        self->spVoicetune->samplerate = samplerate;
        self->spVoicetune->process(input, input, true, numberOfFrames);
    }

    if (self->timeStretching) {
        self->timeStretching->samplerate = samplerate;

        self->timeStretching->addInput(input, numberOfFrames);
        self->timeStretching->getOutput(input, numberOfFrames);
    }

    if (self->threeBandEQ) {
        self->threeBandEQ->samplerate = samplerate;
        self->threeBandEQ->process(input, input, numberOfFrames);
    }

    // ...

    if (self->echo) {
        self->echo->samplerate = samplerate;
        self->echo->process(input, input, numberOfFrames);
    }

    if (self->delayIsEnabled) {
        self->delay->samplerate = samplerate;
        const float *delayOutput = self->delay->process(input, numberOfFrames);
        memcpy(input, delayOutput, numberOfFrames * sizeof(float) * 2);
    }

    // ...

    if (self->_recording) {
        Superpowered::Volume(input, output, 0.6f, 0.6f, numberOfFrames);
        self->playerA->processStereo(output, true, numberOfFrames, 0.6f);
    }

    return true;
}
ivannador commented 1 month ago

Two things at first glance:

  1. You shouldn't call recorder->prepare(...) on the real-time thread. (Check here)
  2. It would be wise to check if float *input is not NULL. If for some reason the audio system can't render into the input buffers, it can be NULL.
AndrewRudyk commented 1 month ago

Two things at first glance:

  1. You shouldn't call recorder->prepare(...) on the real-time thread. (Check here)
  2. It would be wise to check if float *input is not NULL. If for some reason the audio system can't render into the input buffers, it can be NULL.

I have two notes and questions:

  1. I had to move 'recorder->prepare(...)' from another Prepare function to this location because 'static bool audioProcessing(...)' returned a different Samplerate value than '- (void)startProcessing'. For example, I had set 48000 in '- (void)startProcessing', 'static bool audioProcessing(...)' returned 44100 and I had problems in recording.

  2. If the input can be NULL, why do the crashes not occur at the beginning of the chain?

AndrewRudyk commented 3 weeks ago

Hello, I have a question: SuperpoweredSDK's objects modify input-data in "static bool audioProcessing(...)" by chain. One of the objects modifies data in such a way that next object can fail when using this data. How can I check this data and fix it?

ivannador commented 3 weeks ago

Hey Andrew!

Best way would be to log specific edge points of the buffer that goes through the chain and capture where it might go wrong. Something seems to be amiss at some point, so definitely a good idea to log the buffer.

Also, you could use a fixed sample rate on the recorder when preparing, and use a Resampler if the actual sample rate differs in the real-time loop.

AndrewRudyk commented 3 weeks ago

Hello Ivan! Could you please explain how to define specific edge points of the buffer. Thank you very much!

ivannador commented 3 weeks ago

Log values around the supposed end of the buffer, check if values get messed up that causes one of the steps in the chain to crash.

ivannador commented 6 days ago

Hey @AndrewRudyk, any updates from your side?

AndrewRudyk commented 6 days ago

Hi I don't have any updates, I haven't researched it from your point of view yet.

I'll return to this topic next spring, when I'm preparing for the next release.

Sorry