Audio capture callback never triggered with pulseaudio and SDL2 >= 2.30.0

SDL2 is expected to invoke a callback to provide audio data captured from microphone.

With SDL2 2.30.3 and 2.30.0, that callback is never triggered.

With SDL2 2.28.4 and 2.28.5, the callback works as expected.

Git bisect indicates that the commit introducing the behavior change is e51760e11166a81dd4ba504df559b9b16100815c. This is related to #7427.

The issue seems to be related to some threading aspect of the pulseaudio driver (or maybe resampling?).

pavucontrol shows that the recording is in progress so pulse is aware of the capture attempt.

Using SDL_AUDIODRIVER=alsa does enable the callback to be triggered with 2.30.3 but the data seems to contain a lot of additional blank audio (like 30s of blank audio before the actual recorded audio).

I stumbled upon this behavior trying out whisper.cpp but this can be reproduced with the following test code that uses the same audio format for capture.

#include <SDL2/SDL.h>

const int SAMPLE_RATE = 16000;
const int NUM_CHANNELS = 1;
const int BUFFER_SIZE = 1024;
const int RECORDING_TIME = 2000; // 2 seconds in milliseconds

int callbackCount = 0;

// Audio callback function
void audioCallback(void* userdata, Uint8* stream, int len) {
    // Do nothing with the audio data, just count the callback invocations
    callbackCount++;
}

int main() {
    if (SDL_Init(SDL_INIT_AUDIO) < 0) {
        std::cerr << "SDL initialization failed: " << SDL_GetError() << std::endl;
        return 1;
    }

    SDL_AudioSpec wantedSpec;
    SDL_zero(wantedSpec);
    wantedSpec.freq = SAMPLE_RATE;
    wantedSpec.format = AUDIO_F32;
    wantedSpec.channels = NUM_CHANNELS;
    wantedSpec.samples = BUFFER_SIZE;
    wantedSpec.callback = audioCallback;

    SDL_AudioSpec obtainedSpec;
    SDL_zero(obtainedSpec);

    // Open the default audio capture device
    SDL_AudioDeviceID audioDevice = SDL_OpenAudioDevice(NULL, 1, &wantedSpec, &obtainedSpec, 0);
    if (audioDevice == 0) {
        std::cerr << "Failed to open audio: " << SDL_GetError() << std::endl;
        SDL_Quit();
        return 1;
    }

    // Start recording
    SDL_PauseAudioDevice(audioDevice, 0);

    // Wait for recording time
    SDL_Delay(RECORDING_TIME);

    // Stop recording
    SDL_PauseAudioDevice(audioDevice, 1);

    // Close the audio device
    SDL_CloseAudioDevice(audioDevice);

    // Print out the number of callback invocations
    std::cout << "Callback invoked " << callbackCount << " times." << std::endl;

    SDL_Quit();
    return !callbackCount;
}

Tested using g++ -o test test.cpp -lSDL2 && ./test

libsdl-org / SDL

Audio capture callback never triggered with pulseaudio and SDL2 >= 2.30.0 #9706