mackron / miniaudio

Audio playback and capture library written in C, in a single source file.
https://miniaud.io
Other
4.07k stars 361 forks source link

Request for comments: How to solve WASAPI data discontinuity #572

Closed justinshannon closed 2 years ago

justinshannon commented 2 years ago

Hello,

Firstly, thank you for this great library. I converted from PortAudio, and it has been an absolute pleasure to use.

I use MiniAudio in a VOIP application. Each audio frame is 20ms (and the size is 960).

I have a user that is experiencing issues with their capture device ceasing to capture audio after 10-15 minutes (this is when the data discontinuity warnings start spamming the log file). There log file indicates the following:

Oct 29 09:02:46 2022: MiniAudioAudioDevice: DEBUG: [WASAPI] Trying IAudioClient3_InitializeSharedAudioStream(actualPeriodInFrames=480)
Oct 29 09:02:46 2022: MiniAudioAudioDevice: DEBUG:     defaultPeriodInFrames=480
Oct 29 09:02:46 2022: MiniAudioAudioDevice: DEBUG:     fundamentalPeriodInFrames=480
Oct 29 09:02:46 2022: MiniAudioAudioDevice: DEBUG:     minPeriodInFrames=480
Oct 29 09:02:46 2022: MiniAudioAudioDevice: DEBUG:     maxPeriodInFrames=480
Oct 29 09:02:46 2022: MiniAudioAudioDevice: DEBUG: [WASAPI] Not using IAudioClient3 because the desired period size is larger than the maximum supported by IAudioClient3.
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO: [WASAPI]
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:   Lautsprecher (Sound BlasterX G6) (Playback)
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:     Format:      32-bit IEEE Floating Point -> 32-bit IEEE Floating Point
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:     Channels:    1 -> 2
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:     Sample Rate: 48000 -> 48000
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:     Buffer Size: 960*3 (2880)
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:     Conversion:
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:       Pre Format Conversion:  NO
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:       Post Format Conversion: NO
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:       Channel Routing:        YES
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:       Resampling:             NO
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:       Passthrough:            NO
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:       Channel Map In:         {CHANNEL_MONO}
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:       Channel Map Out:        {CHANNEL_FRONT_LEFT CHANNEL_FRONT_RIGHT}
Oct 29 09:02:46 2022: MiniAudioAudioDevice: DEBUG: [WASAPI] Trying IAudioClient3_InitializeSharedAudioStream(actualPeriodInFrames=480)
Oct 29 09:02:46 2022: MiniAudioAudioDevice: DEBUG:     defaultPeriodInFrames=480
Oct 29 09:02:46 2022: MiniAudioAudioDevice: DEBUG:     fundamentalPeriodInFrames=480
Oct 29 09:02:46 2022: MiniAudioAudioDevice: DEBUG:     minPeriodInFrames=480
Oct 29 09:02:46 2022: MiniAudioAudioDevice: DEBUG:     maxPeriodInFrames=480
Oct 29 09:02:46 2022: MiniAudioAudioDevice: DEBUG: [WASAPI] Not using IAudioClient3 because the desired period size is larger than the maximum supported by IAudioClient3.
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO: [WASAPI]
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:   Externes Mikro (Sound BlasterX G6) (Capture)
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:     Format:      32-bit IEEE Floating Point -> 32-bit IEEE Floating Point
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:     Channels:    2 -> 1
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:     Sample Rate: 48000 -> 48000
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:     Buffer Size: 960*3 (2880)
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:     Conversion:
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:       Pre Format Conversion:  NO
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:       Post Format Conversion: NO
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:       Channel Routing:        YES
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:       Resampling:             NO
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:       Passthrough:            NO
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:       Channel Map In:         {CHANNEL_FRONT_LEFT CHANNEL_FRONT_RIGHT}
Oct 29 09:02:46 2022: MiniAudioAudioDevice: INFO:       Channel Map Out:        {CHANNEL_MONO}
Oct 29 09:02:46 2022: MiniAudioAudioDevice: DEBUG: [WASAPI] Data discontinuity (possible overrun). Attempting recovery. mappedBufferCaptureCap=480
Oct 29 09:03:46 2022: MiniAudioAudioDevice: DEBUG: [WASAPI] Trying IAudioClient3_InitializeSharedAudioStream(actualPeriodInFrames=480)
Oct 29 09:03:46 2022: MiniAudioAudioDevice: DEBUG:     defaultPeriodInFrames=480
Oct 29 09:03:46 2022: MiniAudioAudioDevice: DEBUG:     fundamentalPeriodInFrames=480
Oct 29 09:03:46 2022: MiniAudioAudioDevice: DEBUG:     minPeriodInFrames=480
Oct 29 09:03:46 2022: MiniAudioAudioDevice: DEBUG:     maxPeriodInFrames=480
Oct 29 09:03:46 2022: MiniAudioAudioDevice: DEBUG: [WASAPI] Not using IAudioClient3 because the desired period size is larger than the maximum supported by IAudioClient3.
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO: [WASAPI]
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:   Lautsprecher (Sound BlasterX G6) (Playback)
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:     Format:      32-bit IEEE Floating Point -> 32-bit IEEE Floating Point
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:     Channels:    1 -> 2
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:     Sample Rate: 48000 -> 48000
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:     Buffer Size: 960*3 (2880)
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:     Conversion:
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:       Pre Format Conversion:  NO
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:       Post Format Conversion: NO
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:       Channel Routing:        YES
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:       Resampling:             NO
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:       Passthrough:            NO
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:       Channel Map In:         {CHANNEL_MONO}
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:       Channel Map Out:        {CHANNEL_FRONT_LEFT CHANNEL_FRONT_RIGHT}
Oct 29 09:03:46 2022: MiniAudioAudioDevice: DEBUG: [WASAPI] Trying IAudioClient3_InitializeSharedAudioStream(actualPeriodInFrames=480)
Oct 29 09:03:46 2022: MiniAudioAudioDevice: DEBUG:     defaultPeriodInFrames=480
Oct 29 09:03:46 2022: MiniAudioAudioDevice: DEBUG:     fundamentalPeriodInFrames=480
Oct 29 09:03:46 2022: MiniAudioAudioDevice: DEBUG:     minPeriodInFrames=480
Oct 29 09:03:46 2022: MiniAudioAudioDevice: DEBUG:     maxPeriodInFrames=480
Oct 29 09:03:46 2022: MiniAudioAudioDevice: DEBUG: [WASAPI] Not using IAudioClient3 because the desired period size is larger than the maximum supported by IAudioClient3.
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO: [WASAPI]
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:   Externes Mikro (Sound BlasterX G6) (Capture)
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:     Format:      32-bit IEEE Floating Point -> 32-bit IEEE Floating Point
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:     Channels:    2 -> 1
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:     Sample Rate: 48000 -> 48000
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:     Buffer Size: 960*3 (2880)
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:     Conversion:
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:       Pre Format Conversion:  NO
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:       Post Format Conversion: NO
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:       Channel Routing:        YES
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:       Resampling:             NO
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:       Passthrough:            NO
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:       Channel Map In:         {CHANNEL_FRONT_LEFT CHANNEL_FRONT_RIGHT}
Oct 29 09:03:46 2022: MiniAudioAudioDevice: INFO:       Channel Map Out:        {CHANNEL_MONO}
Oct 29 09:03:46 2022: MiniAudioAudioDevice: DEBUG: [WASAPI] Data discontinuity (possible overrun). Attempting recovery. mappedBufferCaptureCap=480
Oct 29 09:14:03 2022: MiniAudioAudioDevice: DEBUG: [WASAPI] Data discontinuity (possible overrun). Attempting recovery. mappedBufferCaptureCap=480
Oct 29 09:14:03 2022: MiniAudioAudioDevice: DEBUG: [WASAPI] Data discontinuity (possible overrun). Attempting recovery. mappedBufferCaptureCap=480
Oct 29 09:14:03 2022: MiniAudioAudioDevice: DEBUG: [WASAPI] Data discontinuity (possible overrun). Attempting recovery. mappedBufferCaptureCap=480
Oct 29 09:14:03 2022: MiniAudioAudioDevice: DEBUG: [WASAPI] Data discontinuity (possible overrun). Attempting recovery. mappedBufferCaptureCap=480
Oct 29 09:14:03 2022: MiniAudioAudioDevice: DEBUG: [WASAPI] Data discontinuity (possible overrun). Attempting recovery. mappedBufferCaptureCap=480
Oct 29 09:14:03 2022: MiniAudioAudioDevice: DEBUG: [WASAPI] Data discontinuity (possible overrun). Attempting recovery. mappedBufferCaptureCap=480
...

The log is filled with over 150k lines of Data discontinuity warnings (the user let the application run for about 45 minutes). I provided the user with a build that forces the use of WinMM and they had no issues. When testing on my end, I get occasional data discontinuity warnings, but nothing to the extent that this user is getting.

In my application, I initialize the output and capture devices separately, instead of using duplex. I ran into a lot of issues with using the duplex mode, so I opted for separate devices. I didn't see anything in the documentation that would suggest this is a bad idea, so hopefully this is OK.

Here's a snippet of the playback and capture device initialization.

/* playback device */
ma_device_config cfg = ma_device_config_init(ma_device_type_playback);
cfg.playback.pDeviceID = &outputDeviceId;
cfg.playback.format = ma_format_f32;
cfg.playback.channels = 1;
cfg.playback.shareMode = ma_share_mode_shared;
cfg.sampleRate = sampleRateHz; /* 48000khz */
cfg.periodSizeInFrames = frameSizeSamples; /* (48000 * 20 / 1000) = 960 */
cfg.pUserData = this;
cfg.dataCallback = maOutputCallback;

ma_result result;

result = ma_device_init(&context, &cfg, &outputDev);
if(result != MA_SUCCESS) {
    return false;
}

result = ma_device_start(&outputDev);
if(result != MA_SUCCESS) {
    return false;
}
/* capture device */
ma_device_config cfg = ma_device_config_init(ma_device_type_capture);
cfg.capture.pDeviceID = &inputDeviceId;
cfg.capture.format = ma_format_f32;
cfg.capture.channels = 1;
cfg.capture.shareMode = ma_share_mode_shared;
cfg.sampleRate = sampleRateHz; /* 48000khz */
cfg.periodSizeInFrames = frameSizeSamples; /* (48000 * 20 / 1000) = 960 */
cfg.pUserData = this;
cfg.dataCallback = maInputCallback;

ma_result result;

result = ma_device_init(&context, &cfg, &inputDev);
if(result != MA_SUCCESS) {
    return false;
}

result = ma_device_start(&inputDev);
if(result != MA_SUCCESS) {
    return false;
}

You can find the full implementation here: https://github.com/xpilot-project/afv-native/blob/master/src/audio/MiniAudioAudioDevice.cpp

I've tried increasing the period size, but it garbles the audio, since the program expects it a specific size.

I'm posting this in hopes that you might have an idea of what I can do to fix this? I guess worse case I can add a configuration option to let the user choose their preferred audio backend, in case the default doesn't work.

Thank you for your time!

mackron commented 2 years ago

This is a strange one. Not entirely sure what's going on here. When an overrun happens it means the captured data hasn't been processed fast enough by the application which results in the internal buffer used by WASAPI not having enough room for the newly captured data. When this happens, miniaudio tries to completely drain the buffer in order to avoid straddling the end of the buffer and never-ending glitching. It's essentially a full reset. I've added some additional debugging logs to this recovery process to the dev branch which might shed some light into what is happening during this recovery process. Are you able to get the user to try the version in the dev branch?

To test this on my side, I modified the simple_capture example to just sleep for a bit to simulate a slow data callback and the recovery process seems to work quite well. It's hard for me to tell what might be happening from here.

(If you're curious about the logic that's being used for recovery, you can do a search for if ((flags & MA_AUDCLNT_BUFFERFLAGS_DATA_DISCONTINUITY) != 0) { in miniaudio.h)

justinshannon commented 2 years ago

Thanks for the quick response! I've sent this version to the user; I'll provide the logs once he is able to test it.

justinshannon commented 2 years ago

Attached is the log file.

Log.txt

mackron commented 2 years ago

This is strange. So the user has reported that capturing just stops completely? Looking at the times on the logs, it looks like once it starts, every single attempt at retrieving the buffer returns the data discontinuity flag. I've reworked the recovery logic in the dev branch - are you able to get the user to try that again?

justinshannon commented 2 years ago

The user tested the change for about 30 minutes and didn’t report any microphone dropouts like they previously experienced. The log file was still spammed with “Data discontinuity” warnings, however. (I truncated the log, as it was about 200k lines long). Log_20221031.txt

mackron commented 2 years ago

OK, that's somewhat good to hear. In the last commit I changed the recovery logic to not throw away the buffer that reports the data discontinuity flag which explains why the user can at least capture something now. Did the user report any glitching? Also, are you doing anything where the data callback for the capture side might stall? For example, are you writing to a buffer, and if that buffer is full, are you waiting for space to become available? If so, could that be something to look at? In that case, it could be that a desync between the playback and capture sides is happening which is resulting in the capture side not being processed fast enough therefore resulting in WASAPI's internal buffer constantly being overflowed. Just an thought I had.

As for spamming of the debug message, I'm not entirely sure what to do there short of just commenting it out. It's not ideal because I kind of like having that message there. In any case, for the moment I've updated the dev branch so that those messages are only logged when MA_DEBUG_OUTPUT is defined at compile time.

mackron commented 2 years ago

I've released the changes I made in response to this ticket. It sounds like the user's issue is sorted out now? I'll go ahead and close this one, but if there's more to tweak you can reopen this ticket or create a new one.