WASAPI: Audio distortion 22kHz

[glebm]

The WASAPI audio backend introduces distortion when resampling from 22 kHZ.

https://github.com/libsdl-org/SDL/blob/01b14e14e63ac85d295755880bcf7d441c4ff0ce/src/audio/wasapi/SDL_wasapi.c#L562-L566

Spectrogram recorded by @ikonomov:

Original report and more details: https://github.com/diasurgical/devilutionX/issues/1390

[glebm]

Reading the docs over at https://docs.microsoft.com/en-us/windows/win32/coreaudio/audclnt-streamflags-xxx-constants, it seems that we're supposed to use IAudioClockAdjustment with AUDCLNT_STREAMFLAGS_RATEADJUST.

The AUDCLNT_STREAMFLAGS_RATEADJUST flag enables an application to get a reference to the IAudioClockAdjustment interface that is used to set the sample rate for the stream

Something like this:

    if (flags & AUDCLNT_STREAMFLAGS_RATEADJUST) {
        IAudioClockAdjustment *adjustment;
        IAudioClient_GetService(client, &IID_IAudioClockAdjustment, (void **)&adjustment);
        IAudioClockAdjustment_SetSampleRate(adjustment, rate);
        IAudioClockAdjustment_Release(adjustment);
    }

[glebm]

Looks like the issue only occurs with certain audio drivers.

One user reported this happening with the RME "Pro" Audio driver but not the default Windows one.

[sezero]

vkQuake received a report about popping / crackling audio present in 2.0.22 but not before: https://github.com/Novum/vkQuake/issues/469 - possibly because of commit https://github.com/libsdl-org/SDL/commit/9e264b921b2f81e2c3d55693f76e06d143c2fbb7?

[slouken]

Possibly, does backing that out fix it?

[icculus]

I couldn't hear it in the example video. 😬

I'll listen again with headphones tonight, but can someone else make sure I'm not missing the obvious crackle?

[sezero]

@icculus: is there any progress on this?

[zenxero]

@slouken Hello, I was the one who opened the issue with vkQuake. Yes, replacing the SDL2.dll file with an older one like this fixes the audio distortion.

http://libsdl.org/release/SDL2-2.0.20-win32-x64.zip

[zenxero]

@icculus The popping distortion sound should be the most noticeable from the axe "tings" at timestamp 2:43 in the video.

https://www.youtube.com/watch?v=oGWM9bWph2A

I'm not sure if it helps, but I've tried this on both of my Windows 10 machines (laptop and desktop), and it happens on both. They are AMD systems with Realtek audio chips. The distortion happens regardless of if I'm using the 3.5mm jack or a USB audio interface.

[icculus]

Going to look at this today; my suspicion is that it isn't our resampler in general but likely that our resampler dropped precision to float from double, but I could be wrong.

[sezero]

but likely that our resampler dropped precision to float from double, but I could be wrong.

IIRC, did that not happen in git master? The issue is reported for 2.0.22.

[icculus]

IIRC, did that not happen in git master? The issue is reported for 2.0.22.

Oh, hmm, yes.

[icculus]

I went back and checked again...I have to be honest, I can't hear it in the youtube video even with headphones on, I can't hear it in a Linux build using the latest git of vkQuake and SDL. It sounds the same to me in both versions in the video.

My hearing is not great, but I don't hear any crackle in it at all.

Here, this is the audio output of the youtube video during the axe strikes. Can you tell me which one is from 2.0.20 and which is from 2.0.22?

https://www.dropbox.com/s/gvkbrepl3vmydd8/axes.wav?dl=0

[icculus]

(I don't see anything that looks significantly different between the waveforms in that .wav file, when loaded in Audacity, either, but looks can be deceiving in this regard.)

[vladkosi]

They look slightly different (right side is 2.0.22), but I can't hear the crackling.

[zenxero]

The one on the right has the crackling. I can hear it in the audio file played through audacity.

[zenxero]

These large spikes here are the popping/crackling sound

[sezero]

Is this purely personal experience or is the hardware in use really have a play?

[zenxero]

@sezero I'm not sure I understand the question

[vladkosi]

I've tried with headphones and I could hear popping in the sample file, but I couldn't reproduce the bug. All my WASAPI loopback recordings look and sound like the one on the left. Tested with vkQuake 1.13.1, SDL 2.0.20 and 2.0.22.

[sezero]

@sezero I'm not sure I understand the question

You hear clicks/pops, @icculus can not, and I could not either (but I'm not audiophile.) Some hear, some don't: That was the question, i.e.: is the issue really an issue or does the hardware (the sound card, speakers, drivers, etc) has an effect?

I initially thought that commit 9e264b9 might be at fault, however that commit was meant to workaround distortions cause by WASAPI's resampler.

It have a feeling that this might be a non-bug, but I really don't know.

[zenxero]

I'm not sure if my particular audio drivers or DAC are the problem or not. There is definitely extra popping/crackling there where there shouldn't be. But that's what I'm trying to figure out. But I've tried it on 2 different systems, my desktop and laptop, and they both have the popping distortion. And it definitely didn't use to happen since I can replace the SDL2.dll file with an older one and it goes away.

At least one other person in the original vkQuake issue reported that they had the same problem. https://github.com/Novum/vkQuake/issues/469#issuecomment-1130560702

@vladkosi said that they could hear it, but couldn't reproduce it on their system.

[zenxero]

Ok, so I tested the vkQuake 1.13.1 app image on the Steam Deck (Linux) and it does NOT have the audio crackling or popping issue that my Windows machines have.

[vladkosi]

@zenxero What is your audio device's sample rate setting in Windows?

At 44100 Hz there should be no resampling done by SDL: https://github.com/Novum/vkQuake/blob/master/Quake/snd_sdl.c#L89 (snd_mixspeed is 44100).

[zenxero]

Looks like it's 48000 Hz

I'm using one of these desktop USB DAC + AMP devices: https://jdslabs.com/product/element-ii/

[zenxero]

Ah, well now I feel stupid. Turns out if I force my audio device to use 44100 Hz instead of 48000 Hz, the popping/crackling goes away. I'm sorry for the trouble, this seems to have solved the issue.

[slouken]

SDL should be resampling without popping and crackling from 44100 Hz to 48000 Hz. @icculus, can you reproduce this if you force 48Khz output?

[slouken]

Or maybe it's the Windows resampler that introduced the crackling, and it's now fixed because we're using SDL's resampler in the latest code?

[vladkosi]

I can reproduce it at 48000 Hz. Left side is 2.0.20, right side 2.0.22:

[icculus]

What do you do to make Audacity show that view? I just get a simple waveform in mine.

[vladkosi]

https://manual.audacityteam.org/man/spectrogram_view.html

[vladkosi]

Here is the recording: 1.zip

[vladkosi]

Seems like the 48 kHz output is the most broken. 96 kHz and 192 kHz don't look as bad:

[icculus]

I dunno, I can see it in all resamples. Here it is, pushed through SDL_sound's playsound app, which lets you resample before feeding to SDL, then I used SDL's disk audio driver to write PCM to disk instead of to an audio device...

(Original wav at 44100, playback with no resampling, then 48000, 96000, 192000)

[icculus]

...although it definitely elongates in lower sampling rates if you zoom in to look at it...

[icculus]

I really can't believe this small change in the waveform is making this much of a difference.

[vladkosi]

You can switch to dB view to see the change more clearly: https://manual.audacityteam.org/man/audacity_waveform.html#db

[icculus]

Just for comparison, SDL's built-in resampler vs SDL using libsamplerate...

...this has got to be a filter issue...

[icculus]

Okay, I literally dropped in the filter generation code from resample-1.8.1 and it still happens, so the silver lining here is knowing that less of my original code is untrustworthy now. :)

Poking the resampling implementation now that I can assume the filter data is probably good.

[icculus]

I still haven't figured this one out; I'm fried, I'll try again in the morning.

[icculus]

I've been looking at this forever and can't see what's wrong; I'm spending too much time on this, so I'm going to come back to this later in the milestone.

[icculus]

Wondering if these old bug reports are related: #2715 #2716

[ericoporto]

Sorry to ping, I can't reproduce this but am curious if there's any updates on this.

[icculus]

Went back and tested this with SDL3, and we have fixed this.

I don't have the original data from the first time I generated these graphs, but here's a resampling from the first half of the axes.wav I posted up above:

This is what it looks like with SDL2's resampler...the garbage is less severe, either because we made fixes or this is working from data we already resampled once, I don't know, but you can see there's still garbage in the spectrogram in any case, especially at 48000Hz...

So I think we're good to go in SDL3.

There's no chance we're backporting this to SDL2, though, as all this code was completely rewritten. Even discounting the idea of migrating bugfixes to SDL2 here, I don't think I could even drop the new resampler code into SDL2 as a complete replacement, in any reasonable way. But for games not migrating to SDL3, sdl2-compat will take advantage of the new resampler.