Wav/WavStream audio crackling

leiradel commented 6 years ago

Expected behavior:

Perfect audio.

Actual behavior:

Audio crackling, right channel only.

Steps to reproduce the problem:

Load a WAV or OGG using either Wav or WavStream, start playing and mixing. It's easier to hear the crackling with a low volume for the voice.

SoLoud version, operating system, backend used, any other potentially useful information:

#define SOLOUD_VERSION 201800

I'm using the null backend to mix the sound to use in a libretro core. I've just integrated SoLoud, before I was using my own audio mixer with WAV and OGG support only and the mixing was ok. I was using stb_vorbis too, but it doesn't seem to be related to it wince WAV files also present the issue.

leiradel commented 6 years ago

Hi, I'm also playing Sfxr and Speech sounds and they sound ok. It's odd that the crackling seems to be in the right channel only.

Everything in my setup is pretty standard, I'm clueless about what could be causing the issue.

leiradel commented 6 years ago

sketch008.zip

This file has one of the OGG files that I tested, and a WAV file with the output of mixSigned16.

leiradel commented 6 years ago

Some more info: I'm using a SoLoud::File subclass that wraps a PHYSFS_File. To make sure that wasn't the source of the issue, I tried putting the OGG in an array and loading it via a SoLoud::MemoryFile and the issue persisted.

I would be surprised if that was the problem since any issues while reading the OGG would cause the loading to fail, but it does load and play, only with the crackling.

Also, I'm compiling SoLoud along with my source code. Those are the SoLoud files I've added to the Makefile:

       soloud/src/audiosource/sfxr/soloud_sfxr.o soloud/src/audiosource/speech/soloud_speech.o \
       soloud/src/audiosource/speech/darray.o soloud/src/audiosource/speech/klatt.o \
       soloud/src/audiosource/speech/resonator.o soloud/src/audiosource/speech/tts.o \
       soloud/src/audiosource/vizsn/soloud_vizsn.o soloud/src/audiosource/wav/soloud_wav.o \
       soloud/src/audiosource/wav/soloud_wavstream.o soloud/src/audiosource/wav/stb_vorbis.o \
       soloud/src/backend/null/soloud_null.o soloud/src/core/soloud_audiosource.o \
       soloud/src/core/soloud_bus.o soloud/src/core/soloud_core_3d.o soloud/src/core/soloud_core_basicops.o \
       soloud/src/core/soloud_core_faderops.o soloud/src/core/soloud_core_filterops.o \
       soloud/src/core/soloud_core_getters.o soloud/src/core/soloud_core_setters.o \
       soloud/src/core/soloud_core_voicegroup.o soloud/src/core/soloud_core_voiceops.o \
       soloud/src/core/soloud.o soloud/src/core/soloud_fader.o soloud/src/core/soloud_fft.o \
       soloud/src/core/soloud_fft_lut.o soloud/src/core/soloud_file.o soloud/src/core/soloud_filter.o \
       soloud/src/core/soloud_queue.o soloud/src/core/soloud_thread.o \
       soloud/src/filter/soloud_bassboostfilter.o soloud/src/filter/soloud_biquadresonantfilter.o \
       soloud/src/filter/soloud_dcremovalfilter.o soloud/src/filter/soloud_echofilter.o \
       soloud/src/filter/soloud_fftfilter.o soloud/src/filter/soloud_flangerfilter.o \
       soloud/src/filter/soloud_lofifilter.o soloud/src/filter/soloud_robotizefilter.o \
       soloud/src/filter/soloud_waveshaperfilter.o

The only SoLoud-specific flag I'm passing to the compiler is -DWITH_NULL.

Some more relevant information:

$ g++ --version
g++ (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ uname -a
Linux leiradel-Lenovo-G40-80 4.13.0-43-generic #48~16.04.1-Ubuntu SMP Thu May 17 12:56:46 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

jarikomppa commented 5 years ago

This was probably fixed by: https://github.com/jarikomppa/soloud/commit/b54020bfcbed24df19406a06fdbc32403150cc2d because you said the crackling was in right channel only and that mono sources worked. Please reopen if this wasn't the case.

leiradel commented 5 years ago

Thanks for responding. I've pulled the latest code from the repository and the issue persists.

jarikomppa commented 5 years ago

Ah, I thought the problem was that there's weird crackling on the right channel, but now that I check your waves.. there's only sound on the right, and it's crackling.

The issue can't be about the file loading, because then it just wouldn't work.
I doubt this is a platform-specific issue, but you never know.
You're using the null backend, so it's (probably) not the back end.
Symptoms: Mono sources work. Stereo sources act up.

I'd love to be able to reproduce the bug..

Can you compile and run sanity? (src/tools/sanity, link against soloud sources). Sanity doesn have a test to check if left or right channels are quiet, though, so it probably doesn't hit this specific issue, but at least if something pops up, we know it's a portability issue.

jarikomppa commented 5 years ago

Further elimination: since mono sources work, it's not about the 16 bit mix function. I've also played the ogg through various backends with no problems, so it's not the ogg decoding. The ogg file itself is also fine. Playing the ogg through the null driver looks fine too.

jarikomppa commented 5 years ago

This produced a raw file that I could import into audacity and play fine:

#include <stdlib.h>
#include <stdio.h>
#include "soloud.h"
#include "soloud_wav.h"

int main(int argc, char *argv[])
{
    SoLoud::Soloud soloud;
    SoLoud::Wav wav;
    wav.load("audio/sketch008.ogg");
    soloud.init(SoLoud::Soloud::CLIP_ROUNDOFF, SoLoud::Soloud::NULLDRIVER);
    soloud.play(wav);

    FILE * f = fopen("audiodump.raw", "wb");

    while (soloud.getActiveVoiceCount() > 0)
    {
        short ibuf[256 * 2];
        soloud.mixSigned16(ibuf, 256);
        fwrite(ibuf, sizeof(short), 256 * 2, f);
    }
    fclose(f);

    soloud.deinit();
    return 0;
}

leiradel commented 5 years ago

Can you compile and run sanity?

$ ./sanity 
lastknown.wav not found, writing one.
SoLoud version 201800
Backend 13: null driver, 2 channels, 44100 samplerate, 2048 buffersize

168 tests, 0 error(s) 
lastknown.wav written.

$ ./sanity 
SoLoud version 201800
Backend 13: null driver, 2 channels, 44100 samplerate, 2048 buffersize

241 tests, 0 error(s)

I also tested the raw file converted to a wave by Audacity. In audacity the audio is correct, but the crackling persists in my application. I must be doing something wrong on my side, probably something in my audio backend. I use PhysicsFS, but I'm pretty sure I'd had a much bigger problem if it was messing up with the OGG reading.

Sorry for the false positive.

jarikomppa commented 5 years ago

I still don't get it though; you said mono sources worked, and you recorded off the mix function..

Fwiw, right channel seems to be the first one.

If you can figure out what was wrong, I'd love to hear about it; if you find that the problem is on soloud's side, a minimal test source would be nice.

On Sat, Nov 10, 2018, 01:02 Andre Leiradella <notifications@github.com wrote:

Closed #183 https://github.com/jarikomppa/soloud/issues/183.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/jarikomppa/soloud/issues/183#event-1958041309, or mute the thread https://github.com/notifications/unsubscribe-auth/AEQ_R15KY8K7suRBn6rEnVchn8aZSjtEks5utgmPgaJpZM4Uqf3N .

leiradel commented 5 years ago

Ok, I've tried somethings and the cracks were always there:

I've dumped the raw buffer contents right after the call to the mixing function
- Tried Stereo S16, stereo F32, mono S16
- No tampering with the samples
I've zeroed the buffer before calling the mixing function
- Just in case
I've used double buffering in my mixing function
- Maybe the code that was sending the samples to the hardware was messing with the contents of the buffer
- Unlikely because it copies the samples to a different buffer
I've used standard SoLoud IO by directly passing the path to the OGG file on disk
- No PhysicsFS involved
I've converted the OGG to WAV and used it in the application
- No OGG decoding involved
I've removed all other sounds and let only the OGG playing
- No mixing of multiple voices

The only time it worked was when I set the OGG volume to 1.0f, but then I don't know if it really worked or if the cracks were just inaudible. I've converted the OGG to raw using a minimal application using a volume of 0.2f and it worked.

All source code was always compiled with the same compiler flags.

I'm puzzled. If you have any other test that I can make please let me know.

jarikomppa commented 5 years ago

You say it worked when you set the ogg volume to 1.0f. What are you normally setting it to?

On Sat, Nov 10, 2018 at 1:30 PM Andre Leiradella notifications@github.com wrote:

Ok, I've tried somethings and the cracks were always there:

I've dumped the raw buffer contents right after the call to the mixing function

Tried Stereo S16, stereo F32, mono S16

No tampering with the samples

I've zeroed the buffer before calling the mixing function

Just in case

I've used double buffering in my mixing function

Maybe the code that was sending the samples to the hardware was messing with the contents of the buffer

Unlikely because it copies the samples to a different buffer

I've used standard SoLoud IO by directly passing the path to the OGG file on disk

No PhysicsFS involved

I've converted the OGG to WAV and used it in the application

No OGG decoding involved

I've removed all other sounds and let only the OGG playing

No mixing of multiple voices

The only time it worked was when I set the OGG volume to 1.0f, but then I don't know if it really worked or if the cracks were just inaudible. I've converted the OGG to raw using a minimal application using a volume of 0.2f and it worked.

All source code was always compiled with the same compiler flags.

I'm puzzled. If you have any other test that I can make please let me know.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/jarikomppa/soloud/issues/183#issuecomment-437577409, or mute the thread https://github.com/notifications/unsubscribe-auth/AEQ_R8v3toHs8Eei1KSC8mc8Sd5T7tvUks5utrjkgaJpZM4Uqf3N .

leiradel commented 5 years ago

0.2f, but I dumped the OGG to WAV using this volume and the result was ok.

jarikomppa commented 5 years ago

Try enabling floating point debug to see if NaNs creep in somehow: define FLOATING_POINT_DEBUG

On Sat, Nov 10, 2018, 19:23 Andre Leiradella <notifications@github.com wrote:

0.2f, but I dumped the OGG to WAV using this volume and the result was ok.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/jarikomppa/soloud/issues/183#issuecomment-437600431, or mute the thread https://github.com/notifications/unsubscribe-auth/AEQ_RzXUcnm2wVRffOli_yhLwWY8Ih-wks5utwuPgaJpZM4Uqf3N .

leiradel commented 5 years ago

I had to change the code to use feenableexcept since I'm using Linux, but then I got a FP exception on an unrelated code :(

My application is a libretro core and I'm using RetroArch to run it, I can't easily fix the exception, nor compile my application to run without RetroArch.

jarikomppa commented 5 years ago

This starts to be a complete shot in the dark, but maybe libretro is setting the fpu into some mode that causes soloud to generate NaNs in some cases..

Does your compiler have options for floating point accuracy? As in fast vs precise stuff. Those might be worth a try.

On Sun, Nov 11, 2018 at 2:51 PM Andre Leiradella notifications@github.com wrote:

I had to change the code to use feenableexcept since I'm using Linux, but then I got a FP exception on an unrelated code :(

My application is a libretro core and I'm using RetroArch to run it, I can't easily fix the exception, nor compile my application to run without RetroArch.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/jarikomppa/soloud/issues/183#issuecomment-437667700, or mute the thread https://github.com/notifications/unsubscribe-auth/AEQ_R7jR0ppO8bYY7E4-YKmLClolx9U2ks5uuB1jgaJpZM4Uqf3N .

lamelizard commented 5 years ago

Peace, I'm facing a mostly same problem:

using Null driver in a libretro core using mix() (not the 16bit version) the right channel crackles (I manually switched the channels in between soloud and retroarch -> it's the right soloud channel that crackles)

So I compiled with FLOATING_POINT_DEBUG on Windows x64 and ran it again, but nothing new happened. But I don't know what should have happened, does nothing mean that there are no NANs?

jarikomppa commented 5 years ago

Any chance you could zip up the whole project with visual studio project so I could see if it happens here too? The winning combination seems to be libretro + soloud null driver..

On Sun, Nov 11, 2018 at 6:08 PM lamelizard notifications@github.com wrote:

Peace, I'm facing a mostly same problem:

using Null driver in a libretro core using mix() (not the 16bit version) the right channel crackles (I manually switched the channels in between soloud and retroarch -> it's the right soloud channel that crackles)

So I compiled with FLOATING_POINT_DEBUG on Windows x64 and ran it again, but nothing new happened. But I don't know what should have happened, does nothing mean that there are no NANs?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/jarikomppa/soloud/issues/183#issuecomment-437682589, or mute the thread https://github.com/notifications/unsubscribe-auth/AEQ_R1F_tIpKjx_VaT7Y5E2JPVj9FjrIks5uuEtlgaJpZM4Uqf3N .

leiradel commented 5 years ago

This starts to be a complete shot in the dark

Yeah, I'm out of ideas too.

maybe libretro is setting the fpu into some mode

Unlikely. I've had trouble in the past with DirectX changing the FP rounding mode, but I'm on linux with OpenGL.

Does your compiler have options for floating point accuracy?

It does, but I don't really think it could be the reason of the cracks.

@lamelizard if you dump raw samples right after mix and open the file in Audacity, are the cracks there?

jarikomppa commented 5 years ago

This is one ping-pong open-close issue but I want to see how deep this rabbit hole goes..

lamelizard commented 5 years ago

@leiradel Yes, I dumped with sndfile, opened in audacity -> crackling Edit: I'm not sure, but the dumped wav seems to have parts where output is zero / close to zero for several samples (right channel only). @leiradel is it the same for you?

jarikomppa commented 5 years ago

Again, if either of you can either zip me up the project or give me instructions how to build one myself, I'd appreciate it..

lamelizard commented 5 years ago

Sadly my project has a plugin manager and several abstration layers between soloud and libretro plus a custom cmake file for soloud... So my guess is that @leiradel 's core might be far easier to test with. @leiradel, your retroluxury core was it what you opened this issue for?

leiradel commented 5 years ago

@leiradel, your retroluxury core was it what you opened this issue for?

Yes, I'm removing my WAV/OGG audio mixer in favor of SoLoud.

Again, if either of you can either zip me up the project or give me instructions how to build one myself, I'd appreciate it..

I'm not sure how to make it compile with MSVC. The code doesn't have anything platform dependent: https://github.com/leiradel/retroluxury

There's a Makefile in the src folder which lists all files that must be compiled, and has the defines I use to build libretroluxury.a. It should be simple to create a project in MSVC to build a .lib.

The test folder has another Makefile to build the libretro core as a .so. Again, it should be easy to create a project that compiles the source code and links with the .lib to create a .dll.

libretro.c has a hard coded path to the pack.zip file created along with the .so, so it must be changed. pack.zip is just a zip file with the contents of the res folder without any folder names.

You have to have RetroArch installed to run the core:

C:\> retroarch -v -L test.dll

Running this from inside Visual Studio or attaching to the process should let you debug the mixer, which is called from within rl_sound_mix in rl_sound.cpp.

Let me know if I can help further.

jarikomppa commented 5 years ago

Got it compiled and running under visual studio. There's definitely some noise on the right channel, but the left channel isn't quiet.. anyway, this is something I can look at.

jarikomppa commented 5 years ago

Righty-o. The requested number of samples is not divisible by 4, but the clip function expects that as it uses SIMD to handle four samples at a time. 735 samples / 4 = 183 * 4 = 732, so a few samples get missed. Fix is to round things up, and process a few samples too many (these are scratch buffers so there's room for a couple extra samples). Fix is in 487c0140509576235d3d5b7c589b552e4ee160f4

jarikomppa commented 5 years ago

I'm waiting for confirmation before closing this, even though that was a pretty clear bug.

leiradel commented 5 years ago

Ah yes, the core runs at 60 FPS and sets the audio output to 44100 Hz, so it wants 735 stereo samples per video frame.

Thanks so much for looking into this, I'll test the fix later and will let you know.

jarikomppa commented 5 years ago

fwiw for visual studio build, I dumped all the source files to a visual studio project, set up paths, removed all restrict keywords, added dllexport declspecs to all of the libretro functions, and the thing compiled. Didn't get the zip files to work but using a direct directory did.

leiradel commented 5 years ago

I confirm the fix works for me, thanks again.

jarikomppa commented 5 years ago

Thanks.

jarikomppa / soloud

Wav/WavStream audio crackling #183