Drifting in case of differing sample rate or speed

[ghost]

Expected behavior:

Audio files play as expected when their sample rate differs from SoLoud's configured sample rate (ie playing 44100Hz files through SoLoud initialized at 48000Hz), and/or when their speed is set to something else from 1.0.

Actual behavior:

Audio files play at a slightly different speed from expected.

As a form of verification, I used ffmpeg to create a 44100Hz ogg file, played it back through SoLoud, and recorded it in Audacity. After around 3 minutes, the offset is determined to be 0.1 seconds by using the second track being the audio file as the reference:

Steps to reproduce the problem:

Initialize SoLoud at one sample rate, play back an audio file of a different sample rate.

SoLoud version, operating system, backend used, any other potentially useful information:

SoLoud version 20200207. OS: GNU/Linux, at least on Ubuntu MATE 20.04 and Pop!OS 20.10. Backend used: MiniAudio, though this issue seems to occur with any backend.

[ghost]

For clarification, this indeed also occurs if you play it back at some non-1.0-speed, say 1.5. The position as reported by getStreamPosition still keeps up correctly, so nothing wrong there, but the audio played back is incorrect. I think this may be a bug in the resampler.

[ghost]

More or less confirmed that this is due to accumulative errors in the resampler. I guess I'll dig around in the code to see how I can fix it.

[ghost]

I think the core of this issue is coming from void Soloud::mixBus_internal in the src/core/soloud.cpp file, where resampling happens. The code assumes that it will always be able to get the same number of samples in and out every time, which of course for sample rate ratios like 44100/48000 never is the case.

Span line 1638 -> 1657 is the following:

// Get a block of source data

int readcount = 0;
if (!voice->hasEnded() || voice->mFlags & AudioSourceInstance::LOOPING)
{
    readcount = voice->getAudio(voice->mResampleData[0], SAMPLE_GRANULARITY, SAMPLE_GRANULARITY);
    if (readcount < SAMPLE_GRANULARITY)
    {
        if (voice->mFlags & AudioSourceInstance::LOOPING)
        {
            while (readcount < SAMPLE_GRANULARITY && voice->seek(voice->mLoopPoint, mScratch.mData, mScratchSize) == SO_NO_ERROR)
            {
                voice->mLoopCount++;
                int inc = voice->getAudio(voice->mResampleData[0] + readcount, SAMPLE_GRANULARITY - readcount, SAMPLE_GRANULARITY);
                readcount += inc;
                if (inc == 0) break;
            }
        }
    }
}

which seems to be always feeding in the same number of samples SAMPLES_GRANULARITY, then resamples it out to a buffer of an integer length (rounded down from multiplying with sample rate ratio), which clearly isn't correct.

[ghost]

I went for a third attempt at fixing this today, and I am sorry to say that the code here is so wrong in so many ways, and I cannot even comprehend what the code is actually doing due to the lack of basic comments on frequently-used class fields.

I will not fix this myself as I initially wanted to, because I can't. I'm sorry.

[leduyquang753]

Having the same issue which affects my game where precise tempo matters, so I hope this will be addressed.

[ghost]

I'm experimenting with Allegro's audio addon (https://www.allegro.cc/) and it does not have this issue. You can use the audio addon alongside anything you want. I believe you can use this alongside SDL2/winit, no issues. Factorio uses it. The API is nice and dev-friendly, I recommend trying it and see how it turns out.

[ghost]

also @leduyquang753 nice to meet fellow vietnamese dev, how's it going?