UStreamingSoundWave and GetPlaybackTime

[keenua]

Hi,

Let me start by thanking you for such a great library!

Now to the question.

I'm using UStreamingSoundWave to play streaming audio. In order to match my automatically generated animations with the sound, I call GetPlaybackTime and set the frame time of my animation. However, there seems to be difference between the playback time that UStreamingSoundWave reports and the sound that is actually played. It results in the animations being played a few hundred miliseconds (or even seconds) earlier than the sound.

This is the stripped down version of my code:

void AMyController::Tick(float DeltaTime)
{
    Super::Tick(DeltaTime);

    FrameTime = SoundWave->GetPlaybackTime();

    Frame = GetFrame(FrameTime);
    SetMotion(Frame->Motion);

    if (GetSound(FrameTime).Num() > 0)
    {
        SoundWave->AppendAudioDataFromEncoded(GetSound(FrameTime), ERuntimeAudioFormat::Mp3);
    }
}

I populate the audio every i.e. 120 frames, and it plays just fine, except for the delay. I assume that's because AppendAudioDataFromEncoded is async and it only adds it sometime later, however I was hoping that GetPlaybackTime would take it into account. Could it be that GetPlaybackTime is reporting it incorrectly? Or am I using it wrong?

It's worth noting that my actual code is a bit more complicated than the sample I provided, so there's a chance there are other things in play. However, I've been meddling with it for a couple of days and can't find anything that can explain the delay.

Thanks in advance!

[keenua]

This is how I create the sound wave, in case you think it's important:

[keenua]

Apparently switching from mp3 to ogg fixes the issue. Not sure why is that, but I guess my issue is resolved.

[gtreshchev]

Hi, thank you for your feedback!

Regarding the issue, when you call AppendAudioDataFromEncoded, the importer takes some time to decode the audio data and update the internal PCM buffer. This, in turn, affects the value returned by GetPlaybackTime. As this process occurs on an asynchronous thread, you need to wait for the PCM buffer to be fully updated before obtaining the new playback time. You can use the delegate OnPopulateAudioDataNative, which is triggered when the append operation is fully complete, to handle this situation more effectively.

[gtreshchev]

Apparently switching from mp3 to ogg fixes the issue. Not sure why is that, but I guess my issue is resolved.

It might be because the mp3 format is decoded faster than the ogg format. But anyway, relying on decoding speed is not a reliable solution, I recommend redesigning your implementation to use the OnPopulateAudioData delegate or somehow else.

[keenua]

Thanks, I'll try that as well.