Video delay, and visualisations (and playback speed)

[RJVB]

I just noticed that visualisations do not take the configured delay into account, causing a noticeable lag when listening over my BT speaker. It's a nice speaker for the place it's in (Klipsch The One Mk2) but it uses the basic audio codec, so I have to set a "video delay" (it's actually a transport delay of course) of +0.3 seconds.

More importantly, I wonder if there's something that can be done to improve the experience at the beginning.

• In videos where the sound track plays at frame 0 it sounds as if a tiny bit of sound gets lost when first starting the video
• Curiously, the video always starts first

That latter symptom is something I cannot really get my head around when the correct transport has been set, and if I were to guess I'd say that this initial bit of "silent movie" is actually longer than those 0.3 seconds.

EDIT: something strange goes on when reducing playback speed, regardless of whether this is done keeping the audio pitch constant or not. At 0.75x speed I need to configure a transport delay of about 0.1 seconds only, and at 0.5x it has to be set to 0. It looks like speeding up the playback doesn't require changing the transport delay though of course it becomes increasingly difficult to detect sync issues.

[zaps166]

Bluetooth speakers has variable audio latency - depends on codec and on signal strength - low signal might increase latency because of larger buffer to prevent audio stuttering (and also can lower the bitrate).

Use cable whenever possible (many BT speakers has JACK input). With cable you're always full quality, lowest and fixed latencies and you can see your "signal" range.

Visualizations doesn't respect video latency - I don't want to buffer it there and complicate things. It's not video anyway :smile:

About playback speed - I don't know, but I think when speed is too low, the latency doesn't matter that much. When speed is too high, you should still hear the latency. Also rubberband filter has latency, but it should be implicitly added to video delay.

[RJVB]

Use cable whenever possible (many BT speakers has JACK input). With cable you're always full quality, lowest and fixed latencies and you can see your "signal" range.

Well, DOH, but a cable isn't always feasible, is it. Can't just span one the middle of a (kitchen) table across the space to where my speaker sits...

Visualizations doesn't respect video latency - I don't want to buffer it there and complicate things. It's not video anyway :smile:

For visualisations it's one thing, for the FFT spectrum another, IMHO. I don't know how you implemented the former, but I suspect you have to buffer for the latter anyway.

About playback speed - I don't know, but I think when speed is too low, the latency doesn't matter that much.

I don't see how that could make sense, unless the slowing down introduces its own delay w.r.t. the audio. Which might be the case since there is indeed a noticeable sync problem when I play the audio of a slowed-down video over the builtin speakers.

When speed is too high, you should still hear the latency. Also rubberband filter has latency, but it should be implicitly added to video delay.

Is the rubberband filter always in the loop, with just an on/off switch to determine whether it does something to pitch? Because that noticeable sync problem mentioned above is there whether I keep audio pitch constant or not.

I think I had actually noticed it before, as the slowed-down playback isn't as helpful in figuring out what musicians are doing with their hands (which is what I hoped to use the constant-pitch feature for).

I'm attaching the audio/video sync test I use - and just FWIW: visualisations and FFT appear to be completely out of sync with the audio for me.

https://github.com/user-attachments/assets/d106101a-c14f-41c5-b1bb-407fe583d8de

[zaps166]

For me this video plays perfect in QMPlay2 regardless of keep pitch and playback speed (using PC's sound card and pipewire).

Is the rubberband filter always in the loop, with just an on/off switch to determine whether it does something to pitch?

Rubberband filter is running only when your playback speed != 1.0 and you have keep pich enabled. Otherwise it's unused, resources are freed.

visualisations and FFT appear to be completely out of sync with the audio for me.

How about video? As I told before - video delay parameter doesn't work in visualizations .

For visualisations it's one thing, for the FFT spectrum another, IMHO.

What do you mean? The only buffer there is to grab enough samples to display and/or calculate FFT. They should display at same time (I mean some milliseconds doesn't matter like 1-2x screen refresh rate or so).

[RJVB]

For me this video plays perfect in QMPlay2 regardless of keep pitch and playback speed (using PC's sound card and pipewire).

I suppose your PC is a lot faster than mine, but is there a reason to expect that slowing down the video increases computational load such that video playback becomes delayed w.r.t. audio playback? On the Linux system where I observed the sync issues I actually have to dial in a negative video delay when I slow down playback to 0.25x! I'll see what happens on my Mac.

For visualisations it's one thing, for the FFT spectrum another, IMHO.

What do you mean?

I consider visualisations mostly eye candy, but the FFT spectrum is supposed to correspond to what you're hearing. Showing it for something else doesn't really make sense (not much more than showing a random spectrum).

The only buffer there is to grab enough samples to display and/or calculate FFT.

Let's just stick to the situation where the video delay setting is used to dial in a transport delay in the audio playback (so it's a positive delay). In that case, can't you simply fill the buffer for calculating the FFT with audio samples from X seconds ago? Don't you keep any decoded audio data after it has been handed off to the audio device?

They should display at same time (I mean some milliseconds doesn't matter like 1-2x screen refresh rate or so).

Who are "they" here?

( 1 screen refresh is already almost 17 ms for a 60Hz display ;) )

[RJVB]

On Friday July 12 2024 13:12:52 Błażej Szczygieł wrote:

For me this video plays perfect in QMPlay2 regardless of keep pitch and playback speed (using PC's sound card and pipewire).

Basically idem on my Mac. At some point I thought I noticed a slight offset at 0.25x speed but couldn't reproduce that. A few important differences of course:

• VTB acceleration rather than VAAPI
• OpenGL rendering instead of Vulkan
• "direct" audio rendering rather than via Pulseaudio.

I did manage to build Pipewire on my Linux rig but haven't yet figured out how to start it instead of Pulseaudio. (I tried once to launch it in addition to assuming the daemons shouldn't bite but it messed up the sound volume levels until I restarted the PA daemon.)

Did you notice that QMPlay2 identifies the Twitch video as 120Hz in the Info panel instead of the actual 60Hz? ffprobe shows it's also a "120 tbr" (??) video track; maybe that's where the confusion comes from?