search

love2d / love

LÖVE is an awesome 2D game framework for Lua.
https://love2d.org
Other
5.06k stars 401 forks source link

Audio source :tell() gets out of sync for streaming sources #1423

Closed slime73 closed 5 years ago

slime73 commented 6 years ago

Original report by fluffy (Bitbucket: fluffycritter, ).

I am building a game where I need the in-game action to stay in sync with the music; in-game events are timed to the music playback position, as determined by source:tell(). Unfortunately as of LÖVE 11.1 the rate at which tell() moves forward on streaming sources lags behind the actual music playback rate, and so the in-game events drift considerably.

If I seek the music with source:seek() then the playback position and tell() position line up for a while but then they drift again.

As a workaround I can set the sources to be static instead of stream and everything stays in sync, but then of course I'm using a lot more memory for the music (upwards of 60MB, which I guess isn't that big a deal these days but still, it's the principle of the matter).

This worked fine in LÖVE 0.10.2, so this feels like something of a regression. I imagine that the major changes in the audio subsystem had something to do with this. :)

For an example of this working or not, you can see the following revisions of my game project:

The difference is obvious in track 1 (Little Bouncing Ball). If you enable debug mode (in the userconf.lua that's written into the config directory) you can press . to skip forward to the next section of the song and see how things return to sync but then it drifts again by the end of the section.

Also this appears to be CPU-speed-dependent; things don't drift as much on my current-gen iMac as they do on my old Mac mini.

I have only tested this on macOS; other OSes may experience things differently.

slime73 commented 6 years ago

Original comment by Gabe Stilez (Bitbucket: z0r8, ).

One solution to this could be a bit more complicated, but much more precise;

Now, in update or somewhere that gets called once each frame/tick, decode some data, and queue the sounddata object returned up to the qsource object (and call :play on that). You can also keep track of the elapsed samplepoints this way, hence, you can calculate the elapsed milliseconds; this might give you more precision.

slime73 commented 6 years ago

Original comment by fluffy (Bitbucket: fluffycritter, ).

That seems a lot more complicated than simply fixing streaming audio in the engine. :) It's good to know about that functionality for other purposes though, thanks!

slime73 commented 6 years ago

Original comment by Gabe Stilez (Bitbucket: z0r8, ).

Whether or not :tell being broken notwithstanding, the solution i proposed will always be more precise, since you would be manually counting samplepoints, basically; with :tell, you can never be sure.

But, of course, you are right in that if there's a regression with :tell (or :seek, whichever might cause your desync issue), it should be fixed.

slime73 commented 6 years ago

Original comment by fluffy (Bitbucket: fluffycritter, ).

Ah, yeah. I appreciate that, although I don't need that level of precision; as long as it's within like 100ms or so it's fine. :)

The desync issue is almost certainly with :tell, as :seek fixes the synchronization. In the case of Little Bouncing Ball the desync is very obvious; the scheduled events come many beats after the moment in the music, but skipping forward to the next music section plays the correct music for that event. (The other games so far in Refactor also use musical synchronization but it's not quite as visibly-obvious when synchronization is lost.)

slime73 commented 6 years ago

Original comment by Mi 28 (Bitbucket: rcoaxil, ).

The bug lies in the fact that internally Sources use somewhat backwards way of tracking playhead positions and passing them around. It's an easy fix but seems no one is bothered to do it.

slime73 commented 5 years ago

Original comment by Alex Szpakowski (Bitbucket: slime73, GitHub: slime73).

Is there a simple .love file that demonstrates the problem, which we can test? Also does it still happen with the latest 11.3 nightlies? There have been a few fixes to streaming sources recently, and it’s possible this issue was incidentally fixed by one of them.

slime73 commented 5 years ago

Original comment by fluffy (Bitbucket: fluffycritter, ).

I will try to make a minimal reproduction case for 11.2 and test it against 11.3.

slime73 commented 5 years ago

Original comment by fluffy (Bitbucket: fluffycritter, ).

Okay, I’ve made a reproduction case at http://beesbuzz.biz/static/temp/love-stream-repro.love (unfortunately the file is too large for Bitbucket to let me attach it). At least on my system the music is quite clearly out of sync with the bouncing within 12 measures or so, and it just gets worse as time goes on, both on 11.2 and on the latest 11.3 nightly (macOS).

slime73 commented 5 years ago

Original comment by fluffy (Bitbucket: fluffycritter, ).

I ended up replacing the file with a test which shows things more clearly; there are two players, one static and one streaming. The blue ball bounces based on the static player, and the yellow ball (and background flash) bounce based on the streaming player. Both players play at 50% volume and the actual audio playback stays in perfect sync.

slime73 commented 5 years ago

Original comment by Alex Szpakowski (Bitbucket: slime73, GitHub: slime73).

I tested it a bit on my macOS system with my local build of 11.3 and so far I haven’t been able to notice any desync issues (and I get 0.000000 drift even after leaving it running for a minute). I’ve uploaded my local build of 11.3, including a version of OpenAL Soft that has some fixes compared to 11.2, if you want to try that: https://bitbucket.org/slime73/love_macbin/get/tip.zip

slime73 commented 5 years ago

Original comment by fluffy (Bitbucket: fluffycritter, ).

Ah, nice! That build seems to be stable for me as well.

slime73 commented 5 years ago

Original comment by Alex Szpakowski (Bitbucket: slime73, GitHub: slime73).

Glad to hear it!

slime73 commented 4 years ago

Original comment by fluffy (Bitbucket: fluffycritter, ).

For what it’s worth I just tried the current official build of 11.3a for macOS and the sync issue seems to still be present, although the slime73 build tip still has the fix. Was that release cut before this fix went in?

In case you still need the test I’ve uploaded it the code at https://gist.github.com/fluffy-critter/2de489091a05f6a3746bef44e46e3025 - edit the mp3 filename and (optionally) BPM accordingly for whatever song you have at the ready.

Thanks!

slime73 commented 4 years ago

Original comment by fluffy (Bitbucket: fluffycritter, ).

Wait never mind, I was being silly and had the wrong löve binary in my ~/bin symlink. False alarm!