search

mupen64plus / mupen64plus-core

Core module of the Mupen64Plus project
1.31k stars 257 forks source link

Goldeneye audio delay problem. #397

Open AmbientMalice opened 7 years ago

AmbientMalice commented 7 years ago

Basically, the game's audio goes out of sync fairly quickly. Demonstrable by idling on mission select screen for 5 mins or so. Azimer's had/has a similar problem.

https://github.com/Azimer/AziAudio/issues/182

I assume this is a recent aggression caused by some of the accuracy improvements.

Narann commented 7 years ago

Thanks! Can you provide more information about your setup? ARM? x86? With which video plugin?

AmbientMalice commented 7 years ago

64 bit version of Logan's m64p+GLideN64+GUI. GLideN64. CPU is an i5. I noticed it while testing that new Goldfinger romhack, and confirmed by testing the original game.

loganmc10 commented 7 years ago

I'll look into/bisect this. Just so I know what I'm looking for @AmbientMalice how much delay are we talking? If I let it idle at the mission select screen for 5 minutes, how off can I expect it to be?

AmbientMalice commented 7 years ago

Maybe 1-2 seconds behind?

fzurita commented 7 years ago

Hmmm, I wonder if Mupen64plus-AE has the same problem. It has its own audio plugin.

This sounds a lot like you are trying to set an audio sampling rate requested be the game, but the hardware can only give you an approximation. If there is any difference in the sampling rate between the game and the hardware, audio will slowly drift away from the game. The easy solution is to resample audio into a frequency that the hardware allows.

If I remember correctly m64p's audio plugin is not doing any resampling.

And one more thing @loganmc10 , I highly recommend the SoundTouch library. In addition to doing resampling, you also get time stretched audio.

loganmc10 commented 7 years ago

A couple weeks back I switched back to the upstream audio plugin, so it is doing resampling. The plugin works pretty well except for the poor performance of audio sync, which is disabled by default.

fzurita commented 7 years ago

I just tested Mupen64plus-AE, I was not able to reproduce the issue there. I was in the mission select screen for about 20 minutes.

AmbientMalice commented 7 years ago

I think this issue might be inconsistently triggered. I'll have to do some further testing. Apologies if it turns out to be a false alarm, but the issue was reproducible a few days ago.

loganmc10 commented 7 years ago

I'll try and test this tomorrow, but it might be good to see if any of the audio plugin settings make a difference, like enabling audio sync or decreasing the size of the buffers

AmbientMalice commented 7 years ago

I did some testing ingame, and I'm fairly certainly there's a latency that creeps in for weapon firing sounds. (This was what I first noticed when testing Goldfinger.) I'm guessing maybe 100-200ms of extra delay between muzzle flash and firing sound over the normal delay. It could possibly get longer with more playtime. I wonder whether things like saving/loading are a contributing factor, too.

loganmc10 commented 7 years ago

I was able to reproduce this. It didn't happen the first time I tested, but it did the 2nd. Pausing and unpausing the game fixed it (allowed the audio to catch up).

Decreasing the size of the audio buffer will fix/reduce this issue (changing the value of PRIMARY_BUFFER_SIZE in the audio plugin settings). However, doing that will trigger more audio buffer overflows. I believe this is probably what's going on in Azimer's audio, since in the issue you references, they said there was occasional stuttering/popping. I reduced PRIMARY_BUFFER_SIZE to 4096 and it kept the audio in sync (it did trigger a number of buffer overflows, but I didn't hear anything strange).

If you enable AUDIO_SYNC in the audio plugin that should fix the problem as well.

fzurita commented 7 years ago

A bigger question, why is the game getting ahead of the audio? Could we be running too fast in some instances?

loganmc10 commented 7 years ago

Yeah, but in this case it's not consistent, it's not like the game is constantly running faster than the audio, something triggers it, I can't reliably reproduce it. What does the SLES plugin do in these cases? If time stretch was enabled and sync to audio disabled, would it slightly speed up the audio to keep up?

fzurita commented 7 years ago

Yes, it would slightly speed up the audio to keep the buffer size small. It's very slight though, it would be hard to perceive.

loganmc10 commented 7 years ago

@AmbientMalice I've updated the m64p package to include the "sdl2" audio plugin. It'll keep the audio in better sync, in this case, it will cause a bit of "popping" when the audio gets too far behind. Using a library like SoundTouch to stretch the audio would be a good solution, perhaps I'll look into it in the future.