Too fast emulation speed causes sound problems

[thrust26]

When selecting an emulation speed which cannot be handled by the CPU, the sound becomes extremely distorted and even the lowest Audio setting cannot cure that. It seems that the video emulation eats up all cycles required for audio.

BTW: The video seems to skip frames causing a slightly stuttering display, but that may be a false impression.

For testing, I suggest Debug mode and River Raid. This runs at ~90 fps max on my machine then.

[DirtyHairy]

I wouldn't classify this as a bug, but as expected behavior. If you set the emulator to 10x speed, it will try to emulate a console running at that speed, and the audio code will assume that samples a delivered from the emulation core at ~370kHz. If the speed achieved by the system is slower than that, the audio code will run through the samples actually delivered too quickly, and you will get audio underruns.

While it is waiting for new audio data, the audio driver will resort to playing back the last samples again. If the difference between actual and target speed is large, audio will sound severely distorted.

[thrust26]

I suppose you have to hear it yourself to understand what I mean. This is no normal distortion but completely corrupted sound. When you are on the edge of maximum speed settings it becomes more obvious what is going on. E.g in my case, the game maxes at ~90 fps. So with a speed of 150% I have correct and completely corrupted sound on and off.

I think what we might be missing is something acceptable like video frame dropping for the sound. And maybe a higher priority for the sound over video, because the effect is so terrible.

[DirtyHairy]

I'll try it myself later. However, it is important to realize that, even if the game maxes out at 90fps, the sound core will still assume that samples will be delivered at the target speed. if you set speed to 10x and get 90fps, there won't be anything to play back 85% percent of the time. The audio code will try to cover up by looping through old sample data before it gets new data, but I can totally image that this might sound terrible if the discrepancy is that high.

[thrust26]

I think even with minimal discrepancy between requested speed and possible speed, the sound gets terrible. Of course no problem on our machines, but slower ones may suffer badly.

Isn't there a way to adapt to the actual speed?

[DirtyHairy]

I think even with minimal discrepancy between requested speed and possible speed, the sound gets terrible. Of course no problem on our machines, but slower ones may suffer badly.

If there is a discrepancy, you will get audio stuttering; that's unavailable --- it is just like decoding a movie and not managing to decode at full framerate. I just checked (my system even maxes at 70fps with -O0), and the result is pretty much what I imagined. If you go to high speeds, it gets particularly bad as the sound will be stuttering and very high pitched.

Isn't there a way to adapt to the actual speed?

I have thought about this before and have come to the conclusion that this would be much more fragile and prone to audio issues than the current approach.

[thrust26]

IMO video skipping frames is not a bad thing compared to the sound distortions. IIRC, both audio and video are done on their own dedicated threads, correct?

If so, I would prefer that audio is kept correct at the cost of extra frame skips. So IMO we maybe should do audio first and video only later on. That would keep the emulation acceptable a bit longer, wouldn't it?

[DirtyHairy]

Skipping video frames won't help I think, at least not on machines with more than on core. The separation between the threads is not audio and video, but emulation and frame rendering 😉 The only reason why we need two threads is because video updates block.

Emulation is CPU bound, and if the system is too slow, then skipping frames will not free up any time on the emulation thread.

[DirtyHairy]

I should add that I have no trouble going to 10x speed with an optimized build on my system, so I think the discussion of systems that are too slow to run at 100% speed is likely to be academic 😉 Even then, you could just try to lower speed to 90% and try to get away with it.

[thrust26]

OK, then there is little we can do without major changes.

My system maxes out at ~560fps (River Raid, Direct3D, no TV effects) with 37% total CPU load (i5-3570K@4.1MHz). If my math is right, this is by about a factor of 25 fast enough for 100%.

Not sure if the problem is purely academic. ARM and Atom chips can be really slow.

[DirtyHairy]

I agree that it would be interesting to get a data point from a raspberry pi or similar.

However, I don't think Stella 6 is actually slower than Stella 5.1, so if Stella 6 runs too slow, Stella 5.1 would run too slow, too. The difference is that the new sound code is much less forgiving than the old code. If you reduce emulation speed on such a system to match the actual speed achieved, however, audio should be fine.

[thrust26]

More forgiving meant going out of sync. Not exactly a better solution. :smile:

[sa666666]

Just seeing this thread now. I wonder why on @thrust26 system the system is maxing out but only 37% CPU is utilized. Shouldn't maxing mean that most of the CPU is being used? It seems kind of low; the system is less than half utilized and already runs into problems??

[DirtyHairy]

I guess this is not per CPU core, but total load.

[sa666666]

Barring anything that can be done to make the effect less obnoxious, I think we just need to document that if you try to run the emulation too fast, at some point the system just won't be able to handle it, and sound/video may come out of sync.

It's really no different than if your computer is not fast enough to run an app. At some point you just have to say "your computer is too slow to do that".

Now if we can improve the current approach and make the code a little more forgiving, then that may be an option. But if we can't, I guess we just document this as a feature, not a bug :smiling_imp:

[thrust26]

I guess this is not per CPU core, but total load..

Yup, total load (4 cores). I suppose one core (thread) is running at ~100% and the other one at ~50%. I cannot easily measure this, because Windows moves the threads around.

[sa666666]

BTW, on my development system I can get to 820% and sound stays in sync, without any distortion. Any more than that, when the effective fps goes over 500 or so, and the problem occurs. At 820%, I see that one core is 97% loaded, so just enough to handle it.

So I would say as long as a system can run at 60fps, then we never need to worry about it. If we do come across such systems, then we can perhaps try to optimize or do things differently, etc.

[thrust26]

That means my 6 yo single cores are still faster than yours? :smile:

Anyway, (even though most people never read it) I agree documentation can handle this for now. I just opened the issue because I hoped for a simple fix.

[sa666666]

Yep, the one place that Intel is still better than AMD (for now at least). Intel can do better single-threaded performance, but AMD can do better multi-threaded (more cores). For software development many cores is much better, but for some apps (especially emulators and games), single-core performance is still king.

So my 8C/16T system can max at 4.1GHz/core, but your system can probably do 4.5Ghz/core.

[thrust26]

4.2 GHz is the max here. I guess I could go higher, but its not worth the hassle.

[sa666666]

Just looked it up: 3.6GHz/core, turbo of 4GHz. So aggregate speed is much faster (8/16 cores @ up to 4Ghz each), but single-core speed is still slower than yours. Also, Linux vs. Windows graphics driver performance may play a part too. And I'm running at 8x zoom on a 4K monitor, so that may change things too :smiley:

In any event, much faster than Stella will ever need.

[sa666666]

One thing I wanted to mention to @DirtyHairy though. At some point (not necessarily before 6.0), I'd like the SOUND_SUPPORT define to be honoured by the new sound core. So when we disable sound support, we disable it completely. The reason for this is that some 3rd-parties are using Stella in experiements, and try to get as fast a framerate as possible, and turn off sound to do so. ALE (Arcade Learning Environment) is one such group, that uses AI to run 2600 games.

Maybe I should create an issue for it.

Anyway, stepping away from the computer for most of today; have more work to do on the house.

[DirtyHairy]

Interesting, wasn't aware of it. That's pretty much trivial, I'll add it for 6.0

[thrust26]

I have just created an issue.