Crackling sound on some cores

[Ize-dev]

I've tested various systems / cores and on some like snes9x and pcsx rearmed the sound is crackling alot.

Very noticable for example in snes9x while music plays, game doesn't matter.

Also thanks for your amazing project, all cores worked fine so far with minor issues :D

[Skurdt]

Unity audio seems to perform rather well compared to when this project started (they changed something in the backend or something).

To try that instead, you only have to enable the AudioProcessor and AudioSource components on the screen gameobject. If using your own scene, simply add the AudioProcessor component on your libretro instance gameobject, or in a child gameobject. Note that using this will not work for multiple running instances of emulators if they use a different samplerate.

Fixing audio issues is probably not something I'm capable of, unless someone with audio programming knowledge comes along to help, it's gonna be a while until I can fix any of it myself :)

[Ize-dev]

I've tested your suggestion and it's even worse, now with dropouts :( You can try it with Axelay for snes using snes9x core. The Konami logo already has dropouts and once you press start on the main menu it breaks totally.

Hopefully more people notice this project and can help, i know some c# but this is over my head unfortunately :D

[Skurdt]

The issue may come from the thread sleeping every so often. You could try removing the sleep in Bridge.cs but CPU usage for the thread will go to 100%. I'll try that later and try to find a proper way to make the thread sleep if it comes from there... The NAudio path should really be the most optimal.

[Ize-dev]

I've commented out the else statement and the sound definately improved alot with naudio. Native unity using audio processor, not so much. So i think you're on the right track.

[Skurdt]

The sleep is causing the issues. Besides some minor glitches, audio on my end works good enough for me without the sleep. I'll remove it for now but I still think some kind of thread synchronization is needed to help on the cpu usage (especially since this runs inside of Unity, and it itself needs resources for the scenes to work properly. A simple unlit quad is fine but it will probably won't scale well on a busy scene...). Having a better CPU should help though...

The unity audio path works pretty well with Super Mario All-Stars. For Axelay, it works only when commenting-out the part about setting the AudioConfig's samplerate. Basically it depends on the core but also on the game... some report proper/common samplerates some don't, among other weird things happening sometimes... Adding audio resampling would solve most of the issues, NAudio already sort of does that "automatically" but even there, issues can come up on some core/game combination where manually authoring this stuff should definitely help...

Unity had DSPGraph in the works, which would have helped a lot regarding latency but also proper audio handling, but they put that on hold unfortunately... Though I couldn't figure out how to make it work at all... (low level audio programming stuff)

Anyways, thanks for the feedback, this is very welcome :)

[XenuIsWatching]

I did some of my own investigation on this. This wasn't the sleep causing the issues but rather a fundamental issue with the Audio Resampler. It's not looping over the full range of samples that it needs to resample up to, so many samples are not processed in the full buffer and gets left over with what is curring in the audio sample buffer which is where that crackling comes from.

In other words, it's using the same source samples count as the value for the destination samples count which should not be equal if you are resampling to a different rate over the sample period of time.

[XenuIsWatching]

As for latency/performance, these types of functions are perfect for SIMD instructions such as ARM NEON or x86 SSE.

A resampler is simpler enough that something custom can be written in C/C++ for it..... but I'd have to do some investigation on how to compile and link in C/C++ code in to unity's C# code.

[Skurdt]

I replaced the NAudio backend for SDL3. With the very small amount of api calls used, the result sounds fine for me. But it does open more options for other platforms and possibly easier further audio processing.

[Skurdt]

As for latency/performance, these types of functions are perfect for SIMD instructions such as ARM NEON or x86 SSE.

A resampler is simpler enough that something custom can be written in C/C++ for it..... but I'd have to do some investigation on how to compile and link in C/C++ code in to unity's C# code.

With il2cpp only, you can actually put .h .c .cpp files in your project's assets subfolders. It will not work in editor's playmode but they will be compiled when building. The process is the same as building a dll (using extern "C" and all that) but instead of using [DllImport("xyz")] you do [DllImport("__Internal")].

I tried using miniaudio as an alternative (and actually putting directly the native code in the project) , it does output sound but worse than any current solutions thar are in the project and I can't figure out how to use their resampler properly.

[Skurdt]

Not using a job and doing a small update to the AudioProcessor fixed the audio quite a bit for me. Probably not perfect and not thoroughly tester but at least there's sound on the quest 2.

[Skurdt]

Still not working on blastem (probably others, but the SDL3 path is fine on this core)