sjoerdvankreel / xt-audio

Platform independent low-latency audio for C, C++, Java and .NET.
https://sjoerdvankreel.github.io/xt-audio/
Other
64 stars 12 forks source link

Realtime priority on Windows #20

Closed LuzianU closed 2 years ago

LuzianU commented 2 years ago

When the java application on Windows is out of focus, the audio starts crackling slightly. This can be pushed to the extreme when running a light CPU benchmark, rendering the audio completely unusable due to constant audio stutters.

I'm not entirely sure if this is the way to change thread priorities on Windows, however this completely removes all audio stutters even on very high CPU usage.

sjoerdvankreel commented 2 years ago

Hey,

the Raise/RevertThreadPriority functions are meant to be used in combination with the Linux PulseAudio and ALSA backends only. On Windows, things work a bit different:

So, I think the net effects of your proposed change are: 1) For WASAPI, get in the way of MMCSS 2) For DirectSound, raise the entire process prio to realtime 3) For both of them, hog the rest of the system

As it stands this is a less-than-ideal solution. What backend are you actually using? I'm guessing it's DirectSound since that's the only one that relates the audio thread to an application window, see https://docs.microsoft.com/en-us/previous-versions/windows/desktop/ee418049(v=vs.85). If so, please keep in mind that DirectSound is not in any way suitable for high-performance/low latency audio (it has a relatively large overhead compared to asio and wasapi, especially wasapi exclusive mode). So for DSound you should probably just raise the buffer size (might even be as high as a couple 100 ms). If it's WASAPI, then it seems MMCSS isn't doing it's job properly, which is to keep your audio stutter-free while still not interfering with the rest of the system too much. If that's the case we can add an option to the wasapi backend to use EITHER realtime process priority class or MMCSS, but not both. However then we'd have to investigate a little further on why MMCSS isn't doing it's job before bringing out the big guns.

Edit: if you are in fact using WASAPI in shared mode, could you change WasapiStream::StartMasterBuffer() to always use "Pro Audio" and see if this helps? I would much prefer letting the user pick the MMCSS task name instead of messing with the process prio directly, seeing as this affects all threads in the process.

Edit2: when you say cpu benchmark, do you mean pushing the limits of the audio thread or something external to the process hosting xt-audio?

LuzianU commented 2 years ago

Hey, thank you for this detailed answer! Maybe I should have opened an issue about that audio crackling instead, in hindsight.

You are totally right, when using ASIO it in fact still stutters a bit while under heavy load compared to WASAPI (which I was using all along, shared and exclusive). I haven't tested/used DirectSound at all because I'm aiming for low latency.

I tested both shared and exclusive, so I have not tried your first edit yet, which from what I can tell should be used in exclusive mode anyway.

Regarding your second edit, poorly worded on my end, I was running a CPU benchmark in my browser.

I tested it again on stock xt-audio and both exclusive and shared WASAPI, but also ASIO (asio4all) slightly crackle once in a while when the java application is in background. Running a CPU benchmark in my browser significantly worsens this, but it also happens without any major CPU load while in background.

Since this is the case for both WASAPI and ASIO I think the only thing that can fix this is just running the java application on higher priority - even above normal solves that. So I guess I will just have to set the priority when starting it.

sjoerdvankreel commented 2 years ago

Yeah, setting process prio at app startup is your best bet here. Doesn't require any changes from xt-audio also, so that's lucky on my end :) I'm still curious though, both asio and wasapi (shared as well as exclusive) should be able to provide you with glitch-free playback even without realtime priority. Are you doing anything special? Low buffer sizes, high load on the audio thread (think real-time synthesis, audio effects, etc)? Also, are you absolutely positive this is related to window focus? Doesn't happen with a console app? How about a c++ console app? I would really like for xt-audio to behave well in the face of system load, at least for supposedly low-latency backends.

Edit: are you positive your code is suitable for real-time processing? No I/O (file, console, pipe, ...), no blocking (lock/monitor/synchronized/mutex/criticalsection, ...), no allocating memory (malloc/new/some 3th party lib you dont expect to allocate), no other syscalls (createthread, fiber, whatever the JDK has to offer these days)? See http://www.rossbencina.com/code/real-time-audio-programming-101-time-waits-for-nothing. If you don't follow these rules your audio thread WILL be rescheduled by the OS scheduler eventually, I think process prio won't help with that in the long run. Also, this might be a good time to review my java bindings to the core xt-audio library :) I sincerely hope them to be free of all this stuff but it isn't always obvious.

LuzianU commented 2 years ago

Oops, didn't see your response before closing. Yes, primarily low buffer sizes, but it also happened with a "stream latency" of >20ms, so I assumed it's not because of xtBufferSize.min I'm not really familiar with c++ that much, but I will do some more tests with a more simple java program, even though I don't think there's too much going on in mine right now.