RustAudio / cpal

Cross-platform audio I/O library in pure Rust
Apache License 2.0
2.66k stars 352 forks source link

Can you get the system latency? #773

Open tGrothmannFluffy opened 1 year ago

tGrothmannFluffy commented 1 year ago

Hi there,

I am trying to get the system latency. Basically, I want to know (roughly) when the samples I wrote to the output are ending up at the speaker. I know this is probably highly os and device dependent.

I found this oboe (android) function to get this value. But I was wondering if/how I could access that.

I am working on Android and iOS.

morajabi commented 1 year ago

I'm interested in this for macOS. Use case is for using acoustic echo cancellation that works better with exact latency metrics.

j-n-f commented 4 months ago

Also interested in this.

From what I can tell the StreamInstant values in callbacks are just giving block_size / sample_rate. The value doesn't match what my driver reports (it's close, but less than the actual values).

From a quick look at ASIO and ALSA they both return this information in frame/sample counts. It would be useful to know what this looks like for the other backends.

It should be possible to use that data to provide more accurate StreamInstant values (e.g. for callback and playback values). Since ASIO/ALSA provide the value in frames/samples, it might be nice to include that at the level of stream config (without having to derive it from a bunch of floating point values in callbacks).

j-n-f commented 4 months ago

Here's what I was able to scrounge up.

My previous comment on putting this latency information at the stream config level may not always be possible. If Oboe can even provide this value, it sounds like it might be dynamically changing block size and even latency while a stream is still running.

If some backend can't provide more accurate latencies then the existing heuristic was the best you could do anyways.

This should be enough to start experimenting.

backend api format comment
ALSA snd_pcm_delay frames/samples
JACK ??? ??? not sure, guessing you just use ALSA to fetch this information
WASAPI IAudioClient::GetStreamLatency 100ns increments
ASIO ASIOGetLatencies (ASIO SDK 2.3, page 14) samples/frames must call after ASIOCreateBuffers

Driver may notify host (cpal) of kAsioLatenciesChanged (ASIO SDK 2.3, page 9)
CoreAudio AudioObjectGetPropertyData (for kAudioDevicePropertyLatency) ??? There's also some discussion on a mailing list on how we may need to read several properties
Oboe ??? ??? Oboe is complicated, and it seems like it might dynamically adjust block size while running. Might just have to use existing heuristic for this one. (further reading)
Emscripten ??? ??? Not actually sure how this backend works, can't find documentation