orottier / web-audio-api-rs

A Rust implementation of the Web Audio API, for use in non-browser contexts
https://docs.rs/web-audio-api/
MIT License
300 stars 16 forks source link

Audio is delayed on Android #515

Closed kuviman closed 5 months ago

kuviman commented 5 months ago

For some reason, audio is delayed by about 1 second when I run on Android.

I checked rodio and it does not have that issue.

Since both libraries use cpal Im assuming the issue is in this repository

I have made a repository which can reproduce the issue here

It plays the audio every time the touch ends

orottier commented 5 months ago

Thanks for the report, this should not happen. I don't have a setup to run for Android. Could you help me provide with some timings? Print the timestamp before and after AudioContext construction, audio buffer load, instantiate the buffer source, and start? I think it has to do with pre-loading the HRTF engine..

kuviman commented 5 months ago

Ok, so I enabled more logs and I saw that the cpal output stream was created with a huge buffer size (almost a second)

Looks like the difference with rodio is that you clamp the buffer size to the supported range instead of using Default

If I log the default device config here's what I get:

web_audio_api::io::cpal: default device config = SupportedStreamConfig { channels: 2, sample_rate: SampleRate(44100), buffer_size: Range { min: 42528, max: 2147483647 }, sample_format: F32 }

As you can see, the min buffer size is huge and thats the delay that im getting

Also, if I try running again it seems like this min buffer size is different every time (ranges 30000-45000).

If I remove the clamping code from this crate and just use Default instead of Fixed buffer size for the output stream I seem to no longer get the delay.

So, maybe the issue is inside cpal actually (reporting wrong supported buffer size range?) and I should report there?

kuviman commented 5 months ago

Looking into cpal source, they just get the min buffer size from java AudioTrack.getMinBufferSize

Testing using Java I get same huge min buffer sizes so im not sure what can be done here, except for not clamping (either using default or fixed buffer size but without clamping)

orottier commented 5 months ago

Thanks for looking into that. Unless you want to file the issue all the way up to the Android source, we could consider a platform specific hack. So if cfg(target_os) = android use Default cpal buffer size for AudioContextLatencyCategory::Interactive and AudioContextLatencyCategory::Balanced ?

kuviman commented 5 months ago

I'm personally happy with a platform specific hack here, made a PR.

My thoughts are that Java API is maybe not the correct one to use though, since we are not using Java to render audio, but Oboe which is a C++ library, but I have no idea where to take it, I'd rather not dig any deeper here :)

orottier commented 5 months ago

Thanks for the PR. @b-ma do you have any comments? Otherwise I intend to merge.

Wondering if we should maybe perform a similar 'fix' (with a log message) to override the Interactive buffer size in Alsa to solve #306

b-ma commented 5 months ago

Hey,

Sure I agree with that fix (and thanks @kuviman)! In any case it can be easily removed once/if fixed upstream

For #306 , I don't know... The README and all our examples mention the problem, and it's pretty easy to fix on the application side... So while it's not a really really strong opinion neither, I wouldn't go for doing magical things in this case (worth case would be some kind of false positive where we degrade latency while it would work...).

"Alsa is broken, please use a modern audio backend such as pipewire." could be a nice warning actually :)

orottier commented 5 months ago

Released in v0.45.2

Thanks again @kuviman

b-ma commented 5 months ago

@kuviman

My thoughts are that Java API is maybe not the correct one to use though, since we are not using Java to render audio, but Oboe which is a C++ library, but I have no idea where to take it, I'd rather not dig any deeper here :)

Me neither :) but maybe would be nice to report your findings to cpal, maybe they wont fix it soon but at least they know the problem