search

google / oboe

Oboe is a C++ library that makes it easy to build high-performance audio apps on Android.
Apache License 2.0
3.72k stars 573 forks source link

docs: Update guide to note additional latency with default sample rate conversion quality #2021

Closed thomas-coldwell closed 6 months ago

thomas-coldwell commented 6 months ago

Hey Oboe folks 👋

My company has been using Oboe to build a realtime music learning app where simultaneously a video of the band is played whilst the microphone listens to the user's playing of their instrument. We use Oboe for both the audio input (recording of the user playing) and output (playback of the band, part of a low-level custom player using FFmpeg and Oboe for the audio backend). As you can imagine having precise synchronisation between the playback and recording in our setup is paramount as we analyse the user's rhythm and pitch after they have performed.

After doing a deep dive into some poor scores I noticed that there was a large amount of additional and device dependant latency in our recordings. Initially, I thought this was an issue with our playback (asset encoding or internal timing of the playback) but after creating an isolated testbench (repro can be found here https://github.com/thomas-coldwell/oboe/tree/%40thomas/loopback-test and tested by running the HelloOboe project which I tweaked and pulling off the raw PCM data file) I finally figured out that the root cause of this latency was the sample rate conversion quality setting

With this set to None I got +250ms on a OnePlus 6 and +180ms (on a Samsung A40) of additional latency in the recording. However, as soon as I switched this to Fastest I then got around ~20ms of latency on both which corresponds nicely with the actual input and output stream latency value I get when I call calculateLatencyMillis().

^^ Sorry for the full story but thought it would be good to give a repro and context around this. As its such a big factor in ensuring minimal round trip latency I think adding this section to the docs would help point other devs in the right direct here 🙌

google-cla[bot] commented 6 months ago

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

philburk commented 6 months ago

@thomas-coldwell - Thanks for pointing out this issue. We had made None the default because we were not sure if the Oboe sample rate converter would work reliably. At this point we are pretty confident in it. So we may change the default in a future version.

Until then your new documentation will help others achieve lower latency.

If you are interested low-latency, then this doc may be of interest to you: https://developer.android.com/games/sdk/oboe/low-latency-audio#oboe