Replace the current audio driver with Oboe on Android

[mattkwang]

Describe the project you are working on

Godot Audio on Android

Describe the problem or limitation you are having in your project

Godot's audio setup on Android currently uses OpenSL ES, an older audio API supported on Android API 9 and above. This API has high device coverage, but is bulkier and less suitable for low-latency applications compared to the native AAudio API introduced in Android API 26.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Oboe is Android's recommended modern audio API, acting as a C++ wrapper library that chooses between OpenSL ES and AAudio depending on device support and best performance. Oboe is compatible with API 16 onwards (covers 99% of Android devices). We expect this to modernize Godot's audio setup on Android, with lower audio latency and minimal loss of device coverage.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

We will create a class AudioDriverOboe housed under godot/platform/android, which will take the place of AudioDriveOpenSL in OS_Android. Much of its functionality will mirror AudioDriverOpenSL, informed by Oboe's migration guide.

Steps:

1. Add Oboe to godot/thirdparty and link with SCons build system.
2. Implement AudioDriverOboe with the set of methods inherited from AudioDriver, and modify os_android.h to use it. More implementation details are available if requested.
3. Develop test apps for latency measurement and potentially device coverage.

AudioDriverOboe Member variables:

Mutex *mutex
static AudioDriverOboe *s_ad (singleton)
AudioStream player
AudioStream recorder
bool active
bool pause

AudioDriverOboe Implemented API:

const char *get_name()
Error init()
void start()
int get_mix_rate()
SpeakerMode get_speaker_mode()
void lock()
void unlock()
void finish()
void set_pause(bool p_pause)
Error capture_start()
Error capture_stop()
Constructor AudioDriverOboe()

For latency testing, the simplest version is the Larsen test with two AudioStreamPlayer nodes, one that records audio input and one that plays the input continuously. By measuring the time between sounds, we capture the sum of input and output latency, with some device/engine overhead. This should serve as an adequate measure for approximate latency improvement.

For testing device coverage, we should have a test app that covers audio features on Godot. This may be using the mic_record demo or some variation upon it which will use both input and output. If run on some device farm or set of devices, we may use app crashes or bad sound behavior to verify minimal loss in coverage.

If this enhancement will not be used often, can it be worked around with a few lines of script?

N/A. This is expected to be used for all Godot games exported for Android.

Is there a reason why this should be core and not an add-on in the asset library?

This proposal changes the audio (a core element of the engine) intending to impact all games for the Android platform.

Android Games DevTech Team @ Google

[BastiaanOlij]

Sounds like a good enhancement to me. Oboe is under an Apache 2.0 license? So shouldn't be anything in our way to include it.

[dvdfu]

Just seconding to say I'd love to see this coming to Godot 4.x! I'm interested in developing music-playing apps with Godot, but haven't been able to achieve Android audio latency shorter than 240ms, which is a dealbreaker for these kinds of applications.

[Calinou]

@dvdfu Can you test https://github.com/godotengine/godot/pull/86776 on your project to see if it improves latency?

[dvdfu]

@dvdfu Can you test godotengine/godot#86776 on your project to see if it improves latency?

Sorry, I hadn't read this reply until now, and it seems like the artifacts are no longer available for download. Happy to test again if they are!

[Calinou]

Sorry, I hadn't read this reply until now, and it seems like the artifacts are no longer available for download. Happy to test again if they are!

I built a release Android export template APK from that PR: android_release.apk.zip

You should be able to specify it in the custom export template section of your Android export presets.

[dvdfu]

If I download that file and reference it in Android Export > Options > Custom Template > Release (or Debug), the project will build, but the application won't load (appears as a black screen). Removing it allows building like normal.

[kus04e4ek]

If I download that file and reference it in Android Export > Options > Custom Template > Release (or Debug), the project will build, but the application won't load (appears as a black screen). Removing it allows building like normal.

What Godot version are you using? The PR is close to the 4.3 dev 4 version, quickly checked that version and it seems compatible with Calinou's build

[jruiz94]

Hello! Just stumbled across this issue when exporting my game for Android. Does anyone know if this will get shipped to 4.3. Just curious, sending all my support for the huge work of the people involved in this development ❤️

[Calinou]

Does anyone know if this will get shipped to 4.3.

4.3 is in release freeze, so new features can only be merged in 4.4 at the earliest.

[HiroDane]

I built a release Android export template APK from that PR: android_release.apk.zip

Does this work on any dev version of godot currently?

I am having an issue where mic access is getting cut off after a few seconds on android even when app is in foreground. Got reffered to this proposal from a reddit post. Was hoping this could fix it.

[Alex2782]

I built a release Android export template APK from that PR: android_release.apk.zip Does this work on any dev version of godot currently?

should also work with v4.3.stable.official

[kus04e4ek]

I am having an issue where mic access is getting cut off after a few seconds on android even when app is in foreground. Got reffered to this proposal from a reddit post. Was hoping this could fix it.

See godotengine/godot#86428 and related PRs. The PR that closes this proposal doesn't solve this issue

[hayderkharrufa]

I'm developing an instrument app and dealing with noticeable audio latency on Android. Really looking forward to seeing this merged. I tried testing the provided template but got version errors since my project is built with 4.3 stable (tried with 4.3-dev4 but saw version mismatch errors in logcat).