Android Audio Best Practices (Google I/O 17)

[rpattabi]

Recently at Google I/O 17, Android Audio team put forth some recommendations to improve audio performance: https://youtu.be/C0BPXZIvG-Q?t=10m13s

I am wondering if pd-for-android can take advantage of these recommendations and how. I have summarized the best practices here.

Obtain low latency audio paths

Check for audio.pro hardware flag, which means that the device is capable of less than 20ms latency

Recording

• use VOICE_RECOGNITION preset to by pass system effects
• use PCM16 format which avoids any format shifting

  Playback

• use Fast Mixer thread
• use native sample rate from AudioManager API by using PROPERTY_OUTPUT_SAMPLE_RATE
• use optimal chunk size from AudioManager API by using PROPERTY_OUTPUT_FRAMES_PER_BUFFER
• feed DAC with the help of provided callback which executes on high priority thread

Meet audio deadlines

Additional recommendations for the callback implementation to avoid glitches

• avoid logging. use systrace instead.
• avoid memory allocation
• don't wait on other threads
• don't do file i/o. Instead, use another thread to do read file and transfer the data using circular buffer or non-blocking queue
• don't sleep
• prevent cpu core migration by setting thread affinity, or using getExclusiveCores()
• prevent cpu frequency scaling by adding a stabilizing load
• use Sustained Performance Mode

[sinewave440hz]

Also, in Android 8.0+ there is AAudio and MMAP which provides lower latency possibilities. As I understand it, the Oboe library handles this and the legacy path (FastMixer) appropriately to the device used... See https://www.youtube.com/watch?v=Pjgfje52Yv0 for more inspiration...

[JamesNitsch]

I'm trying my hand at getting Oboe integrated, but I don't have a ton of expertise with C/C++ interop in Android. All published Oboe examples use Cmake, whereas currently pd-for-android uses the older method of ndk-build. I've gotten as far as including the Oboe library into the project, and with a few changes to the Application.mk ... APP_STL := c++_static APP_CPPFLAGS := -std=c++14 ... the new jni Oboe bindings seem to be finding all the right dependencies on the C++14 standard library, as well as the required Oboe headers. Adding the flags above, however, causes additional compilation problems, both in my new z_jni_oboe.cpp class, as well as a few other C classes in libpd (z_jni_shared, z_jni, etc).

One common error among them is:

error: member reference type 'JNIEnv' (aka '_JNIEnv') is not a pointer; did you mean to use '.'? jobjectArray jarray = (*env)->NewObjectArray(env, argc, objClass, NULL);

I'm not familiar enough with C / C++ to understand what might have caused this, though I have my hunches. Removing the Android.mk rule to build the new Oboe bindings fixes these issues.

My new addition to the Android.mk file is as follows:

#Build Oboe JNI binary

include $(CLEAR_VARS)

LOCAL_MODULE := pdnativeoboe
LOCAL_C_INCLUDES := $(PD_C_INCLUDES) $(LOCAL_PATH)/jni
LOCAL_SRC_FILES := jni/z_jni_oboe.cpp
include $(BUILD_SHARED_LIBRARY)

And the z_jni_oboe.cpp class mirrors the z_jni_opensl.c file with the exception of including a few Oboe objects.

Any guidance from someone who may be more familiar with C/C++, or Android's native interop would be helpful and appreciated. Would the task of migrating pd-for-android to Cmake be less painful than trying to include the Oboe library via ndk-build in the long run?

[rpattabi]

The following code should work:

#include <jni.h>

JNIEXPORT jstring JNICALL
Java_com_example_myapp_mymodule_MyClass_myNativeMethod(JNIEnv *env, jclass type) {
    return (*env)->NewStringUTF(env, ...

If not, could you share z_jni_oboe.cpp code?

[rpattabi]

I don't know where to share this information. So sharing it here.

App Bundle may break your patches

Please be aware when you chose to use app bundles to distribute your app that uses pd-for-android. Normally there is no problem. But if you have custom externals, pd patches may fail to load them when you use app bundles!

The reason is, native libraries are not installed but remain as part of the generated apk itself. This is a feature introduced in Marsmallow. Apk keeps native libs in uncompressed form and the installed apps maps them saving space. Generally this is not an issue, but in order for pd to load externals we shall provide search path. This is typically ApplicationInfo.nativeLibraryDir. But with bundle, the generated apk has uncompressed native libs. Thus nativeLibraryDir is an empty directory as the .so files are never copied to here.

We could reproduce this problem with an Android Pie device using app bundle consistently. However on Android Oreo, the same app bundle copies native libs and this wasn't an issue there. This means, though your patch works fine on some devices, it may fail to load on others!

Take a look at: https://github.com/google/bundletool/issues/39

Workaround

Prevent having uncompressed native libs feature.

1. Update gradle.properties with android.bundle.enableUncompressedNativeLibs=false
2. In the manifest under <Application> specify android:extractNativeLibs="true"