Gradle/AGP brings in inconsistent builds which prevents androidaudioplugin-manager.aar from MavenLocal

[atsushieno]

from https://github.com/atsushieno/aap-core/issues/172#issuecomment-1716034775:

It seems the latest implementation seems to work locally, if I replace the library references in app/build.gradle to the submodules (using the methodology documented at README.md, with the diff gist I have updated a few days ago). The actual behavioral difference somehow appears at aap::AudioDataSourceNode::setAudioSource() for WTF reason. AudioFileReader.readFrames() from OggAudioFileFormat fails to consume SeekableByteBuffer. It was discovered through [this local patch](https://gist.github.com/atsushieno/47bbbd40bdddb2e322ef77109966ad71) against androidaudioplugin-manager sources (also logging changes to choc). I was forced to do this ugly logging debugging because if I want to put breakpoints on any code in androidaudioplugin-manager.aar it must be part of the project, and that means, we have a WORKING build without this problem! HOW COME such a behavioral difference could happen, depending on whether androidaudioplugin.aar is resolved as a project() or Maven package!? I have no idea. Gradle has been bringing in incompatibility and unstability changes and AGP has to follow that, which brings in even more nightmare. And Android Studio and Jetpack Compose requires fairly recent versions of Gradle i.e. 8.0+, so we have no way to go back to good (okay, "better" to be precise) old versions. At this state. the only fix I can think of is to make structural changes to all those aap-lv2-XXX plugins to NOT resolve aars from MavenLocal but rather from the project, adding those aap-lv2 and aap-core modules into it. It is ugly, but better than lame not working version. We really want to dump Gradle, but unfortunately we can't.

[atsushieno]

I have created a reverting branch that uses Gradle 7.6 and AGP 7.4.2, and the problem still persists. There are workaround builds that resolves aap-core and aap-lv2 modules from git submodules, and they successfully builds. But aap-lv2-mda on GitHub Actions (e.g. this), unlike local builds on Android Studio, resulted in missing lv2 assets(!). The build system still blocks...

[atsushieno]

The same workaround is added in main branch in aap-lv2 and aap-lv2-mda i.e. their dependencies { ... } is full of implementation (project (...)) instead of module name artifacts. We will stick to that solution until we could get working builds...

We keep this issue open until then.

[atsushieno]

The MavenLocal problem also seems to affect the new realtime preset setter implementation. It works well if androidaudioplugin-manager is resolved via project(...). Looks like it is caused by the same dependency resolution failure.

[atsushieno]

When I tried to build the release version 0.2.8 (with aap-core 0.7.8, both without -pre), the revised version of aap-lv2 that resolves aap-core aars somehow broke aap-ayumi build because of missing libandroidaudioplugin-lv2.so(!)

Gradle and/or AGP with NDK support is such a wild field.

Fortunately, aap-lv2-mda effect plugins generate valid audio outputs from those MavenLocal .aars this time, I decided to go with MavenLocal for this release. The same issue would come back at any time so I leave this issue open.

[atsushieno]

Actually the observation ^ was wrong, the audio resource resolution failure issue that occurs only with MavenLocal was never gone. We have to revert the change and find some viable solution (but how?? I have no idea at all).

[atsushieno]

It also prevents aap-lv2 package releases because Maven publication task results in publishing androidaudioplugin.aar and androidaudioplugin-manager.aar to Nexus OSSRH (because they are dependencies). I have no idea how many more problems Gradle will bring in by trying hard to fix/support it so I leave it as is.

[atsushieno]

Now the same problem occurs in aap-juce apps. Sigh.

[atsushieno]

This also affects testImplementation. So if a project has testImplementation(libs.aap.testing) it will also cause the problem. I had been stuck at aap-ayumi for a while due to this issue.

[atsushieno]

After facing the same issue at ktmidi, I figured that it's not going to work because we cannot "publish" the dependency modules. Either we need to figure out how to exclude them or we need to fix this issue.

[atsushieno]

I have bumped Gradle to 8.5 and AGP to 8.2.2, and the problem still persists.

[atsushieno]

Same for Gradle 8.6-rc1 and AGP 8.4.0-alpha07.

[atsushieno]

Comparing androidaudioplugin-manager.aar from ~/.m2/repository/org/androidaudioplugin/androidaudioplugin-manager/0.8.0/, the native libraries are certainly different:

$  issue-174  ls -l working/lib/arm64-v8a
total 35600
-rw-r--r--  1 atsushi  staff    75584 Jan  1  1981 libaapbarebonepluginsample.so
-rw-r--r--  1 atsushi  staff  4381656 Jan  1  1981 libandroidaudioplugin-manager.so
-rw-r--r--  1 atsushi  staff  9321360 Jan  1  1981 libandroidaudioplugin.so
-rw-r--r--  1 atsushi  staff  1822720 Jan  1  1981 libc++_shared.so
-rw-r--r--  1 atsushi  staff  2620112 Jan  1  1981 liboboe.so
$  issue-174  ls -l problematic/lib/arm64-v8a 
total 16304
-rw-r--r--  1 atsushi  staff    75584 Jan  1  1981 libaapbarebonepluginsample.so
-rw-r--r--  1 atsushi  staff   442896 Jan  1  1981 libandroidaudioplugin-manager.so
-rw-r--r--  1 atsushi  staff  5716576 Jan  1  1981 libandroidaudioplugin.so
-rw-r--r--  1 atsushi  staff  1822720 Jan  1  1981 libc++_shared.so
-rw-r--r--  1 atsushi  staff   280128 Jan  1  1981 liboboe.so

Since they are already different at .aar level, dealing with native libs at app level would not make any difference. Yet, those options on androidaudioplugin-manager/build.gradle.kts did not matter:

• packaging.jniLibs.keepDebugSymbols.add("**/*.so")
• packaging.jniLibs.useLegacyPackaging = true

[atsushieno]

Here are what I found so far:

• It is a matter of Release build. Debug build does not matter.
• It seems caused by something in libandroidaudioplugin-manager.so being stripped.
• It does not seem to be related to keepDebugSymbols in build.gradle.kts
  • the input library libandroidaudioplugin-manager.so is already stripped (explained later)
  • and it should not be a matter of "debug symbols"
  • AGP resolves those native libraries without stripping when we use implementation(project(...)). This smells like a bug, but does not matter this time
  • Release builds should respect our cxx compiler flags (explaned later), nonetheless.
  • It is not a Prefab issue, or any Gradle task around it (e.g. inflexible to be unable to add export_libraries options - it is true, but does not matter this time)
  • It is not an issue StripDebugSymbolsTask itself - the task seems working as expected in general
• It is rather one or more CMake issues.
  • The problem is that no matter what arguments we pass to cmake section in build.gradle.kts, they cannot beat the optimization flagsCMAKE_BUILD_TYPE.
  • Neither CMAKE_CXX_FLAGS nor CMAKE_CXX_FLAGS_RELEASE works.
  • arguments("-DCMAKE_BUILD_TYPE=Debug") on release configuration makes it work again, but that means we are left not optimized on release builds, so it's better to avoid it

[atsushieno]

https://gitlab.kitware.com/cmake/cmake/-/issues/23378

[atsushieno]

arguments("-DCMAKE_CXX_FLAGS_RELWITHDEBINFO=-O2 -g -DNDEBUG") is not working, and arguments("-DCMAKE_CXX_FLAGS_RELWITHDEBINFO=-O2 -g") is working, somehow.

We can go without -DNDEBUG.