Android SDK install CI "osx-install-android-sdk-automated" fails regularly, further install issues

[the-bay-kay]

Issues with Android SDK Install

Personal Build Issues

I've been working on building the android .apk. I've been able to perform the following steps successfully:

• Running bash setup/prereq_android_sdk_install.sh installs successfully. When running this, I receive the following error:
  • Warning: Failed to find package 'patcher;v4'
  • Running $ANDROID_SDK_ROOT/cmdline-tools/latest/bin/sdkmanager  --list and comparing it to setup/android_sdk_packages, it seems that the only package missing is patcher
  • I haven't been able to find much information on this package: ⁠this thread didn't shed much light on the subject, and neither did ⁠this thread, which suggests fixing the issue in the Studio GUI.
• Despite the warning described above, I am able to run bash setup/setup_android_native.sh and source setup/activate_native.sh without issue.

• Running npm run build-prod-android or npx cordova build android result in the following error:

  FAILURE: Build failed with an exception.
  
  * Where:
  Script '/Users/{user}/Documents/OpenPATH_Repos/e-mission-phone/platforms/android/CordovaLib/cordova.gradle' line: 69
  
  * What went wrong:
  A problem occurred evaluating script.
  > No installed build tools found. Please install the Android build tools version 33.0.2.
  
  * Try:
  > Run with --stacktrace option to get the stack trace.
  > Run with --info or --debug option to get more log output.
  > Run with --scan to get full insights.
  
  * Get more help at https://help.gradle.org/
  
  Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
  
  You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
  
  See https://docs.gradle.org/7.6/userguide/command_line_interface.html#sec:command_line_warnings
  
  BUILD FAILED in 363ms
  Command failed with exit code 1: /Users/{user}/Documents/OpenPATH_Repos/e-mission-phone/platforms/android/gradlew -b /Users/{user}/Documents/OpenPATH_Repos/e-mission-phone/platforms/android/build.gradle cdvBuildDebug
  

CI Failure

• When going to the osx-install-android-sdk-automated CI to better understand this issue, I discovered this GitHub Action has been failing for the past four months (⁠link).
  • I've been reading through ⁠this thread in the docs to better understand how we use the automatically install android CI.
• The CI regularly fails at the "Verify Directory Structure" stage. I believe the reason for this is because we're checking for packages that aren't installed in the prereq script.
  • I believe this is because we only curl for the cmdline-tools ( ⁠link ). As such, the $ANDROID_SDK_ROOT only has one directory beneath it: the cmdline-tools/. The CI has succeeded in the past, though I do not know what was changed between the success and failure -- and, because those actions were run months ago, we no longer have the logs.

Observations

• I believe the missing patcher package is not related to the CI failure. The failure seems to be a lack of utilities being installed, not a missing package.
• I'm also not entirely sure the missing patcher is what's causing my build issue. This line in the error:

  > No installed build tools found. Please install the Android build tools version 33.0.2.

  suggests that the issue is caused by a conflict with either (i) the build tool version, or (ii) an issue with the build tool's path.

• I attempted to solve this by adding the following to my .bash_profile, and sourcing the profile accordingly:

  export PATH=$PATH:$ANDROID_HOME/cmdline-tools/latest/bin

  unfortunately, adding the cmdline-tools to my path doesn't seem to have fixed the gradle issue.

• Looking at cordova.gradle, it appears that the failure occurs within the function getAndroidSdkDir(). This function (if I understand correctly) checks for $ANDROID_HOME (or $ANDROID_SDK_ROOT), or an equivalent SDK home, and then checks to confirm that the build_tools utilities are installed. This is the connection between the CI Failure and my build issues

Conclusion, Next Steps

It appears to me that, when running bash setup/prereq_android_sdk_install.sh, we are failing to install the build tools. This does not affect our actual build CI (link), because the android-automated-sdk-install CI creates a separate temporary directory that it installs into, which does not overwrite the preexisting, successful install. This would also explain why:

• Those building with a full install of Android Studio on MacOS have not had this issue
• Those who have not run the prereq script before the CI's "4 month failure point" have not run into this issue.

With all of this information in mind, it appears that bash setup/prereq_android_sdk_install.sh is failing to install incorrectly several components that are vital to the build process. As mentioned above, I believe this is because we only curl for the cmdline-tools ( ⁠link ). It appears that sdkmanager is installing packages, including the build tools, in a location other than the expected $ANDROID_HOME. Once I have a fix written, I'll run some tests and update this thread with further information.

EDIT: Corrected the final thoughts on the error to reflect the actual cause, as mentioned below.

[the-bay-kay]

Build Tool Notes

It appears that we do install andoird-build-tools, but the install is not where we'd expected it. Running $ANDROID_SDK_ROOT/cmdline-tools/latest/bin/sdkmanager --list | grep build returns build tools 19.1 through 35-rc1. I'm unsure where these files are installed, but they are certainly not being installed in $ANDROID_HOME/build_tools/ , as the documentation would suggest (It's not in ~/.android/, either). I'll update this post when I figure out where these packages are going.

[the-bay-kay]

I haven't been able to find where the packages are being kept. If nothing else, it seems that we may be able to manually set the path with sdkmanager (link).

When inspecting my environment variables (env | grep sdk), the following ENV variables appear:

ANDROID_SDK_ROOT=/Users/{user}/Library/Android/sdk
SDKMAN_PLATFORM=darwinarm64
SDKMAN_CANDIDATES_API=https://api.sdkman.io/2
SDKMAN_DIR=/Users/{user}/.sdkman
SDKMAN_CANDIDATES_DIR=/Users/{user}/.sdkman/candidates

Checking for any variation of build-tools (e.g., find $SDKMAN_DIR -name 'build-tools) reveals nothing...

[shankari]

we should fix the prereq SDK CI before trying to dig deeper into what is wrong (go from working to working). There was a prior error in the SDK CI that was resolved by removing an obsolete package (as we can see from the last commit to the prereq_android_sdk_install.sh file (https://github.com/e-mission/e-mission-phone/commit/5ea9f5883cea177e672724bc8cfff8550c3dec3e) https://github.com/e-mission/e-mission-docs/issues/958#issuecomment-1712314266

From the comment above, the line just below HAXM is patcher. I would try removing patcher (unless it is required), getting the CI to work and then running the fixed scripts on your laptop.

Again, the goal of the CI is to maintain a working version of the scripts. If the CI is broken, we should fix the CI Then, we should make our laptops match the CI

[the-bay-kay]

Changed merged in 1136, closing Issue!