search

kivy / buildozer

Generic Python packager for Android and iOS
https://buildozer.readthedocs.io
MIT License
1.71k stars 497 forks source link

Allow specifiying sdkmanager binary location in buildozer specs to avoid NoClassDefFoundError and other issues #1492

Open damascene opened 2 years ago

damascene commented 2 years ago

Versions

Description

While trying to build my app for f-droid with Java 11, buildozer seems trying to use an older version of sdkmanager under

/opt/android-sdk/tools/bin/sdkmanager

instead of:

/opt/android-sdk/cmdline-tools/tools/bin/sdkmanager

and I guess because it needs java 8, it fails with:

# Run '/opt/android-sdk/tools/bin/sdkmanager --sdk_root=/opt/android-sdk platform-tools'
# Cwd /opt/android-sdk
Exception in thread "main" java.lang.NoClassDefFoundError: javax/xml/bind/annotation/XmlSchema
    at com.android.repository.api.SchemaModule$SchemaModuleVersion.<init>(SchemaModule.java:156)
    at com.android.repository.api.SchemaModule.<init>(SchemaModule.java:75)
    at com.android.sdklib.repository.AndroidSdkHandler.<clinit>(AndroidSdkHandler.java:81)
    at com.android.sdklib.tool.sdkmanager.SdkManagerCli.main(SdkManagerCli.java:73)
    at com.android.sdklib.tool.sdkmanager.SdkManagerCli.main(SdkManagerCli.java:48)
Caused by: java.lang.ClassNotFoundException: javax.xml.bind.annotation.XmlSchema
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
    ... 5 more
# Command failed: /opt/android-sdk/tools/bin/sdkmanager --sdk_root=/opt/android-sdk platform-tools

I'm using Java 11 to solve an issue in building with Java 8.

Android sdkmanager location change

I noticed that some Android paths have changed :

the tools directory hierarchy has been changed

https://stackoverflow.com/a/61176718/7346913

I've noticed great amount of bugs related to NoClassDefFoundError and maybe there could be an Exception added to catch those and reference user to the correct path.

buildozer.spec

Command:

buildozer android release

Spec file:

https://gitlab.com/uak/kivy-hello-world/-/blob/master/buildozer.spec

Logs

https://gitlab.com/uak/fdroiddata/-/jobs/2886953981#L6738

My fdroid build recipe:

Categories:
  - Development
License: GPL-3.0-or-later
SourceCode: https://gitlab.com/uak/kivy-hello-world

Summary: Hello World app built with Kivy for Development
Description: |-
    This app is a very simple Kivy app to help developers explore development\nwith Kivy to learn how to build and submit an app to f-droid repository.
    It has zero dependency other than Python and Kivy.

RepoType: git
Repo: https://gitlab.com/uak/kivy-hello-world

Builds:
  - versionName: '0.1'
    versionCode: 10211
    commit: 6ebed5ceb4813cfd830311e5e7485578026c88c8
    sudo:
      - apt-get update || apt-get update
      - apt-get install -y build-essential libffi-dev libltdl-dev openjdk-11-jdk-headless
        wget
      - update-alternatives --auto java
    output: bin/kivy_hello_world-*-arm64-v8a_armeabi-v7a-release-unsigned.apk
    srclibs:
      - cpython@v3.8.13
      - buildozer@1.4.0
    prebuild:
      - export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
      - wget -O /tmp/cmdline-tools.zip -t 5 "https://dl.google.com/android/repository/commandlinetools-linux-7583922_latest.zip"
      - echo "124f2d5115eee365df6cf3228ffbca6fc3911d16f8025bebd5b1c6e2fcfa7faf /tmp/cmdline-tools.zip"  |
        sha256sum -c -
      - unzip -q -o /tmp/cmdline-tools.zip -d /tmp/cmdline-tools
      - rm /tmp/cmdline-tools.zip
      - mkdir -p ${ANDROID_SDK_ROOT}/cmdline-tools/tools
      - mv /tmp/cmdline-tools/cmdline-tools/* ${ANDROID_SDK_ROOT}/cmdline-tools/tools
      - export PATH=${ANDROID_SDK_ROOT}/cmdline-tools/tools/bin:${PATH}
      - yes | sdkmanager --licenses
      - export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
      - sdkmanager 'platforms;android-33' 'build-tools;33.0.0'
      - sed -r "s:#?android.sdk_path =.*:android.sdk_path = $$SDK$$:" -i buildozer.spec
      - sed -r "s:#?android.ndk_path =.*:android.ndk_path = $$NDK$$:" -i buildozer.spec
      - sed -r "s:#?android.accept_sdk_license =.*:android.accept_sdk_license = False:"
        -i buildozer.spec
      - sed -i -e 's/# android.release_artifact = aab/android.release_artifact = apk/'
        buildozer.spec
    build:
      - pushd $$cpython$$
      - mkdir -p /tmp/cpython
      - ./configure --prefix=/tmp/cpython --enable-optimizations
      - make -j`nproc`
      - make install
      - popd
      - export PATH=/tmp/cpython/bin:$PATH
      - pip3 install --user --upgrade $$buildozer$$ Cython
      - export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
      - PATH="$HOME/.local/bin:$PATH" buildozer android release
    ndk: r23b

AutoUpdateMode: None
UpdateCheckMode: None
CurrentVersion: '0.1'
CurrentVersionCode: 10211
damascene commented 2 years ago

I've solved the issue with location of binary sdkmanager by hacking the hard coded path:

        sdkmanager_path = join(
            self.android_sdk_dir, 'tools', 'bin', 'sdkmanager')

https://github.com/damascene/buildozer/blob/master/buildozer/targets/android.py#L247

Using this sed command in the build recipe:

      - sed -r "s/'tools', 'bin'/'cmdline-tools\'\,\'latest', 'bin'/" -i ~/.local/lib/python*/site-packages/buildozer/targets/android.py

to change the lines to:

        sdkmanager_path = join(
            self.android_sdk_dir, 'cmdline-tools', 'latest', 'bin', 'sdkmanager')

Moving files to latest fixes an avdmanager issue too as it has specific places to expect binaries: https://github.com/kivy/python-for-android/blob/28151d1864df9564ee796910f38a995b54d73631/pythonforandroid/build.py#L28-L43

Now sdkmanager location at:

/opt/android-sdk/cmdline-tools/latest/bin/sdkmanager

And Python for Android will recognize binaries there and use it.

My fdroid build recipe:

Categories:
  - Development
License: GPL-3.0-or-later
SourceCode: https://gitlab.com/uak/kivy-hello-world

Summary: Hello World app built with Kivy for Development
Description: |-
    This app is a very simple Kivy app to help developers explore development\nwith Kivy to learn how to build and submit an app to f-droid repository.
    It has zero dependency other than Python and Kivy, enjoy!

RepoType: git
Repo: https://gitlab.com/uak/kivy-hello-world

Builds:
  - versionName: '0.1'
    versionCode: 10211
    commit: 6ebed5ceb4813cfd830311e5e7485578026c88c8
    sudo:
      - apt-get update || apt-get update
      - apt-get install -y build-essential libffi-dev libltdl-dev openjdk-11-jdk-headless
        wget
      - update-alternatives --auto java
    output: bin/kivy_hello_world-*-arm64-v8a_armeabi-v7a-release-unsigned.apk
    srclibs:
      - cpython@v3.8.13
      - buildozer@1.4.0
    prebuild:
      - export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
      - wget --no-verbose -O /tmp/cmdline-tools.zip -t 5 "https://dl.google.com/android/repository/commandlinetools-linux-7583922_latest.zip"
      - echo "124f2d5115eee365df6cf3228ffbca6fc3911d16f8025bebd5b1c6e2fcfa7faf /tmp/cmdline-tools.zip"  |
        sha256sum -c -
- echo "test"
      - unzip -q -o /tmp/cmdline-tools.zip -d /tmp/cmdline-tools  2>&1 >/dev/null
      - rm /tmp/cmdline-tools.zip
      - mkdir -p ${ANDROID_SDK_ROOT}/cmdline-tools/latest
      - mv /tmp/cmdline-tools/cmdline-tools/* ${ANDROID_SDK_ROOT}/cmdline-tools/latest
      - export PATH=${ANDROID_SDK_ROOT}/cmdline-tools/latest/bin:${PATH}
      - yes | sdkmanager --licenses
      - sdkmanager 'platforms;android-33' 'build-tools;33.0.0' | grep -v = || true
      - sed -r "s:#?android.sdk_path =.*:android.sdk_path = $$SDK$$:" -i buildozer.spec
      - sed -r "s:#?android.ndk_path =.*:android.ndk_path = $$NDK$$:" -i buildozer.spec
      - sed -r "s:#?android.api =.*:android.api = 33:" -i buildozer.spec
      - sed -r "s:log_level =.*:log_level = 1:" -i buildozer.spec
      - sed -r "s:#?android.accept_sdk_license =.*:android.accept_sdk_license = False:"
        -i buildozer.spec
      - sed -i -e 's/# android.release_artifact = aab/android.release_artifact = apk/'
        buildozer.spec
      - sed -r "s:#?android.skip_update =.*:android.skip_update = True:" -i buildozer.spec
    build:
      - pushd $$cpython$$
      - mkdir -p /tmp/cpython
      - ./configure --prefix=/tmp/cpython --enable-optimizations | grep -v checking
        || true
      - make -j`nproc` 2>&1 >/dev/null
      - make install 2>&1 >/dev/null
      - popd
      - export PATH=/tmp/cpython/bin:$PATH
      - pip3 config --user set global.progress_bar off
      - pip3 install --user --upgrade $$buildozer$$ Cython 2>&1 >/dev/null
      - cat ~/.local/lib/python*/site-packages/buildozer/targets/android.py
      - sed -r "s/'tools', 'bin'/'cmdline-tools\'\,\'latest', 'bin'/" -i ~/.local/lib/python*/site-packages/buildozer/targets/android.py
      - export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
      - PATH="$HOME/.local/bin:$PATH" buildozer android release
    ndk: r23b

AutoUpdateMode: None
UpdateCheckMode: None
CurrentVersion: '0.1'
CurrentVersionCode: 10211

https://gitlab.com/uak/fdroiddata/-/commit/c93133b2d9819ac4268b829df3b3b117916c9173