Buildozer fails to include project files when building for Android

Versions

Python: 3.10.2
OS: Rocky8
Buildozer: 1.5.0

Description

I have made a basic project skeleton myapp that replicates the general structure of my usual Python development flow, but adds Buildozer in for android targets. It looks like this.

myapp-proj
├── buildozer.spec
├── main.py
├── myapp
│   ├── cli
│   │   ├── cli.py
│   │   └── __init__.py
│   ├── gui
│   │   ├── gui.py
│   │   └── __init__.py
│   ├── __init__.py
│   ├── myapp.egg-info
│   │   ├── dependency_links.txt
│   │   ├── entry_points.txt
│   │   ├── PKG-INFO
│   │   ├── requires.txt
│   │   ├── SOURCES.txt
│   │   └── top_level.txt
│   └── utils
│       ├── __init__.py
│       └── logging.py
└── setup.py

It's pretty simple. I have a main.py that just calls a function in gui/gui.py (imported correctly up into gui/__init__.py. The main.py looks like this:

import os
import sys
import traceback

from myapp.gui import gui

def main():
    print(f"sys.path: {sys.path}")
    try:
        gui()
    except Exception as err:
        print(f"Critical exception. Error:\n{err}")
        traceback.print_exc()

if __name__=='__main__':
    main()

And gui.py:

import kivy

from kivy.app import App
from kivy.uix.label import Label

from myapp.utils.logging import log_generator
log = log_generator(__name__)

class AppGui(App):

    def build(self):
        return Label(text='myapp: hello world')

def gui():
    log.info(f"hello world")
    AppGui().run()

Logging is a boilerplate logger.

I am trying to compile this over into an .apk, and it does that successfully, but when run on the device or emulator (tried both), I get a Module not found: myapp error. Inspecting the .apk that gets generated, the assets/private.tar folder only contains main.py (and no subfolders).

buildozer.spec

Command:

buildozer android debug deploy run ; buildozer android logcat | grep myapp

Spec file:

source.dir = .
source.include_exts = py,png,jpg,kv,atlas
source.include_patterns = main.py, myapp/__init__.py, myapp/gui/__init__.py, myapp/gui/gui.py, myapp/utils/__init__.py, myapp/utils/logging.py
version = 0.1
requirements = python3,kivy
orientation = portrait

#
# Android specific
#
fullscreen = 0
android.permissions = android.permission.INTERNET, (name=android.permission.WRITE_EXTERNAL_STORAGE;maxSdkVersion=18), android.permission.READ_EXTERNAL_STORAGE
android.archs = arm64-v8a, armeabi-v7a
android.allow_backup = True
p4a.branch = develop
android.no-byte-compile-python = True

[buildozer]

log_level = 2
warn_on_root = 1

Logs

The build output does not contain any problems. It builds successfully, it just doesn't have anything except main.py in assets/private.tar. I will then get this output on run:

2024-03-06 13:51:17.390  5227-5299  python                  org.test.myapp                       I  Initialized python
2024-03-06 13:51:17.390  5227-5299  python                  org.test.myapp                       I  AND: Init threads
2024-03-06 13:51:17.400  5227-5299  python                  org.test.myapp                       I  testing python print redirection
2024-03-06 13:51:17.415  5227-5299  python                  org.test.myapp                       I  Android path ['.', '/data/user/0/org.test.myapp/files/app/_python_bundle/stdlib.zip', '/data/user/0/org.test.myapp/files/app/_python_bundle/modules', '/data/user/0/org.test.myapp/files/app/_python_bundle/site-packages']
2024-03-06 13:51:17.417  5227-5299  python                  org.test.myapp                       I  os.environ is environ({'PATH': '/product/bin:/apex/com.android.runtime/bin:/apex/com.android.art/bin:/system_ext/bin:/system/bin:/system/xbin:/odm/bin:/vendor/bin:/vendor/xbin', 'ANDROID_BOOTLOGO': '1', 'ANDROID_ROOT': '/system', 'ANDROID_ASSETS': '/system/app', 'ANDROID_DATA': '/data', 'ANDROID_STORAGE': '/storage', 'ANDROID_ART_ROOT': '/apex/com.android.art', 'ANDROID_I18N_ROOT': '/apex/com.android.i18n', 'ANDROID_TZDATA_ROOT': '/apex/com.android.tzdata', 'EXTERNAL_STORAGE': '/sdcard', 'ASEC_MOUNTPOINT': '/mnt/asec', 'DOWNLOAD_CACHE': '/data/cache', 'BOOTCLASSPATH': '/apex/com.android.art/javalib/core-oj.jar:/apex/com.android.art/javalib/core-libart.jar:/apex/com.android.art/javalib/okhttp.jar:/apex/com.android.art/javalib/bouncycastle.jar:/apex/com.android.art/javalib/apache-xml.jar:/system/framework/framework.jar:/system/framework/framework-graphics.jar:/system/framework/ext.jar:/system/framework/telephony-common.jar:/system/framework/voip-common.jar:/system/framework/ims-common.jar:/apex/com.android.i18n/javalib/core-icu4j.jar:/apex/com.android.adservices/javalib/framework-adservices.jar:/apex/com.android.adservices/javalib/framework-sdksandbox.jar:/apex/com.android.appsearch/javalib/framework-appsearch.jar:/apex/com.android.btservices/javalib/framework-bluetooth.jar:/apex/com.android.configinfrastructure/javalib/framework-configinfrastructure.jar:/apex/com.android.conscrypt/javalib/conscrypt.jar:/apex/com.android.devicelock/javalib/framework-devicelock.jar:/apex/com.android.healthfitness/javalib/framework-healthfitness.jar:/apex/com.android.ipsec/javalib/android.net.ipsec.ike.jar:/apex/com.android.media/javalib/updatable-media.jar:/apex/com.android.mediaprovider/javalib/framework-mediaprovider.jar:/apex/com.android.ondevicepersonalization/javalib/framework-ondevicepersonalization.jar:/apex/com.android.os.statsd/javalib/framework-statsd.jar:/apex/com.android.permission/javalib/framework-permission.jar:/apex/com.android.permission/javalib/framework-permission-s.jar:/apex/com.android.scheduling/javalib/framework-scheduling.jar:/apex/com.android.sdkext/javalib/framework-sdkextensions.jar:/apex/com.android.tethering/javalib/framework-connectivity.jar:/apex/com.android.tethering/javalib/framework-connectivity-t.jar:/apex/com.android.tethering/javalib/framework-tethering.jar:/apex/com.android.uwb/javalib/framework-uwb.jar:/apex/com.android.virt/javalib/framework-virtualization.jar:/apex/com.android.wifi/javalib/framework-wifi.jar', 'DEX2OATBOOTCLASSPATH': '/apex/com.android.art/javalib/core-oj.jar:/apex/com.android.art/javalib/core-libart.jar:/apex/com.android.art/javalib/okhttp.jar:/apex/com.android.art/javalib/bouncycastle.jar:/apex/com.android.art/javalib/apache-xml.jar:/system/framework/framework.jar:/system/framework/framework-graphics.jar:/system/framework/ext.jar:/system/framework/telephony-common.jar:/system/framework/voip-common.jar:/system/framework/ims-common.jar:/apex/com.android.i18n/javalib/core-icu4j.jar', 'SYSTEMSERVERCLASSPATH': '/system/framework/com.android.location.provider.jar:/system/framework/services.jar:/apex/com.android.adservices/javalib/service-adservices.jar:/apex/com.android.adservices/javalib/service-sdksandbox.jar:/apex/com.android.appsearch/javalib/service-appsearch.jar:/apex/com.android.art/javalib/service-art.jar:/apex/com.android.configinfrastructure/javalib/service-configinfrastructure.jar:/apex/com.android.healthfitness/javalib/service-healthfitness.jar:/apex/com.android.media/javalib/service-media-s.jar:/apex/com.android.ondevicepersonalization/javalib/service-ondevicepersonalization.jar:/apex/com.android.permission/javalib/service-permission.jar:/apex/com.android.rkpd/javalib/service-rkp.jar', 'STANDALONE_SYSTEMSERVER_JARS': '/apex/com.android.btservices/javalib/service-bluetooth.jar:/apex/com.android.devicelock/javalib/service-devicelock.jar:/apex/com.android.os.statsd/javalib/service-statsd.jar:/apex/com.android.scheduling/javalib/service-scheduling.jar:/apex/com.android.tethering/javalib/service-connectivity.jar:/apex/com.android.uwb/javalib/s
2024-03-06 13:51:17.417  5227-5299  python                  org.test.myapp                       I  Android kivy bootstrap done. __name__ is __main__
2024-03-06 13:51:17.417  5227-5299  python                  org.test.myapp                       I  AND: Ran string
2024-03-06 13:51:17.417  5227-5299  python                  org.test.myapp                       I  Run user program, change dir and execute entrypoint
2024-03-06 13:51:17.564  5227-5299  python                  org.test.myapp                       I  Traceback (most recent call last):
2024-03-06 13:51:17.564  5227-5299  python                  org.test.myapp                       I    File "main.py", line 5, in <module>
2024-03-06 13:51:17.567  5227-5299  python                  org.test.myapp                       I      from myapp.gui import gui
2024-03-06 13:51:17.568  5227-5299  python                  org.test.myapp                       I  ModuleNotFoundError: No module named 'myapp'

Here's some (not after clean) build output, though.

# Check configuration tokens
# Ensure build layout
# Check configuration tokens
# Preparing build
# Check requirements for android
# Search for Git (git)
#  -> found at /usr/bin/git
# Search for Cython (cython)
#  -> found at /home/bevans/.local/bin/cython
# Search for Java compiler (javac)
#  -> found at /usr/lib/jvm/java-21-openjdk-21.0.2.0.13-1.rolling.el8.x86_64/bin/javac
# Search for Java keytool (keytool)
#  -> found at /usr/lib/jvm/java-17-openjdk-17.0.10.0.7-2.el8.x86_64/bin/keytool
# Install platform
# Run ['git', 'config', '--get', 'remote.origin.url']
# Cwd /workspace/minexample/.buildozer/android/platform/python-for-android
https://github.com/kivy/python-for-android.git
# Run ['git', 'branch', '-vv']
# Cwd /workspace/minexample/.buildozer/android/platform/python-for-android
* develop b3cc0343 [origin/develop] Merge pull request #2978 from rivian/arch-ref-fix
# Run ['/usr/local/bin/python3.10', '-m', 'pip', 'install', '-q', '--user', 'appdirs', 'colorama>=0.3.3', 'jinja2', 'sh>=1.10, <2.0; sys_platform!="win32"', 'build', 'toml', 'packaging', 'setuptools']
# Cwd None
# Apache ANT found at /home/bevans/.buildozer/android/platform/apache-ant-1.9.4
# Android SDK found at /home/bevans/.buildozer/android/platform/android-sdk
# Recommended android's NDK version by p4a is: 25b
# Android NDK found at /home/bevans/.buildozer/android/platform/android-ndk-r25b
# Run ['/usr/local/bin/python3.10', '-m', 'pythonforandroid.toolchain', 'aab', '-h', '--color=always', '--storage-dir=/workspace/minexample/.buildozer/android/platform/build-arm64-v8a_armeabi-v7a', '--ndk-api=21', '--ignore-setup-py', '--debug']
# Cwd /workspace/minexample/.buildozer/android/platform/python-for-android
usage: toolchain.py aab [-h] [--debug] [--color {always,never,auto}]
                        [--sdk-dir SDK_DIR] [--ndk-dir NDK_DIR]
                        [--android-api ANDROID_API]
                        [--ndk-version NDK_VERSION] [--ndk-api NDK_API]
                        [--symlink-bootstrap-files]
                        [--storage-dir STORAGE_DIR] [--arch ARCH]
                        [--dist-name DIST_NAME] [--requirements REQUIREMENTS]
                        [--recipe-blacklist RECIPE_BLACKLIST]
                        [--blacklist-requirements BLACKLIST_REQUIREMENTS]
                        [--bootstrap BOOTSTRAP] [--hook HOOK] [--force-build]
                        [--no-force-build] [--require-perfect-match]
                        [--no-require-perfect-match] [--allow-replace-dist]
                        [--no-allow-replace-dist]
                        [--local-recipes LOCAL_RECIPES]
                        [--activity-class-name ACTIVITY_CLASS_NAME]
                        [--service-class-name SERVICE_CLASS_NAME]
                        [--java-build-tool {auto,ant,gradle}] [--copy-libs]
                        [--no-copy-libs] [--add-asset ASSETS]
                        [--add-resource RESOURCES] [--private PRIVATE]
                        [--use-setup-py] [--ignore-setup-py] [--release]
                        [--with-debug-symbols] [--keystore KEYSTORE]
                        [--signkey SIGNKEY] [--keystorepw KEYSTOREPW]
                        [--signkeypw SIGNKEYPW]

options:
  -h, --help            show this help message and exit
  --debug               Display debug output and all build info
  --color {always,never,auto}
                        Enable or disable color output (default enabled on
                        tty)
  --sdk-dir SDK_DIR, --sdk_dir SDK_DIR
                        The filepath where the Android SDK is installed
  --ndk-dir NDK_DIR, --ndk_dir NDK_DIR
                        The filepath where the Android NDK is installed
  --android-api ANDROID_API, --android_api ANDROID_API
                        The Android API level to build against defaults to 33
                        if not specified.
  --ndk-version NDK_VERSION, --ndk_version NDK_VERSION
                        DEPRECATED: the NDK version is now found automatically
                        or not at all.
  --ndk-api NDK_API     The Android API level to compile against. This should
                        be your *minimal supported* API, not normally the same
                        as your --android-api. Defaults to min(ANDROID_API,
                        21) if not specified.
  --symlink-bootstrap-files, --ssymlink_bootstrap_files
                        If True, symlinks the bootstrap files creation. This
                        is useful for development only, it could also cause
                        weird problems.
  --storage-dir STORAGE_DIR
                        Primary storage directory for downloads and builds
                        (default: /home/bevans/.local/share/python-for-
                        android)
  --arch ARCH           The archs to build for.
  --dist-name DIST_NAME, --dist_name DIST_NAME
                        The name of the distribution to use or create
  --requirements REQUIREMENTS
                        Dependencies of your app, should be recipe names or
                        Python modules. NOT NECESSARY if you are using Python
                        3 with --use-setup-py
  --recipe-blacklist RECIPE_BLACKLIST
                        Blacklist an internal recipe from use. Allows
                        disabling Python 3 core modules to save size
  --blacklist-requirements BLACKLIST_REQUIREMENTS
                        Blacklist an internal recipe from use. Allows
                        disabling Python 3 core modules to save size
  --bootstrap BOOTSTRAP
                        The bootstrap to build with. Leave unset to choose
                        automatically.
  --hook HOOK           Filename to a module that contains python-for-android
                        hooks
  --local-recipes LOCAL_RECIPES, --local_recipes LOCAL_RECIPES
                        Directory to look for local recipes
  --activity-class-name ACTIVITY_CLASS_NAME
                        The full java class name of the main activity
  --service-class-name SERVICE_CLASS_NAME
                        Full java package name of the PythonService class
  --java-build-tool {auto,ant,gradle}
                        The java build tool to use when packaging the APK,
                        defaults to automatically selecting an appropriate
                        tool.
  --add-asset ASSETS    Put this in the assets folder in the apk.
  --add-resource RESOURCES
                        Put this in the res folder in the apk.
  --private PRIVATE     the directory with the app source code files
                        (containing your main.py entrypoint)
  --use-setup-py        Process the setup.py of a project if present.
                        (Experimental!
  --ignore-setup-py     Don't run the setup.py of a project if present. This
                        may be required if the setup.py is not designed to
                        work inside p4a (e.g. by installing dependencies that
                        won't work or aren't desired on Android
  --release             Build your app as a non-debug release build. (Disables
                        gdb debugging among other things)
  --with-debug-symbols  Will keep debug symbols from `.so` files.
  --keystore KEYSTORE   Keystore for JAR signing key, will use jarsigner
                        default if not specified (release build only)
  --signkey SIGNKEY     Key alias to sign PARSER_APK. with (release build
                        only)
  --keystorepw KEYSTOREPW
                        Password for keystore
  --signkeypw SIGNKEYPW
                        Password for key alias

  Whether to force compilation of a new distribution

  --force-build
  --no-force-build      (this is the default)
  --require-perfect-match
  --no-require-perfect-match
                        (this is the default)
  --allow-replace-dist  (this is the default)
  --no-allow-replace-dist
  --copy-libs
  --no-copy-libs        (this is the default)
# Check application requirements
# Compile platform
# Run ['/usr/local/bin/python3.10', '-m', 'pythonforandroid.toolchain', 'create', '--dist_name=myapp', '--bootstrap=sdl2', '--requirements=python3,kivy', '--arch=arm64-v8a', '--arch=armeabi-v7a', '--copy-libs', '--color=always', '--storage-dir=/workspace/minexample/.buildozer/android/platform/build-arm64-v8a_armeabi-v7a', '--ndk-api=21', '--ignore-setup-py', '--debug']
# Cwd /workspace/minexample/.buildozer/android/platform/python-for-android
# Build the application #52
# Copy application source from /workspace/minexample
# Create directory /workspace/minexample/.buildozer/android/app
# Copy /workspace/minexample/setup.py
# Copy /workspace/minexample/main.py
# Create directory /workspace/minexample/.buildozer/android/app/myapp
# Copy /workspace/minexample/myapp/__init__.py
# Create directory /workspace/minexample/.buildozer/android/app/myapp/utils
# Copy /workspace/minexample/myapp/utils/__init__.py
# Copy /workspace/minexample/myapp/utils/logging.py
# Create directory /workspace/minexample/.buildozer/android/app/myapp/cli
# Copy /workspace/minexample/myapp/cli/__init__.py
# Copy /workspace/minexample/myapp/cli/cli.py
# Create directory /workspace/minexample/.buildozer/android/app/myapp/gui
# Copy /workspace/minexample/myapp/gui/gui.py
# Copy /workspace/minexample/myapp/gui/__init__.py
# Create directory /workspace/minexample/.buildozer/android/app/myapp/myapp.egg-info
# Copy /workspace/minexample/myapp/myapp.egg-info/PKG-INFO
# Create directory /workspace/minexample/.buildozer/android/app/bin/inspect/assets
# Copy /workspace/minexample/bin/inspect/assets/main.py
# Create directory /workspace/minexample/.buildozer/android/app/bin/inspect/res/drawable-hdpi-v4
# Copy /workspace/minexample/bin/inspect/res/drawable-hdpi-v4/ic_launcher.png
# Create directory /workspace/minexample/.buildozer/android/app/bin/inspect/res/drawable-mdpi-v4
# Copy /workspace/minexample/bin/inspect/res/drawable-mdpi-v4/ic_launcher.png
# Create directory /workspace/minexample/.buildozer/android/app/bin/inspect/res/drawable-xhdpi-v4
# Copy /workspace/minexample/bin/inspect/res/drawable-xhdpi-v4/ic_launcher.png
# Create directory /workspace/minexample/.buildozer/android/app/bin/inspect/res/drawable-xxhdpi-v4
# Copy /workspace/minexample/bin/inspect/res/drawable-xxhdpi-v4/ic_launcher.png
# Create directory /workspace/minexample/.buildozer/android/app/bin/inspect/res/drawable
# Copy /workspace/minexample/bin/inspect/res/drawable/presplash.jpg
# Create directory /workspace/minexample/.buildozer/android/app/bin/inspect/res/mipmap
# Copy /workspace/minexample/bin/inspect/res/mipmap/icon.png
# Create directory /workspace/minexample/.buildozer/android/app/build/lib/utils
# Copy /workspace/minexample/build/lib/utils/__init__.py
# Copy /workspace/minexample/build/lib/utils/logging.py
# Create directory /workspace/minexample/.buildozer/android/app/build/lib/cli
# Copy /workspace/minexample/build/lib/cli/__init__.py
# Copy /workspace/minexample/build/lib/cli/cli.py
# Create directory /workspace/minexample/.buildozer/android/app/build/lib/gui
# Copy /workspace/minexample/build/lib/gui/gui.py
# Copy /workspace/minexample/build/lib/gui/__init__.py
# Package the application
# project.properties updated
# Run ['/usr/local/bin/python3.10', '-m', 'pythonforandroid.toolchain', 'apk', '--bootstrap', 'sdl2', '--dist_name', 'myapp', '--name', 'My Application', '--version', '0.1', '--package', 'org.test.myapp', '--minsdk', '21', '--ndk-api', '21', '--private', '/workspace/minexample/.buildozer/android/app', '--permission', 'android.permission.INTERNET', '--permission', '(name=android.permission.WRITE_EXTERNAL_STORAGE;maxSdkVersion=18)', '--permission', 'android.permission.READ_EXTERNAL_STORAGE', '--android-entrypoint', 'org.kivy.android.PythonActivity', '--android-apptheme', '@android:style/Theme.NoTitleBar', '--orientation', 'portrait', '--window', '--enable-androidx', '--copy-libs', '--no-byte-compile-python', '--arch', 'arm64-v8a', '--arch', 'armeabi-v7a', '--color=always', '--storage-dir=/workspace/minexample/.buildozer/android/platform/build-arm64-v8a_armeabi-v7a', '--ndk-api=21', '--ignore-setup-py', '--debug']
# Cwd /workspace/minexample/.buildozer/android/platform/python-for-android
Copying main.py's ONLY, since other app data is expected in site-packages.
Applying Java source code patches...
Applying patch: src/patches/SDLActivity.java.patch
Warning: failed to apply patch (exit code 1), assuming it is already applied:  src/patches/SDLActivity.java.patch
# Android packaging done!
# APK myapp-0.1-arm64-v8a_armeabi-v7a-debug.apk available in the bin directory
# Run ['/home/bevans/.buildozer/android/platform/android-sdk/platform-tools/adb', 'devices']
# Cwd None
List of devices attached
emulator-5554   device

# Deploy on emulator-5554
# Run ['/home/bevans/.buildozer/android/platform/android-sdk/platform-tools/adb', 'install', '-r', '/workspace/minexample/bin/myapp-0.1-arm64-v8a_armeabi-v7a-debug.apk']
# Cwd /home/bevans/.buildozer/android/platform
Performing Streamed Install
Success
# Application pushed.
# Run on emulator-5554
# Run ['/home/bevans/.buildozer/android/platform/android-sdk/platform-tools/adb', 'shell', 'am', 'start', '-n', 'org.test.myapp/org.kivy.android.PythonActivity', '-a', 'org.kivy.android.PythonActivity']
# Cwd /home/bevans/.buildozer/android/platform
Starting: Intent { act=org.kivy.android.PythonActivity cmp=org.test.myapp/org.kivy.android.PythonActivity }
# Waiting for application to start.
# Waiting for application to start.
# Application started.

I'm particularly drawn to the line

Copying main.py's ONLY, since other app data is expected in site-packages.

I'm not sure that it's getting the site-packages correctly?

kivy / buildozer