Building APK with docker fails after first time (Requested API target 28 is not available)

OlafHaag commented 4 years ago

Versions

Python: 3.7.7, also tried 3.7.8
OS: Windows WSL2/Docker
Buildozer: 1.2.0.dev0

Description

I was trying to build an APK using docker with the commands in the README. The first time went through, but the second time I got an error: Build failed: Requested API target 28 is not available, install it with the SDK android tool. Also tried with API target 27. I deleted any .buildozer directories several times over.
I'd also expect the README to give hints as how to prevent the docker run command to download the Android SDK & NDK every time it is run. I did not get any support in the discord support channel. Since this is unexpected behaviour resulting in build errors, I open this issue.

buildozer.spec

Command:

docker run --volume "$PWD":/home/user/hostcwd kivy/buildozer android debug

Also tried after building the Dockerfile from the buildozer repo:

docker run --volume "$PWD":/home/user/hostcwd buildozer android debug

Spec file:

[app]

# (str) Title of your application
title = NeuroPsy Research

# (str) Package name
package.name = NeuroPsyResearch

# (str) Package domain (needed for android/ios packaging)
package.domain = com.olafhaag

# (str) Source code where the main.py live
source.dir = .

# (list) Source files to include (let empty to include all the files)
source.include_exts = py,png,jpg,kv,atlas,ogg,mo,txt

# (list) List of inclusions using pattern matching
#source.include_patterns = assets/*,images/*.png

# (list) Source files to exclude (let empty to not exclude anything)
#source.exclude_exts = spec

# (list) List of directory to exclude (let empty to not exclude anything)
source.exclude_dirs = tests, bin, .idea, .git

# (list) List of exclusions using pattern matching
#source.exclude_patterns = license,images/*/*.jpg

# (str) Application versioning (method 1)
# version = 0.1

# (str) Application versioning (method 2)
version.regex = __version__ = ['"](.*)['"]
version.filename = %(source.dir)s/src/version.py

# (list) Application requirements
# comma separated e.g. requirements = sqlite3,kivy
requirements = hostpython3==3.7.7, python3==3.7.7, kivy==1.11.1, kivymd==0.104.1, plyer, numpy, requests, urllib3, chardet, idna

# (str) Custom source folders for requirements
# Sets custom source for any requirements with recipes
# requirements.source.kivy = ../../kivy

# (list) Garden requirements
#garden_requirements =

# (str) Presplash of the application
presplash.filename = %(source.dir)s/res/loading.png

# (str) Icon of the application
icon.filename = %(source.dir)s/res/icons/web_hi_res_512.png

# (str) Supported orientation (one of landscape, sensorLandscape, portrait or all)
orientation = portrait

# (list) List of service to declare
#services = NAME:ENTRYPOINT_TO_PY,NAME2:ENTRYPOINT2_TO_PY

#
# OSX Specific
#

#
# author = © Copyright Info

# change the major version of python used by the app
osx.python_version = 3

# Kivy version to use
osx.kivy_version = 1.9.1

#
# Android specific
#

# (bool) Indicate if the application should be fullscreen or not
fullscreen = 1

# (string) Presplash background color (for new android toolchain)
# Supported formats are: #RRGGBB #AARRGGBB or one of the following names:
# red, blue, green, black, white, gray, cyan, magenta, yellow, lightgray,
# darkgray, grey, lightgrey, darkgrey, aqua, fuchsia, lime, maroon, navy,
# olive, purple, silver, teal.
android.presplash_color = teal

# (list) Permissions
android.permissions = INTERNET,VIBRATE

# (int) Target Android API, should be as high as possible.
android.api = 28

# (int) Minimum API your APK will support.
android.minapi = 21

# (int) Android SDK version to use
#android.sdk = 20

# (str) Android NDK version to use
#android.ndk = 19c

# (int) Android NDK API to use. This is the minimum API your app will support, it should usually match android.minapi.
#android.ndk_api = 21

# (bool) Use --private data storage (True) or --dir public storage (False)
android.private_storage = True

# (str) Android NDK directory (if empty, it will be automatically downloaded.)
#android.ndk_path =

# (str) Android SDK directory (if empty, it will be automatically downloaded.)
#android.sdk_path =

# (str) ANT directory (if empty, it will be automatically downloaded.)
#android.ant_path =

# (bool) If True, then skip trying to update the Android sdk
# This can be useful to avoid excess Internet downloads or save time
# when an update is due and you just want to test/build your package
# android.skip_update = False

# (bool) If True, then automatically accept SDK license
# agreements. This is intended for automation only. If set to False,
# the default, you will be shown the license when first running
# buildozer.
android.accept_sdk_license = True

# (str) Android entry point, default is ok for Kivy-based app
#android.entrypoint = org.renpy.android.PythonActivity

# (str) Android app theme, default is ok for Kivy-based app
# android.apptheme = "@android:style/Theme.NoTitleBar"

# (list) Pattern to whitelist for the whole project
#android.whitelist =

# (str) Path to a custom whitelist file
#android.whitelist_src =

# (str) Path to a custom blacklist file
#android.blacklist_src =

# (list) List of Java .jar files to add to the libs so that pyjnius can access
# their classes. Don't add jars that you do not need, since extra jars can slow
# down the build process. Allows wildcards matching, for example:
# OUYA-ODK/libs/*.jar
#android.add_jars = foo.jar,bar.jar,path/to/more/*.jar

# (list) List of Java files to add to the android project (can be java or a
# directory containing the files)
#android.add_src =

# (list) Android AAR archives to add (currently works only with sdl2_gradle
# bootstrap)
#android.add_aars =

# (list) Gradle dependencies to add (currently works only with sdl2_gradle
# bootstrap)
#android.gradle_dependencies =

# (list) add java compile options
# this can for example be necessary when importing certain java libraries using the 'android.gradle_dependencies' option
# see https://developer.android.com/studio/write/java8-support for further information
# android.add_compile_options = "sourceCompatibility = 1.8", "targetCompatibility = 1.8"

# (list) Gradle repositories to add {can be necessary for some android.gradle_dependencies}
# please enclose in double quotes 
# e.g. android.gradle_repositories = "maven { url 'https://kotlin.bintray.com/ktor' }"
#android.add_gradle_repositories =

# (list) packaging options to add 
# see https://google.github.io/android-gradle-dsl/current/com.android.build.gradle.internal.dsl.PackagingOptions.html
# can be necessary to solve conflicts in gradle_dependencies
# please enclose in double quotes 
# e.g. android.add_packaging_options = "exclude 'META-INF/common.kotlin_module'", "exclude 'META-INF/*.kotlin_module'"
#android.add_gradle_repositories =

# (list) Java classes to add as activities to the manifest.
#android.add_activities = com.example.ExampleActivity

# (str) OUYA Console category. Should be one of GAME or APP
# If you leave this blank, OUYA support will not be enabled
#android.ouya.category = GAME

# (str) Filename of OUYA Console icon. It must be a 732x412 png image.
#android.ouya.icon.filename = %(source.dir)s/data/ouya_icon.png

# (str) XML file to include as an intent filters in <activity> tag
#android.manifest.intent_filters =

# (str) launchMode to set for the main activity
#android.manifest.launch_mode = standard

# (list) Android additional libraries to copy into libs/armeabi
#android.add_libs_armeabi = libs/android/*.so
#android.add_libs_armeabi_v7a = libs/android-v7/*.so
#android.add_libs_arm64_v8a = libs/android-v8/*.so
#android.add_libs_x86 = libs/android-x86/*.so
#android.add_libs_mips = libs/android-mips/*.so

# (bool) Indicate whether the screen should stay on
# Don't forget to add the WAKE_LOCK permission if you set this to True
#android.wakelock = False

# (list) Android application meta-data to set (key=value format)
#android.meta_data =

# (list) Android library project to add (will be added in the
# project.properties automatically.)
#android.library_references =

# (list) Android shared libraries which will be added to AndroidManifest.xml using <uses-library> tag
#android.uses_library =

# (str) Android logcat filters to use
android.logcat_filters = *:S python:D

# (bool) Copy library instead of making a libpymodules.so
#android.copy_libs = 1

# (str) The Android arch to build for, choices: armeabi-v7a, arm64-v8a, x86, x86_64
android.arch = armeabi-v7a

# (int) overrides automatic versionCode computation (used in build.gradle)
# this is not the same as app version and should only be edited if you know what you're doing
# android.numeric_version = 1

#
# Python for android (p4a) specific
#

# (str) python-for-android fork to use, defaults to upstream (kivy)
#p4a.fork = kivy

# (str) python-for-android branch to use, defaults to master
#p4a.branch = master

# (str) python-for-android git clone directory (if empty, it will be automatically cloned from github)
#p4a.source_dir =

# (str) The directory in which python-for-android should look for your own build recipes (if any)
#p4a.local_recipes =

# (str) Filename to the hook for p4a
#p4a.hook =

# (str) Bootstrap to use for android builds
# p4a.bootstrap = sdl2

# (int) port number to specify an explicit --port= p4a argument (eg for bootstrap flask)
#p4a.port =

#
# iOS specific
#

# (str) Path to a custom kivy-ios folder
#ios.kivy_ios_dir = ../kivy-ios
# Alternately, specify the URL and branch of a git checkout:
ios.kivy_ios_url = https://github.com/kivy/kivy-ios
ios.kivy_ios_branch = master

# Another platform dependency: ios-deploy
# Uncomment to use a custom checkout
#ios.ios_deploy_dir = ../ios_deploy
# Or specify URL and branch
ios.ios_deploy_url = https://github.com/phonegap/ios-deploy
ios.ios_deploy_branch = 1.10.0

# (bool) Whether or not to sign the code
ios.codesign.allowed = false

# (str) Name of the certificate to use for signing the debug version
# Get a list of available identities: buildozer ios list_identities
#ios.codesign.debug = "iPhone Developer: <lastname> <firstname> (<hexstring>)"

# (str) Name of the certificate to use for signing the release version
#ios.codesign.release = %(ios.codesign.debug)s

[buildozer]

# (int) Log level (0 = error only, 1 = info, 2 = debug (with command output))
log_level = 2

# (int) Display warning if buildozer is run as root (0 = False, 1 = True)
warn_on_root = 1

# (str) Path to build artifact storage, absolute or relative to spec file
# build_dir = ./.buildozer

# (str) Path to build output (i.e. .apk, .ipa) storage
# bin_dir = ./bin

#    -----------------------------------------------------------------------------
#    List as sections
#
#    You can define all the "list" as [section:key].
#    Each line will be considered as a option to the list.
#    Let's take [app] / source.exclude_patterns.
#    Instead of doing:
#
#[app]
#source.exclude_patterns = license,data/audio/*.wav,data/images/original/*
#
#    This can be translated into:
#
#[app:source.exclude_patterns]
#license
#data/audio/*.wav
#data/images/original/*
#

#    -----------------------------------------------------------------------------
#    Profiles
#
#    You can extend section / key with a profile
#    For example, you want to deploy a demo version of your application without
#    HD content. You could first change the title to add "(demo)" in the name
#    and extend the excluded directories to remove the HD content.
#
#[app@demo]
#title = My Application (demo)
#
#[app:source.exclude_patterns@demo]
#images/hd/*
#
#    Then, invoke the command line with the "demo" profile:
#
#buildozer --profile demo android debug

Logs

# Check configuration tokens
# Ensure build layout
# Create directory /home/user/.buildozer
# Create directory /home/user/.buildozer/cache
# Create directory /home/user/.buildozer/android/platform/android/platform
# Check configuration tokens
# Read available permissions from api-versions.xml
# Preparing build
# Check requirements for android
# Run 'dpkg --version'
# Cwd None
Debian 'dpkg' package management program version 1.19.7 (amd64).
This is free software; see the GNU General Public License version 2 or
later for copying conditions. There is NO warranty.
# Search for Git (git)
#  -> found at /usr/bin/git
# Search for Cython (cython)
#  -> found at /home/user/.local/bin/cython
# Search for Java compiler (javac)
#  -> found at /usr/lib/jvm/java-13-openjdk-amd64/bin/javac
# Search for Java keytool (keytool)
#  -> found at /usr/lib/jvm/java-13-openjdk-amd64/bin/keytool
# Install platform
# Run 'git config --get remote.origin.url'
# Cwd /home/user/hostcwd/.buildozer/android/platform/python-for-android
https://github.com/kivy/python-for-android.git
# Run 'git branch -vv'
# Cwd /home/user/hostcwd/.buildozer/android/platform/python-for-android
* master 1b3b0e3f [origin/master] Merge pull request #2225 from kivy/release-2020.06.02
WARNING: You are using pip version 20.1.1; however, version 20.2 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.
# Run '/usr/bin/python3 -m pip install -q --user \'appdirs\' \'colorama>=0.3.3\' \'jinja2\' \'six\' \'enum34; python_version<"3.4"\' \'sh>=1.10; sys_platform!="nt"\' \'pep517<0.7.0"\' \'toml\''
# Cwd None
# Android ANT is missing, downloading
# Downloading http://archive.apache.org/dist/ant/binaries/apache-ant-1.9.4-bin.tar.gz
# Run 'tar xzf apache-ant-1.9.4-bin.tar.gz'
# Cwd /home/user/.buildozer/android/platform/apache-ant-1.9.4
# Apache ANT installation done.
# Android SDK is missing, downloading
# Downloading https://dl.google.com/android/repository/commandlinetools-linux-6514223_latest.zip
# Unpacking Android SDK
# Run 'unzip -q /home/user/.buildozer/android/platform/android-sdk/commandlinetools-linux-6514223_latest.zip'
# Cwd /home/user/.buildozer/android/platform/android-sdk
# Android SDK tools base installation done.
# Recommended android's NDK version by p4a is: 19c
# Android NDK is missing, downloading
# Downloading https://dl.google.com/android/repository/android-ndk-r19c-linux-x86_64.zip
# Unpacking Android NDK
# Run 'unzip -q /home/user/.buildozer/android/platform/android-ndk-r19c-linux-x86_64.zip'
# Cwd /home/user/.buildozer/android/platform
# Rename /home/user/.buildozer/android/platform/android-ndk-r19c to /home/user/.buildozer/android/platform/android-ndk-r19c
# Android NDK installation done.
# Read available permissions from api-versions.xml
# Check application requirements
# Compile platform
# Run '/usr/bin/python3 -m pythonforandroid.toolchain create --dist_name=NeuroPsyResearch --bootstrap=sdl2 --requirements=hostpython3==3.7.7,python3==3.7.7,kivy==1.11.1,kivymd==0.104.1,plyer,numpy,requests,urllib3,chardet,idna --arch armeabi-v7a --copy-libs --color=always --storage-dir="/home/user/hostcwd/.buildozer/android/platform/build-armeabi-v7a" --ndk-api=21'
# Cwd /home/user/hostcwd/.buildozer/android/platform/python-for-android
/home/user/hostcwd/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py:84: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
[INFO]:    Recipe hostpython3: version "3.7.7" requested
[INFO]:    Recipe python3: version "3.7.7" requested
[INFO]:    Recipe kivy: version "1.11.1" requested
[INFO]:    Recipe kivymd: version "0.104.1" requested
[INFO]:    Will compile for the following archs: armeabi-v7a
[INFO]:    Found Android API target in $ANDROIDAPI: 28
[INFO]:    Available Android APIs are ()
[ERROR]:   Build failed: Requested API target 28 is not available, install it with the SDK android tool.
# Command failed: /usr/bin/python3 -m pythonforandroid.toolchain create --dist_name=NeuroPsyResearch --bootstrap=sdl2 --requirements=hostpython3==3.7.7,python3==3.7.7,kivy==1.11.1,kivymd==0.104.1,plyer,numpy,requests,urllib3,chardet,idna --arch armeabi-v7a --copy-libs --color=always --storage-dir="/home/user/hostcwd/.buildozer/android/platform/build-armeabi-v7a" --ndk-api=21
# ENVIRONMENT:
#     PATH = '/home/user/.buildozer/android/platform/apache-ant-1.9.4/bin:/home/user/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
#     HOSTNAME = '0affddd59bf1'
#     USER = 'user'
#     HOME_DIR = '/home/user'
#     WORK_DIR = '/home/user/hostcwd'
#     SRC_DIR = '/home/user/src'
#     LANG = 'en_US.UTF-8'
#     LANGUAGE = 'en_US.UTF-8'
#     LC_ALL = 'en_US.UTF-8'
#     HOME = '/home/user'
#     PACKAGES_PATH = '/home/user/.buildozer/android/packages'
#     ANDROIDSDK = '/home/user/.buildozer/android/platform/android-sdk'
#     ANDROIDNDK = '/home/user/.buildozer/android/platform/android-ndk-r19c'
#     ANDROIDAPI = '28'
#     ANDROIDMINAPI = '21'
#
# Buildozer failed to execute the last command
# The error might be hidden in the log above this error
# Please read the full log, and search for it before
# raising an issue with buildozer itself.
# In case of a bug report, please add a full log with log_level = 2

OlafHaag commented 4 years ago

@tshirtman gave me this working workaround: First: sudo chmod -R 777 /tmp then

docker run \
    --rm \
    --volume /tmp/.buildozer:/home/user/.buildozer \
    --volume $PWD:/home/user/hostcwd \
    kivy/buildozer android debug

Works repeatedly. Still downloads gradle each build though. If this is a reliable way to build with docker it should be mentioned in the docs.

tshirtman commented 4 years ago

You shouldn't need to chmod /tmp though It works because since docker creates the directory if it doesn't exist, and thus it has the right owner

OlafHaag commented 4 years ago

I didn't have a /tmp/.buildozer folder and got a Permission Denied error without the chmod at first try. docker run --rm --volume /tmp/.buildozer:/home/user/.buildozer --volume $PWD:/home/user/hostcwd kivy/buildozer android release'

# Check configuration tokens
# Ensure build layout
# Create directory /home/user/.buildozer/cache
Traceback (most recent call last):
  File "/home/user/.local/bin/buildozer", line 8, in <module>
    sys.exit(main())
  File "/home/user/.local/lib/python3.8/site-packages/buildozer/scripts/client.py", line 13, in main
    Buildozer().run_command(sys.argv[1:])
  File "/home/user/.local/lib/python3.8/site-packages/buildozer/__init__.py", line 1003, in run_command
    self.set_target(command)
  File "/home/user/.local/lib/python3.8/site-packages/buildozer/__init__.py", line 154, in set_target
    self.check_build_layout()
  File "/home/user/.local/lib/python3.8/site-packages/buildozer/__init__.py", line 446, in check_build_layout
    self.mkdir(self.global_cache_dir)
  File "/home/user/.local/lib/python3.8/site-packages/buildozer/__init__.py", line 548, in mkdir
    makedirs(dn)
  File "/usr/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/home/user/.buildozer/cache'

kivy / buildozer