search

kivy / buildozer

Generic Python packager for Android and iOS
https://buildozer.readthedocs.io
MIT License
1.74k stars 499 forks source link

ModuleNotFound Error while deploying kivy app using Buildozer #1391

Open cyberkarim opened 2 years ago

cyberkarim commented 2 years ago

Versions

Description

I wasn't able to generate APK file for a very simple kivy app that doesn't import any external packages. It's a basic kivy GUI. i actually followed all the installation steps including dependencies, plugged in my android device with the developer mode turned on while the command "buildozer android debug deploy run" is running. It always mentions "ModuleNotFoundError: No module named '_posixsubprocess'. I actually tried installing subprocess32 on my system, added it in the requirement line in buildozer.spec. The error just keeps popping everytime

Command:

buildozer android debug deploy run

Spec file:

spec.txt


### Logs

 STDOUT:
make: Entering directory '/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/build-armeabi-v7a/build/other_builds/hostpython3/desktop/hostpython3/native-build'
 CC='gcc -pthread' LDSHARED='gcc -pthread -shared    ' OPT='-DNDEBUG -g -fwrapv -O3 -Wall'      _TCLTK_INCLUDES='' _TCLTK_LIBS=''       ./python -E /home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/build-armeabi-v7a/build/other_builds/hostpython3/desktop/hostpython3/setup.py  build
Traceback (most recent call last):
  File "/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/build-armeabi-v7a/build/other_builds/hostpython3/desktop/hostpython3/Lib/subprocess.py", line 64, in <module>
    import msvcrt
ModuleNotFoundError: No module named 'msvcrt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/build-armeabi-v7a/build/other_builds/hostpython3/desktop/hostpython3/setup.py", line 13, in <module>
    from distutils import log
  File "/home/dark/.local/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 92, in create_module
    return importlib.import_module('setuptools._distutils')
  File "/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/build-armeabi-v7a/build/other_builds/hostpython3/desktop/hostpython3/Lib/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/dark/.local/lib/python3.8/site-packages/setuptools/__init__.py", line 8, in <module>
    import _distutils_hack.override  # noqa: F401
  File "/home/dark/.local/lib/python3.8/site-packages/_distutils_hack/override.py", line 1, in <module>
    __import__('_distutils_hack').do_override()
  File "/home/dark/.local/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 73, in do_override
    ensure_local_distutils()
  File "/home/dark/.local/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 60, in ensure_local_distutils
    core = importlib.import_module('distutils.core')
  File "/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/build-armeabi-v7a/build/other_builds/hostpython3/desktop/hostpython3/Lib/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/dark/.local/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 17, in <module>
    from distutils.dist import Distribution
  File "/home/dark/.local/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 19, in <module>
    from distutils.util import check_environ, strtobool, rfc822_escape
  File "/home/dark/.local/lib/python3.8/site-packages/setuptools/_distutils/util.py", line 14, in <module>
    from distutils.spawn import spawn
  File "/home/dark/.local/lib/python3.8/site-packages/setuptools/_distutils/spawn.py", line 11, in <module>
    import subprocess
  File "/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/build-armeabi-v7a/build/other_builds/hostpython3/desktop/hostpython3/Lib/subprocess.py", line 69, in <module>
    import _posixsubprocess
ModuleNotFoundError: No module named '_posixsubprocess'
make: *** [Makefile:614: sharedmods] Error 1
make: Leaving directory '/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/build-armeabi-v7a/build/other_builds/hostpython3/desktop/hostpython3/native-build'

  STDERR:

Traceback (most recent call last):                                                                                                                                                           
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 1276, in <module>
    main()
  File "/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/python-for-android/pythonforandroid/entrypoints.py", line 18, in main
    ToolchainCL()
  File "/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 725, in __init__
    getattr(self, command)(args)
  File "/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 153, in wrapper_func
    build_dist_from_args(ctx, dist, args)
  File "/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/python-for-android/pythonforandroid/toolchain.py", line 212, in build_dist_from_args
    build_recipes(build_order, python_modules, ctx,
  File "/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/python-for-android/pythonforandroid/build.py", line 580, in build_recipes
    recipe.build_arch(arch)
  File "/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/python-for-android/pythonforandroid/recipes/hostpython3/__init__.py", line 113, in build_arch
    shprint(sh.make, '-j', str(cpu_count()), '-C', build_dir)
  File "/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/python-for-android/pythonforandroid/logger.py", line 167, in shprint
    for line in output:
  File "/usr/local/lib/python3.8/dist-packages/sh-1.14.2-py3.8.egg/sh.py", line 953, in next
    self.wait()
  File "/usr/local/lib/python3.8/dist-packages/sh-1.14.2-py3.8.egg/sh.py", line 879, in wait
    self.handle_command_exit_code(exit_code)
  File "/usr/local/lib/python3.8/dist-packages/sh-1.14.2-py3.8.egg/sh.py", line 905, in handle_command_exit_code
    raise exc
sh.ErrorReturnCode_2: 

  RAN: /usr/bin/make -j 2 -C /home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/build-armeabi-v7a/build/other_builds/hostpython3/desktop/hostpython3/native-build

  STDOUT:
make: Entering directory '/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/build-armeabi-v7a/build/other_builds/hostpython3/desktop/hostpython3/native-build'
 CC='gcc -pthread' LDSHARED='gcc -pthread -shared    ' OPT='-DNDEBUG -g -fwrapv -O3 -Wall'      _TCLTK_INCLUDES='' _TCLTK_LIBS=''       ./python -E /home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/build-armeabi-v7a/build/other_builds/hostpython3/desktop/hostpython3/setup.py  build
Traceback (most recent call last):
  File "/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/build-armeabi-v7a/build/other_builds/hostpython3/desktop/hostpython3/Lib/subprocess.py", line 64, in <module>
    import msvcrt
ModuleNotFoundError: No module named 'msvcrt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/build-armeabi-v7a/build/other_builds/hostpython3/desktop/hostpython3/setup.py", line 13, in <module>
    from distutils import log
  File "/home/dark/.local/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 92, in create_module
    return importlib.import_module('setuptools._distutils')
  File "/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/build-armeabi-v7a/build/other_builds/hostpython3/desktop/hostpython3/Lib/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/dark/.local/lib/python3.8/site-packages/setuptools/__init__.py", line 8, in <module>
    import _distutils_hack.override  # noqa: F401
  File "/home/dark/.local/lib/python3.8/site-packages/_distutils_hack/override.py", line 1, in <module>
    __import__('_distutils_hack').do_override()
  File "/home/dark/.local/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 73, in do_override
    ensure_local_distutils()
  File "/home/dark/.local/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 60, in ensure_local_distutils
    core = importlib.import_module('distutils.core')
  File "/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/build-armeabi-v7a/build/other_builds/hostpython3/desktop/hostpython3/Lib/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/dark/.local/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 17, in <module>
    from distutils.dist import Distribution
  File "/home/dark/.local/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 19, in <module>
    from distutils.util import check_environ, strtobool, rfc822_escape
  File "/home/dark/.local/lib/python3.8/site-packages/setuptools/_distutils/util.py", line 14, in <module>
    from distutils.spawn import spawn
  File "/home/dark/.local/lib/python3.8/site-packages/setuptools/_distutils/spawn.py", line 11, in <module>
    import subprocess
  File "/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/build-armeabi-v7a/build/other_builds/hostpython3/desktop/hostpython3/Lib/subprocess.py", line 69, in <module>
    import _posixsubprocess
ModuleNotFoundError: No module named '_posixsubprocess'
make: *** [Makefile:614: sharedmods] Error 1
make: Leaving directory '/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/build-armeabi-v7a/build/other_builds/hostpython3/desktop/hostpython3/native-build'

  STDERR:

# Command failed: /usr/bin/python3 -m pythonforandroid.toolchain create --dist_name=myapp --bootstrap=sdl2 --requirements=python3==3.8.10,kivy,hostpython3==3.8.10 --arch armeabi-v7a --copy-libs --color=always --storage-dir="/home/dark/Myprojects/kivymobileapp/kivyapp/.buildozer/android/platform/build-armeabi-v7a" --ndk-api=21
# ENVIRONMENT:
#     SHELL = '/bin/bash'
#     SESSION_MANAGER = 'local/dark-NEO15C-4BK1T:@/tmp/.ICE-unix/2988,unix/dark-NEO15C-4BK1T:/tmp/.ICE-unix/2988'
#     QT_ACCESSIBILITY = '1'
#     COLORTERM = 'truecolor'
#     XDG_CONFIG_DIRS = '/etc/xdg/xdg-ubuntu:/etc/xdg'
#     XDG_MENU_PREFIX = 'gnome-'
#     TERM_PROGRAM_VERSION = '1.63.2'
#     GNOME_DESKTOP_SESSION_ID = 'this-is-deprecated'
#     APPLICATION_INSIGHTS_NO_DIAGNOSTIC_CHANNEL = 'true'
#     LANGUAGE = 'en'
#     LC_ADDRESS = 'ar_MA.UTF-8'
#     GNOME_SHELL_SESSION_MODE = 'ubuntu'
#     LC_NAME = 'ar_MA.UTF-8'
#     SSH_AUTH_SOCK = '/run/user/1000/keyring/ssh'
#     BREAKPAD_DUMP_LOCATION = '/home/dark/.config/Code/exthost Crash Reports'
#     XMODIFIERS = '@im=ibus'
#     DESKTOP_SESSION = 'ubuntu'
#     LC_MONETARY = 'ar_MA.UTF-8'
#     SSH_AGENT_PID = '2940'
#     BAMF_DESKTOP_FILE_HINT = '/var/lib/snapd/desktop/applications/code_code.desktop'
#     GTK_MODULES = 'gail:atk-bridge'
#     PWD = '/home/dark/Myprojects/kivymobileapp/kivyapp'
#     GSETTINGS_SCHEMA_DIR = '/snap/code/85/usr/share/glib-2.0/schemas'
#     XDG_SESSION_DESKTOP = 'ubuntu'
#     LOGNAME = 'dark'
#     XDG_SESSION_TYPE = 'x11'
#     GPG_AGENT_INFO = '/run/user/1000/gnupg/S.gpg-agent:0:1'
#     XAUTHORITY = '/run/user/1000/gdm/Xauthority'
#     VSCODE_GIT_ASKPASS_NODE = '/snap/code/85/usr/share/code/code'
#     GJS_DEBUG_TOPICS = 'JS ERROR;JS LOG'
#     WINDOWPATH = '2'
#     HOME = '/home/dark'
#     USERNAME = 'dark'
#     IM_CONFIG_PHASE = '1'
#     LC_PAPER = 'ar_MA.UTF-8'
#     LANG = 'en_US.UTF-8'
#     LS_COLORS = 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:'
#     XDG_CURRENT_DESKTOP = 'Unity'
#     DISABLE_WAYLAND = '1'
#     GIT_ASKPASS = '/snap/code/85/usr/share/code/resources/app/extensions/git/dist/askpass.sh'
#     INVOCATION_ID = 'db7393c9019a46b7a880cda181295965'
#     MANAGERPID = '2701'
#     CHROME_DESKTOP = 'code-url-handler.desktop'
#     GJS_DEBUG_OUTPUT = 'stderr'
#     VSCODE_GIT_ASKPASS_EXTRA_ARGS = '--ms-enable-electron-run-as-node'
#     LESSCLOSE = '/usr/bin/lesspipe %s %s'
#     XDG_SESSION_CLASS = 'user'
#     TERM = 'xterm-256color'
#     LC_IDENTIFICATION = 'ar_MA.UTF-8'
#     LESSOPEN = '| /usr/bin/lesspipe %s'
#     USER = 'dark'
#     VSCODE_GIT_IPC_HANDLE = '/run/user/1000/vscode-git-7a03aa50bc.sock'
#     DISPLAY = ':0'
#     SHLVL = '1'
#     LC_TELEPHONE = 'ar_MA.UTF-8'
#     QT_IM_MODULE = 'ibus'
#     LC_MEASUREMENT = 'ar_MA.UTF-8'
#     PAPERSIZE = 'a4'
#     XDG_RUNTIME_DIR = '/run/user/1000'
#     LC_TIME = 'ar_MA.UTF-8'
#     VSCODE_GIT_ASKPASS_MAIN = '/snap/code/85/usr/share/code/resources/app/extensions/git/dist/askpass-main.js'
#     JOURNAL_STREAM = '8:49245'
#     XDG_DATA_DIRS = '/usr/share/ubuntu:/usr/local/share/:/usr/share/:/var/lib/snapd/desktop'
#     GDK_BACKEND = 'x11'
#     PATH = '/home/dark/.buildozer/android/platform/apache-ant-1.9.4/bin:/home/dark/.local/bin:/home/dark/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/dark/.local/bin/'
#     GDMSESSION = 'ubuntu'
#     ORIGINAL_XDG_CURRENT_DESKTOP = 'ubuntu:GNOME'
#     DBUS_SESSION_BUS_ADDRESS = 'unix:path=/run/user/1000/bus'
#     GIO_LAUNCHED_DESKTOP_FILE_PID = '4988'
#     GIO_LAUNCHED_DESKTOP_FILE = '/var/lib/snapd/desktop/applications/code_code.desktop'
#     LC_NUMERIC = 'ar_MA.UTF-8'
#     TERM_PROGRAM = 'vscode'
#     _ = '/home/dark/.local/bin/buildozer'
#     OLDPWD = '/home/dark/Myprojects/kivymobileapp'
#     PACKAGES_PATH = '/home/dark/.buildozer/android/packages'
#     ANDROIDSDK = '/home/dark/.buildozer/android/platform/android-sdk'
#     ANDROIDNDK = '/home/dark/.buildozer/android/platform/android-ndk-r19c'
#     ANDROIDAPI = '27'
#     ANDROIDMINAPI = '21'
# 
# Buildozer failed to execute the last command
# The error might be hidden in the log above this error
# Please read the full log, and search for it before
# raising an issue with buildozer itself.
# In case of a bug report, please add a full log with log_level = 2
tshirtman commented 2 years ago

A few people have reported this issue, we weren't able to reproduce it yet, one user was able to work around it by resetting their environment (WSL) and reinstalling buildozer again following the documentation.

I reset WSL, installed buildozer and dependencies, created kivy example as a project without venv. SUCCESS! I DID NOT update setuptools. I also added ~/.local/bin at the beginning of PATH.

A setuptools update is suspected to be the issue, but hasn't been confirmed yet.

RobertFlatt commented 2 years ago

We are looking for clues to replicate the issue, could you please 1) Share the result of pip3 list | grep setuptools

2) appclean and rebuild after changing requirements to remove the version pinning (it is one minor release ahead of the default which is the max) requirements = python3,kivy

This is about helping us find a place to look, rather than fixing your issue. Thanks.

cyberkarim commented 2 years ago

We are looking for clues to replicate the issue, could you please

1. Share the result of
   `pip3 list | grep setuptools`

2. appclean and rebuild after changing requirements to remove the version pinning (it is one minor release ahead of the default which is the max)
   `requirements = python3,kivy`

This is about helping us find a place to look, rather than fixing your issue. Thanks.

Yes sure 1 - setuptools 60.1.0 2 - same errors about _posixsubprocess module not found

cyberkarim commented 2 years ago

A few people have reported this issue, we weren't able to reproduce it yet, one user was able to work around it by resetting their environment (WSL) and reinstalling buildozer again following the documentation.

I reset WSL, installed buildozer and dependencies, created kivy example as a project without venv. SUCCESS! I DID NOT update setuptools. I also added ~/.local/bin at the beginning of PATH.

A setuptools update is suspected to be the issue, but hasn't been confirmed yet.

I'm not using WSL, I have ubuntu distro installed seperately from windows via dual booting. I added the line : export PATH=$PATH:~/.local/bin/ at the end of my ~/.bashrc file and still having the same issue .

RobertFlatt commented 2 years ago

Two things to try:

1) This worked for somebody on Discord (but I don't understand why) pinning hostpython3==3.9.9 and python3==3.9.9

2) uninstall setuptools and install an older version, say 58.0.0

For each, clean then build. Please let me know what happens.

Android-for-Python commented 2 years ago

It turns out "worked" means built. It is possible step 1 will give a runtime error: ModuleNotFoundError: No module named 'kivy._clock

cyberkarim commented 2 years ago

Two things to try:

1. This worked for somebody on Discord (but I don't understand why)
   pinning hostpython3==3.9.9 and python3==3.9.9

2. uninstall setuptools and install an older version, say 58.0.0

For each, clean then build. Please let me know what happens.

Your second suggestion helped a lot, and the modulenotfound error disappeared. I was able to generate .apk file and deployed the app on my device. Thank you for giving me the hint. Downgrading setupstools to 58.0.0 was the answer.

Android-for-Python commented 2 years ago

Thank you for testing. Another tester on Discord got the same result so I think we can be confident in this.

For somebody else reading this, I didn't yet figure out the critical version, it is probably higher than 58. That was just my be safe guess.

Thanks again.

sylvioalves commented 2 years ago

Reinstalling older setuptool (58.0.0) worked for me as well.

lucasrantunes commented 2 years ago

Reinstalling an older version of setuptools worked for me too.

Just a tip: after downgrading setuptools, be sure that you really installed an older version. In my case, even after setting version 58.0.0 to be installed, it was installing 60.X.

First: pip3 uninstall setuptools && pip3 install setuptools=58.0.0

Second: pip3 list | grep setuptools to check if 58.0.0 is really installed.

cristianmejiaarias commented 1 year ago

just you need to add the module before sdl2_ttf == 2.0.15, pillow if you put the modules after that it not working, I don't now why

example requirements = hostpython3==3.9.9,python3==3.9.9, requests, idna, chardet, charset_normalizer, urllib3, gcloud, oauth2client, requests-toolbelt,pyrebase, kivy==2.1.0, kivymd==1.1.1, sdl2_ttf == 2.0.15, pillow,