search

kivy / buildozer

Generic Python packager for Android and iOS
https://buildozer.readthedocs.io
MIT License
1.74k stars 499 forks source link

buildozer-remote error: TypeError: get() takes 3 positional arguments but 4 were given #1163

Open erm3nda opened 4 years ago

erm3nda commented 4 years ago

Versions

Description

First, the buildozer-remote is not documented. After looking at the script (https://github.com/kivy/buildozer/blob/master/buildozer/scripts/remote.py) i've ended up with something like this in my buildozer.spec:


[remote:sandbox]
host = 8.8.8.8 < my server ip here ofc
user = kivy
build_directory = .

Now, when I issue buildozer-remote sandbox android debug the remote section is readed, the script remote is called, but im getting this error:


Traceback (most recent call last):
  File "/home/m3nda/.local/bin/buildozer-remote", line 8, in <module>
    sys.exit(main())
  File "/home/m3nda/.local/lib/python3.8/site-packages/buildozer/scripts/remote.py", line 273, in main
    BuildozerRemote().run_command(sys.argv[1:])
  File "/home/m3nda/.local/lib/python3.8/site-packages/buildozer/scripts/remote.py", line 73, in run_command
    self.remote_host = remote_host = self.config.get(
TypeError: get() takes 3 positional arguments but 4 were given

If look at the get() and I can't find anything wrong on it:

self.remote_host = remote_host = self.config.get(
        remote_section, 'host', '')

I don't know if I am missing something or it is really a bug. Any hint appreciated.

buildozer.spec

Command:

buildozer-remote sandbox android debug

Spec file: (I've stripped all comments and redundant info)

[app]
title = Kivy service OSC demo
package.name = oscservice
package.domain = org.kivy
source.dir = src
source.include_exts = py,png,jpg,kv,atlas
version.regex = __version__ = '(.*)'
version.filename = %(source.dir)s/main.py
requirements = python3,kivy,oscpy
orientation = all
services = Pong:service.py
fullscreen = 0

[app:android.permissions]
READ_EXTERNAL_STORAGE
WRITE_EXTERNAL_STORAGE
ACCESS_NETWORK_STATE
ACCESS_WIFI_STATE
BROADCAST_STICKY
FOREGROUND_SERVICE
INTERNET
KILL_BACKGROUND_PROCESSES
MANAGE_OWN_CALLS
RECEIVE_BOOT_COMPLETED
REQUEST_COMPANION_RUN_IN_BACKGROUND
REQUEST_COMPANION_USE_DATA_IN_BACKGROUND
VIBRATE
WAKE_LOCK
SYSTEM_ALERT_WINDOW

android.minapi = 21

p4a.branch = master
android.arch = armeabi-v7a
p4a.bootstrap = sdl2

[buildozer]
log_level = 2
warn_on_root = 0

[remote:sandbox]
host = 8.8.8.8 <- my server ip here
user = kivy
build_directory = .

Logs

# Check configuration tokens
Traceback (most recent call last):
  File "/home/m3nda/.local/bin/buildozer-remote", line 8, in <module>
    sys.exit(main())
  File "/home/m3nda/.local/lib/python3.8/site-packages/buildozer/scripts/remote.py", line 273, in main
    BuildozerRemote().run_command(sys.argv[1:])
  File "/home/m3nda/.local/lib/python3.8/site-packages/buildozer/scripts/remote.py", line 73, in run_command
    self.remote_host = remote_host = self.config.get(
TypeError: get() takes 3 positional arguments but 4 were given
erm3nda commented 4 years ago

I would love that any of the devs could fill some minimal info about the buildozer-remote script usage at docs. There's literally no mention about it.

AndreMiras commented 4 years ago

Thanks for the bug report. I'm not surprised if this is broken to be honest. A wild guess would be the config object no longer takes a default. So try to change this:

self.remote_host = remote_host = self.config.get(
        remote_section, 'host', '')

by:

self.remote_host = remote_host = self.config.get(
        remote_section, 'host')

Anyway this whole thing needs some love for sure. So integration/unit test, fixes and documentation

erm3nda commented 4 years ago

That's a nice wild guessing.

Nevermind not my fault :-) so I didn't even try to guess about configparser, and I was starting to be frustated about the whole "remote" thing not working.

I feel very important the remote part, because im not always working from a power computer, sometimes I work with thinclients and kinda like, and having my vps ready to compile everything is a great tool for me.

Now, and because of the absence of defaults values, the config needs to have present every object defined.

Part of the mechanism relly on that empty defaults, especially the "ssh key" management, where the functions that "get the system default keys" will not work at all, so, you have to define even the ssh key at object "identity". Not a drama at all, few config parameters.

Now I have other kind of problems related to the identity key, but I guess that is paramiko/cryptography related, so we can close this issue.

Thank you very much.

AndreMiras commented 4 years ago

Thanks for the update. Let's keep the ticket open because it's a confirmed bug

erm3nda commented 4 years ago

Ok. After fixing all parameters, I've got a new error (__path__ is not defined):

Traceback (most recent call last):
  File "/home/m3nda/.local/bin/buildozer-remote", line 8, in <module>
    sys.exit(main())
  File "/home/m3nda/.local/lib/python3.8/site-packages/buildozer/scripts/remote.py", line 273, in main
    BuildozerRemote().run_command(sys.argv[1:])
  File "/home/m3nda/.local/lib/python3.8/site-packages/buildozer/scripts/remote.py", line 102, in run_command
    self._ensure_buildozer()
  File "/home/m3nda/.local/lib/python3.8/site-packages/buildozer/scripts/remote.py", line 133, in _ensure_buildozer
    self._ssh_sync(__path__[0])  # noqa: F821 undefined name
NameError: name '__path__' is not defined

I've looked at https://github.com/kivy/buildozer/blob/master/buildozer/scripts/remote.py#L130 Seems that __path__ is not defined anywhere.

I've guessed that the current path was already supplied by buildozer, so I just replaced __path__[0] with current path '.' and started to sync properly.

erm3nda commented 4 years ago

After that configparser comes back again with:


Traceback (most recent call last):
  File "/home/m3nda/.local/bin/buildozer-remote", line 8, in <module>
    sys.exit(main())
  File "/home/m3nda/.local/lib/python3.8/site-packages/buildozer/scripts/remote.py", line 273, in main
    BuildozerRemote().run_command(sys.argv[1:])
  File "/home/m3nda/.local/lib/python3.8/site-packages/buildozer/scripts/remote.py", line 103, in run_command
    self._sync_application_sources()
  File "/home/m3nda/.local/lib/python3.8/site-packages/buildozer/scripts/remote.py", line 142, in _sync_application_sources
    config.read('buildozer.spec')
  File "/usr/lib/python3.8/configparser.py", line 697, in read
    self._read(fp, filename)
  File "/usr/lib/python3.8/configparser.py", line 1113, in _read
    raise e
configparser.ParsingError: Source contains parsing errors: 'buildozer.spec'
        [line 90]: 'READ_EXTERNAL_STORAGE,\n'
        [line 91]: 'WRITE_EXTERNAL_STORAGE,\n'
        [line 92]: 'ACCESS_NETWORK_STATE,\n'
        [line 93]: 'ACCESS_WIFI_STATE,\n'
        [line 94]: 'BROADCAST_STICKY,\n'
        [line 95]: 'FOREGROUND_SERVICE,\n'
        [line 96]: 'INTERNET,\n'
        [line 97]: 'KILL_BACKGROUND_PROCESSES,\n'
        [line 98]: 'MANAGE_OWN_CALLS,\n'
        [line 99]: 'RECEIVE_BOOT_COMPLETED,\n'
        [line 100]: 'REQUEST_COMPANION_RUN_IN_BACKGROUND,\n'
        [line 101]: 'REQUEST_COMPANION_USE_DATA_IN_BACKGROUND,\n'
        [line 102]: 'VIBRATE,\n'
        [line 103]: 'WAKE_LOCK,\n'
        [line 104]: 'SYSTEM_ALERT_WINDOW,\n'

After avoid using the section-style and using the oneline-style for android.permissions it "passed".

So, solved using it like: android.permissions = INTERNET, WAKE_LOCK, etc

erm3nda commented 4 years ago

After that, the ssh return is not ok, and outputs this:

# Execute remote buildozer
# Execute remote command source ~/.profile;cd /home/kivy/./org.kivy.oscservice;env PYTHONPATH=/home/kivy/./org.kivy.oscservice:$PYTHONPATH python -c "import buildozer, sys;buildozer.Buildozer().run_command(sys.argv[1:])" --verbose android debug 2>&1
Socket exception: Bad file descriptor (9)
# Closing remote connection
Traceback (most recent call last):
  File "/home/m3nda/.local/lib/python3.8/site-packages/buildozer/scripts/remote.py", line 206, in _ssh_command
    self._interactive_shell(channel)
  File "/home/m3nda/.local/lib/python3.8/site-packages/buildozer/scripts/remote.py", line 212, in _interactive_shell
    self._posix_shell(chan)
  File "/home/m3nda/.local/lib/python3.8/site-packages/buildozer/scripts/remote.py", line 229, in _posix_shell
    stdout.write(x)
TypeError: write() argument must be str, not bytes

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/m3nda/.local/bin/buildozer-remote", line 8, in <module>
    sys.exit(main())
  File "/home/m3nda/.local/lib/python3.8/site-packages/buildozer/scripts/remote.py", line 273, in main
    BuildozerRemote().run_command(sys.argv[1:])
  File "/home/m3nda/.local/lib/python3.8/site-packages/buildozer/scripts/remote.py", line 104, in run_command
    self._do_remote_commands(args[1:])
  File "/home/m3nda/.local/lib/python3.8/site-packages/buildozer/scripts/remote.py", line 162, in _do_remote_commands
    self._ssh_command(cmd)
  File "/home/m3nda/.local/lib/python3.8/site-packages/buildozer/scripts/remote.py", line 208, in _ssh_command
    channel.close()
  File "/home/m3nda/.local/lib/python3.8/site-packages/paramiko/channel.py", line 671, in close
    self.transport._send_user_message(m)
  File "/home/m3nda/.local/lib/python3.8/site-packages/paramiko/transport.py", line 1863, in _send_user_message
    self._send_message(data)
  File "/home/m3nda/.local/lib/python3.8/site-packages/paramiko/transport.py", line 1839, in _send_message
    self.packetizer.send_message(data)
  File "/home/m3nda/.local/lib/python3.8/site-packages/paramiko/packet.py", line 431, in send_message
    self.write_all(out)
  File "/home/m3nda/.local/lib/python3.8/site-packages/paramiko/packet.py", line 367, in write_all
    raise EOFError()
EOFError

Solved by adding proper decode at https://github.com/kivy/buildozer/blob/master/buildozer/scripts/remote.py#L229: stdout.write(x.decode("utf-8") )

Actually, the return was "no module named buildozer" because the ENV tried to use main python2 (at remote) instead the python used to invoke the command. I don't know if there's a solution to tell remote which should it use, or just hardcode it to the remote.py command.

Actually I just hardcoded it at: https://github.com/kivy/buildozer/blob/master/buildozer/scripts/remote.py#L156

Now the build process starts. Having strange issues with missing parameters on print, but cannot figure why because I am using python3 as far as I can see.

I've enter to the server with ssh and issue the build command manually and everything is built ok, so there's nothing missing. I am not using venv, so there's not a problem with configs or modules on my side.

I'll look carefully at env used by buildozer, seems the problem. There's a lot of things with buildozer-remote to fix.

erm3nda commented 4 years ago

Untill I have more free time to digg and fix the buildozer-remote script I'll use this basic rsync + build script: https://gist.github.com/erm3nda/6b05680655c14c8e0c0c32126cdf9268