adb_shell.exceptions.InvalidCommandError in `connect()` with Android emulator

[mal-mel]

Hello, Jeff, i need help) When i running my code:

from adb_shell import adb_device
from adb_shell.auth.sign_pythonrsa import PythonRSASigner
import os

class Controller:
    def __init__(self, host: str, port: int):
        self.host, self.port = host, port
        self.device, self._signer = None, None

    def connect_to_device(self, timeout: int):
        self.device = adb_device.AdbDeviceTcp(self.host, self.port, default_timeout_s=timeout)
        self._make_sign("~/.android/adbkey")
        self.device.connect(auth_timeout_s=0.1, rsa_keys=[self._signer])

    def _make_sign(self, path: str):
        with open(os.path.expanduser(path)) as file:
            rsa_key = file.read()
        self._signer = PythonRSASigner("", rsa_key)

    def shell(self, command: str) -> str:
        output = self.device.shell(command, decode=True)
        return output

if __name__ == '__main__':
    adb_controller = Controller("localhost", 5556)
    adb_controller.connect_to_device(timeout=10)
    print(adb_controller.shell("echo work!"))

An emulator is running on 5556 port. I get an error adb_shell.exceptions.InvalidCommandError: ('Unknown command: 72646e41', 1919184449, (543451503, 1936617283)) or something like this, I would like to know why this is happening.

[JeffLIrion]

def from_int(n):
    return ''.join(chr((n >> (i * 8)) % 256) for i in range(4)).encode('utf-8')

That is the opposite of https://github.com/JeffLIrion/adb_shell/blob/825294d9a4923b8857380b6b5b51a8937e91aa70/tests/test_adb_device.py#L25-L26

I don't know what 72646e41 is, but the following 3 numbers translate to Andr + oid + Cons.

Your exception is occurring here: https://github.com/JeffLIrion/adb_shell/blob/825294d9a4923b8857380b6b5b51a8937e91aa70/adb_shell/adb_device.py#L847

If you want more info, you need to turn on debug logging.

[mal-mel]

Ok, but how do I make it work?)

[JeffLIrion]

More info is needed to see what's going on.

import logging

logging.getLogger().setLevel(logging.DEBUG)

Post a log and I'll take a look.

[mal-mel]

2020-02-29 16:25:44,905 : [adb_device.py.921]: DEBUG : bulk_write: b'CNXN\x00\x00\x00\x01\x00\x10\x00\x00\x13\x00\x00\x00c\x05\x00\x00\xbc\xb1\xa7\xb1'
2020-02-29 16:25:44,905 : [adb_device.py.923]: DEBUG : bulk_write: b'host::9da23738bc43\x00'
2020-02-29 16:25:44,906 : [adb_device.py.784]: DEBUG : bulk_read(24): b'Android Console: Authent'

[JeffLIrion]

Try this with debug logging enabled and post a log.

adb_controller = Controller("localhost", 5556)
try:
    adb_controller.connect_to_device(timeout=10)
except:
    pass
print(adb_controller.device._handle.bulk_read(1000, 5))

[mal-mel]

This is output:

b"ication required\r\nAndroid Console: type 'auth <auth_token>' to authenticate\r\nAndroid Console: you can find your <auth_token> in \r\n'/root/.emulator_console_auth_token'\r\nOK\r\n"
Traceback (most recent call last):
  File "test.py", line 62, in <module>
    print(adb_controller.shell("echo work!"))
  File "test.py", line 26, in shell
    output = self.device.shell(command, decode=True)
  File "/usr/local/lib/python3.6/dist-packages/adb_shell/adb_device.py", line 426, in shell
    return self._service(b'shell', command.encode('utf8'), timeout_s, total_timeout_s, decode)
  File "/usr/local/lib/python3.6/dist-packages/adb_shell/adb_device.py", line 402, in _service
    return b''.join(self._streaming_command(service, command, adb_info)).decode('utf8')
  File "/usr/local/lib/python3.6/dist-packages/adb_shell/adb_device.py", line 953, in _streaming_command
    self._open(b'%s:%s' % (service, command), adb_info)
  File "/usr/local/lib/python3.6/dist-packages/adb_shell/adb_device.py", line 738, in _open
    _, adb_info.remote_id, their_local_id, _ = self._read([constants.OKAY], adb_info)
  File "/usr/local/lib/python3.6/dist-packages/adb_shell/adb_device.py", line 783, in _read
    msg = self._handle.bulk_read(constants.MESSAGE_SIZE, adb_info.timeout_s)
  File "/usr/local/lib/python3.6/dist-packages/adb_shell/handle/tcp_handle.py", line 127, in bulk_read
    raise TcpTimeoutException(msg)
adb_shell.exceptions.TcpTimeoutException: Reading from localhost:5556 timed out (10 seconds)

Log:

2020-02-29 18:21:50,678 : [adb_device.py.921]: DEBUG : bulk_write: b'CNXN\x00\x00\x00\x01\x00\x10\x00\x00\x13\x00\x00\x00c\x05\x00\x00\xbc\xb1\xa7\xb1'
2020-02-29 18:21:50,684 : [adb_device.py.923]: DEBUG : bulk_write: b'host::9da23738bc43\x00'
2020-02-29 18:21:50,685 : [adb_device.py.784]: DEBUG : bulk_read(24): b'Android Console: Authent'
2020-02-29 18:21:50,685 : [adb_device.py.921]: DEBUG : bulk_write: b'OPEN\x01\x00\x00\x00\x00\x00\x00\x00\x11\x00\x00\x00\xf5\x05\x00\x00\xb0\xaf\xba\xb1'
2020-02-29 18:21:50,685 : [adb_device.py.923]: DEBUG : bulk_write: b'shell:echo work!\x00'

[JeffLIrion]

That output is not from the same code I posted.

[mal-mel]

Ok, i commented print(adb_controller.shell("echo work!")) output:

b"ication required\r\nAndroid Console: type 'auth <auth_token>' to authenticate\r\nAndroid Console: you can find your <auth_token> in \r\n'/root/.emulator_console_auth_token'\r\nOK\r\n"

[JeffLIrion]

OK, I see now.

It looks like it's saying that it will only work if you use the key /root/.emulator_console_auth_token.

This is the reply you get when you try to connect:

Android Console: Authentication required
Android Console: type 'auth <auth_token>' to authenticate
Android Console: you can find your <auth_token> in 
'/root/.emulator_console_auth_token'
OK

Have you connected to the emulator using the official ADB binary?

[mal-mel]

Yes, the official adb connect to emulator successfully

[JeffLIrion]

Do you have a file /root/.emulator_console_auth_token?

If so, try using that in your connect_to_device function.

[JeffLIrion]

Have you had a chance to try using the key /root/.emulator_console_auth_token (if it exists)?

[mal-mel]

Hello, Jeff, sorry for the silence, there was no access to the computer Yes, i have /root/.emulator_console_auth_token. Should I use it instead of ~ / .android / adbkey?

[mal-mel]

If i connect to Android console manually and enter auth <my_token> all work fine (i have access to the Android Console, but my goal is android shell). How i to do this with your lib?

[JeffLIrion]

Hello, Jeff, sorry for the silence, there was no access to the computer Yes, i have /root/.emulator_console_auth_token. Should I use it instead of ~ / .android / adbkey?

Yes, please try that.

[mal-mel]

Yes, I tried to make a signature with this key, but nothing

[JeffLIrion]

Can you post the output from this, as you did before.

adb_controller = Controller("localhost", 5556)
try:
    adb_controller.connect_to_device(timeout=10)
except:
    pass
print(adb_controller.device._handle.bulk_read(1000, 5))

[mal-mel]

Traceback (most recent call last):
  File "test.py", line 36, in <module>
    print(adb_controller.device._handle.bulk_read(1000, 5))
  File "/usr/local/lib/python3.6/dist-packages/adb_shell/handle/tcp_handle.py", line 122, in bulk_read
    readable, _, _ = select.select([self._connection], [], [], timeout)
TypeError: argument must be an int, or have a fileno() method.

[JeffLIrion]

I need debug logs to see the data that's being sent and read.

import logging
import os

from adb_shell import adb_device
from adb_shell.auth.sign_pythonrsa import PythonRSASigner

logging.getLogger().setLevel(logging.DEBUG)

class Controller:
    def __init__(self, host: str, port: int):
        self.host, self.port = host, port
        self.device, self._signer = None, None

    def connect_to_device(self, timeout: int):
        self.device = adb_device.AdbDeviceTcp(self.host, self.port, default_timeout_s=timeout)
        self._make_sign("/root/.emulator_console_auth_token")
        self.device.connect(auth_timeout_s=0.1, rsa_keys=[self._signer])

    def _make_sign(self, path: str):
        with open(os.path.expanduser(path)) as file:
            rsa_key = file.read()
        self._signer = PythonRSASigner("", rsa_key)

    def shell(self, command: str) -> str:
        output = self.device.shell(command, decode=True)
        return output

if __name__ == '__main__':
    adb_controller = Controller("localhost", 5556)
    try:
        adb_controller.connect_to_device(timeout=10)
    except:
        pass
    print(adb_controller.device._handle.bulk_read(1000, 5))

[kunzhipeng]

I met the same problem after i ctrl+c a running "shell command".

print device1.shell('busybox ping www.baidu.com', timeout_s=10, total_timeout_s=10) Traceback (most recent call last): File "", line 1, in File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 460, in shell return self._service(b'shell', command.encode('utf8'), timeout_s, total_timeout_s, decode) File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 404, in _service return b''.join(self._streaming_command(service, command, adb_info)).decode('utf8') File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 1013, in _streaming_command for data in self._read_until_close(adb_info): File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 955, in _read_until_close cmd, data = self._read_until([constants.CLSE, constants.WRTE], adb_info) File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 908, in _read_until cmd, remote_id2, local_id2, data = self._read(expected_cmds, adb_info) File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 841, in _read msg = self._handle.bulk_read(constants.MESSAGE_SIZE, adb_info.timeout_s) File "C:\Python27\lib\site-packages\adb_shell\handle\tcp_handle.py", line 122, in bulkread readable, , _ = select.select([self._connection], [], [], timeout) KeyboardInterrupt print device1.shell('busybox ping www.baidu.com') Traceback (most recent call last): File "", line 1, in File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 460, in shell return self._service(b'shell', command.encode('utf8'), timeout_s, total_timeout_s, decode) File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 404, in _service return b''.join(self._streaming_command(service, command, adb_info)).decode('utf8') File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 1011, in _streaming_command self._open(b'%s:%s' % (service, command), adb_info) File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 796, in open , adb_info.remote_id, their_localid, = self._read([constants.OKAY], adb_info) File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 847, in _read raise exceptions.InvalidCommandError('Unknown command: %x' % cmd, cmd, (arg0, arg1)) adb_shell.exceptions.InvalidCommandError: ('Unknown command: 62203436', 1646277686, (1936028793, 1869768224))

I also noticed the timeout seems notworking. Following command stuck all the time: print device1.shell('busybox ping www.baidu.com', timeout_s=10, total_timeout_s=10)

[kunzhipeng]

I can reproduce the "adb_shell.exceptions.InvalidCommandError" problem like following:

Run: print device1.shell('busybox ping www.baidu.com') Press “Ctrl + C”, then run: print device1.shell('echo hi')

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 460, in shell
    return self._service(b'shell', command.encode('utf8'), timeout_s, total_timeout_s, decode)
  File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 404, in _service
    return b''.join(self._streaming_command(service, command, adb_info)).decode('utf8')
  File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 1011, in _streaming_command
    self._open(b'%s:%s' % (service, command), adb_info)
  File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 796, in _open
    _, adb_info.remote_id, their_local_id, _ = self._read([constants.OKAY], adb_info)
  File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 847, in _read
    raise exceptions.InvalidCommandError('Unknown command: %x' % cmd, cmd, (arg0, arg1))
adb_shell.exceptions.InvalidCommandError: ('Unknown command: 62203436', 1646277686, (1936028793, 1869768224))

[JeffLIrion]

@kunzhipeng your issue is different. I changed the title of this issue to reflect that it is about an error when trying to connect to an Android emulator.

I think the Ctrl+C issue you're describing here is even different than the new issue you created. I think the problem is that you're stopping the current task on the host computer -- i.e., running that she'll command via adb_shell in Python -- but one or both of the following scenarios occurs:

• the command you sent is still running on the Android device
• the command has stopped on the Android device, but there is still more output from that command for adb_shell to read

Either way, when you try to run a new shell command, it expects a response to the new command but receives a response from the old command, hence the error.

I'm not sure how to handle that other than closing the connection and reconnecting.

[combacsa]

Glad that I'm facing the exactly same issue here. I'm using Mac OS X Cata...blah, and using good old telnet I was able to do somethings, but facing InvalidCommandError: ('Unknown command: 72646e41', 1919184449, (543451503, 1936617283)).

[combacsa]

I replaced this line self._make_sign("/Users/khbyun/.android/adbkey") and debugging log I got is:

b"ication required\r\nAndroid Console: type 'auth <auth_token>' to authenticate\r\nAndroid Console: you can find your <auth_token> in \r\n'/Users/khbyun/.emulator_console_auth_token'\r\nOK\r\n"

[combacsa]

So @JeffLIrion is the above log message enough for you to debug?

[JeffLIrion]

@combacsa try using the key /Users/khbyun/.emulator_console_auth_token, if it exists.

[combacsa]

@JeffLIrion I tried with it but it failed with error, like it's not a permitable file for a RSA... exact log message is:

[combacsa]

ValueError: No PEM start marker "b'-----BEGIN PRIVATE KEY-----'" found

[combacsa]

(I'm kinda in a hurry, since I'm creating a code for a birthday present now...)

[combacsa]

Interestingly, when I just use telnet, My workflow goes like this:

Command: telnet localhost 5554

Output:

Trying ::1...
Connected to localhost.
Escape character is '^]'.
Android Console: Authentication required
Android Console: type 'auth <auth_token>' to authenticate
Android Console: you can find your <auth_token> in
'/Users/khbyun/.emulator_console_auth_token'
OK

Then I just type the contents from '/Users/khbyun/.emulator_console_auth_token' like

auth <BLAHBLAH>

then

Android Console: type 'help' for a list of commands
OK

So, there could be a sudden change happend in adb's socket protocol?

[JeffLIrion]

ValueError: No PEM start marker "b'-----BEGIN PRIVATE KEY-----'" found

Try making a backup of the key and then pasting that as the first line in the original.

-----BEGIN PRIVATE KEY-----
<The rest of the file>

[kolayne]

+1

Python 3.8.2 (default, Jul 16 2020, 14:00:26) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from adb_shell.adb_device import AdbDeviceTcp, AdbDeviceUsb
>>> from adb_shell.auth.sign_pythonrsa import PythonRSASigner
>>> import os.path as op
>>> with open(op.expanduser('~/.android/adbkey')) as f:
...     priv = f.read()
... 
>>> signer = PythonRSASigner('', priv)
>>> device2 = AdbDeviceTcp('localhost', 5554)
>>> device2.connect(rsa_keys=[signer])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/adb_shell/adb_device.py", line 219, in connect
    cmd, arg0, arg1, banner = self._read([constants.AUTH, constants.CNXN], adb_info)
  File "/usr/local/lib/python3.8/dist-packages/adb_shell/adb_device.py", line 758, in _read
    raise exceptions.InvalidCommandError('Unknown command: %x' % cmd, cmd, (arg0, arg1))
adb_shell.exceptions.InvalidCommandError: ('Unknown command: 72646e41', 1919184449, (543451503, 1936617283))
>>>

[JeffLIrion]

I just published a new release. It doesn't fix the problem, but the exception message will contain more information. Although I'm sure it will just be the same info in this post: https://github.com/JeffLIrion/adb_shell/issues/81#issuecomment-592958149

(Actually, it will only be the first part of it: "Android Console: Authent")

This isn't a use case that I plan to support, but if someone wants to contribute a fix they are welcome to do so.

[barkside]

Hi Jeff! I'm getting this error too. Following on from comments, using emulator_console_auth_token with BEGIN PRIVATE KEY and END_PRIVATE_KEY fails with pyasn1.error.SubstrateUnderrunError: 105<10 at <TagSet object, tags 192:32:1> when decoding the rsa private key.

In my case emulator_console_auth_token is tiny "4ekw5I+6eOro0wvG" -

[barkside]

If I empty the auth token file I get: Unknown command: 1919184449 = 'b'Andr'' (arg0 = 543451503, arg1 = 1936617283, msg = 'b"Android Console: type 'h"')

DEBUG:adb_shell.adb_device:bulk_write(24): b'CNXN\x00\x00\x00\x01\x00\x00\x10\x00\x10\x00\x00\x00K\x04\x00\x00\xbc\xb1\xa7\xb1' DEBUG:adb_shell.adb_device:bulk_write(16): b'host::C-RB-L480\x00' DEBUG:adb_shell.adb_device:bulk_read(24): b"Android Console: type 'h"