ADB Connection Intermitant

[photinus]

I'm seeing the status of the single firetv I have setup flap between unknown and connected.

The error in the HA Log is: Update encountered an exception; will attempt to re-establish the ADB connection in the next update

[JeffLIrion]

Does it successfully re-establish the ADB connection?

[photinus]

It seems so, but looses it pretty quickly. Here is a snippit of this mornings logs, though it only seems to happen when the fire tv is "awake"

Update encountered an exception; will attempt to re-establish the ADB connection in the next update 7:11 AM custom_components/media_player/firetv.py (ERROR) Update encountered an exception; will attempt to re-establish the ADB connection in the next update 7:11 AM custom_components/media_player/firetv.py (ERROR) Update encountered an exception; will attempt to re-establish the ADB connection in the next update 7:10 AM custom_components/media_player/firetv.py (ERROR)

When it's in the unknown state it wont accept any commands either.

This is a model 2017 Pendant style Fire TV. It has a solid wifi connection with no dropped pings over a few minutes while experiencing the issue.

[JeffLIrion]

Are you sending a lot of commands to the Fire TV from Home Assistant when this occurs? If so, that's a known limitation for which I implemented a work-around, but if you send a lot of commands you'll cause the error that you're encountering.

[photinus]

Not sending any commands, that's just the idle behavior when the device is on or playing content. Attempting to send commands to the Fire TV is failing 2 out of 3 times.

[JeffLIrion]

What's your setup? HA installation type, HA version, host system hardware, etc.

[photinus]

Hassio vm image (running in vmware) HA 0.78.3 Dual Xeon E5-2620 @ 2ghz (Hex Core)

VM is configured with 2 vCPUs (not showing heavy load) and 1gb ram. I've increased both just to be safe, but it doesn't seem to be a resource problem.

[JeffLIrion]

My guess is that it's an issue with your virtual machine. Do you have any other components that do local polling, and if so, do they give you issues?

[photinus]

Hmm... I'll keep poking around. None of the other local polling components have any issues. I have the Vizio Smartcast component running without issue. The VM does seem to like having more ram, but this issue persists. The only thing of note so far is that it's only happening when the firetv is active. If it goes to sleep (tv gets turned off) it keeps it connection just fine.

[JeffLIrion]

That makes some sense, given that only one ADB command is necessary when the Fire TV is off. (https://github.com/JeffLIrion/homeassistant_native_firetv/blob/master/media_player/firetv.py#L130)

I still think this issue is due to your virtual machine. Perhaps it's interfering with the ADB connection?

[JeffLIrion]

My suggestion is to install Home Assistant in a different way (e.g., Python virtual environment or Docker container), copy over your configuration, and see if the issue persists. If it doesn't, then the virtual machine is more than likely causing the issue.

[photinus]

I tried setting up a connection to another one of my Fire TVs (Latest gen Fire TV Stick) and it's happy as a clam. No issues, staying connected even when playing media.

One major difference I see is the Pendant is returning a much longer sources lists. The Stick only returns three entries but I was still able to launch another app on the stick (both the Pendant and Stick have the same apps loaded)

Screenshot of device states: https://i.imgur.com/FjBv8d4.png

Config:

media_player:
  - platform: vizio
    host: 172.20.0.58
    access_token: <redacted>
  - platform: firetv
    name: livingroomfire
    host: 192.168.1.111
    adbkey: "/config/android/adbkey"
  - platform: firetv
    name: masterbedfire
    host: 192.168.1.112
    adbkey: "/config/android/adbkey"

I can spin up another home assistant instance in docker and see what it does.

[JeffLIrion]

Thanks for investigating that! I've only got Fire TV sticks, so I was unaware that another version might return a much longer list of sources. Please let me know how the docker instance works out!

[photinus]

Same error with the connection to the Pendant Fire TV from a docker instance of Home Assistant (same version, only component setup is the same fire tv). I did remove it from my other HA config before testing with the docker instance.

[photinus]

If I comment out the line that gets the currently running apps it holds a stable connection so there must either be something with the data it's getting back from ADB or the volume of data it's returning in the running apps list. Do you know the ADB commands it tries to run? I can try and run them manually against my system.

[JeffLIrion]

Here is the function that gets the list of running apps: https://github.com/JeffLIrion/python-firetv/blob/master/firetv/__init__.py#L148

And that function then calls this one: https://github.com/JeffLIrion/python-firetv/blob/master/firetv/__init__.py#L488

You'd have to translate those Python commands into ADB commands.

[photinus]

I'll give those a shot tonight, if nothing else I can gather the list from the pendant and the stick and see what is different.

[photinus]

So, It's pulling back a filtered list of adb shell ps

I do see one odd-ball coming back from the pendant, the logcat line is oddly formatted and the WCHAN column has a very different output. I'm not really fluent with Python (though can generally understand what is going on) but I'm not sure what is throwing things off.

(github formatting did not like the ps output) https://pastebin.com/zEXjf8tK

[photinus]

So, I get a similar crash if I try and use a different implementation of the python firetv server that has implemented the adbkeys, Looks like it might be something in the python adb library. I'll probably just hack together some functionality that doesn't involve pulling the list (or just use this on my firetv sticks)

[JeffLIrion]

Thanks for posting the results of those ADB commands. I don't see any reason why it should crash. I think if you want to figure this out, you'd need to

1. Fork my python-firetv repo and add in some logging statements
2. Modify the firetv.py custom component with the link to your forked firetv repo

Or you could probably just modify the custom component as:

            else:  # this is line 155
                # Get the running apps.
                #self._running_apps = self._firetv.running_apps()

                # Get the current app.
                current_app = self._firetv.current_app
                if isinstance(current_app, dict) and 'package' in current_app:
                    self._current_app = current_app['package']
                else:
                    self._current_app = current_app

                self._running_apps = [self._current_app]

[photinus]

I commented out that line in the module file and it is running and stable with the caveat of home assistant not seeing any "sources". I might dig into it some more and try and do some debugging. I did try setting up another one of the firetv server implementations and it crashes as well (though with different errors) which lead me to think it's actually an issue in the Google python adb library

[JeffLIrion]

OK, here's an easier way to debug this.

In the terminal:

# make a virtual environment
mkvirtualenv --python=python3 firetv_test

workon firetv_test
pip install pycryptodome rsa
pip install https://github.com/JeffLIrion/python-adb/zipball/master
pip install https://github.com/JeffLIrion/python-firetv/zipball/master

While in the virtual environment, run this script (modify the path to the adbkey as needed):

import firetv

host = '192.168.1.111:5555'
adbkey = '/config/android/adbkey'
f = firetv.FireTV(host, adbkey)

# get the running apps
x = f.running_apps()

# get the running apps "manually"
ps = f._adb.StreamingShell('ps')

# everything returned
bad_lines = list(ps)
bad_lines_formatted = '\n'.join([line for bad_line in bad_lines for line in bad_line.splitlines()])

# filter the running apps
result = [line.strip().rsplit(' ',1)[-1] for bad_line in bad_lines for line in bad_line.splitlines() if 'u0_a' in line]

print('\n\nRunning apps (from firetv)\n{0}\n{1}'.format('-'*26, x))
print('\n\nEverything returned\n{0}\n{1}'.format('-'*19, bad_lines_formatted))
print('\n\nFiltered running apps\n{0}\n{1}'.format('-'*21, result))

[photinus]

Set that up this morning and here is the resulting error: Data_length 1582 does not match actual number of bytes read: 1424 Data_length 4095 does not match actual number of bytes read: 1424 Data_length 925 does not match actual number of bytes read: 201 Traceback (most recent call last): File "test.py", line 14, in bad_lines = list(ps) File "/root/.virtualenvs/firetv_test/lib/python3.5/site-packages/adb/adb_protocol.py", line 439, in StreamingCommand for data in connection.ReadUntilClose(): AttributeError: 'NoneType' object has no attribute 'ReadUntilClose'

[photinus]

So, If I comment out the line: x = f.running_apps()

and change the first print line to reflect the ps variable vs the x variable like this: print('\n\nRunning apps (from firetv)\n{0}\n{1}'.format('-'*26, ps))

It returns output (though obviously ps is not a printable object without some filtering/work): https://pastebin.com/x0PzqEK2

[JeffLIrion]

Based on what you found, my theory is that the error is because the current_app command runs before the running_apps command has finished. Please try this modified file and let me know if that fixes it. https://github.com/JeffLIrion/homeassistant_native_firetv/blob/fix-issue7/media_player/firetv.py

[photinus]

Still getting the disconnects in the HA log.

This also seems to have broken other calls. It's unable to change source or issue other commands to the fire tv that would work intermittently before.

[JeffLIrion]

Thanks for helping me to debug this! Here's another attempt: https://github.com/JeffLIrion/homeassistant_native_firetv/blob/fix-issue7/media_player/firetv.py

[photinus]

Still seeing the disconnects.

[JeffLIrion]

OK, I guess showing the running apps as sources just isn't going to work for you. This work-around seems like the best option.

Updated firetv.py file: https://github.com/JeffLIrion/homeassistant_native_firetv/blob/fix-issue7/media_player/firetv.py

Updated configuration (note the get_sources: false entry)

media_player:
  - platform: vizio
    host: 172.20.0.58
    access_token: <redacted>
  - platform: firetv
    name: livingroomfire
    host: 192.168.1.111
    adbkey: "/config/android/adbkey"
    get_sources: false
  - platform: firetv
    name: masterbedfire
    host: 192.168.1.112
    adbkey: "/config/android/adbkey"

[photinus]

Awesome, Thanks for that!

[JeffLIrion]

Is it working correctly now and not giving you errors?

[photinus]

Still throwing occasional errors, it doesn't seem to be throwing errors as often, but it's still doing it.

[JeffLIrion]

Damn, this error just won't give up! Maybe the best solution (I think this should be possible) would be to enable ADB commands using the adb binaries via system calls in Python rather than the adb package.

[JeffLIrion]

I added a get_source option. If you set it to false, it won't retrieve the current app and the state won't be able to differentiate between "standby" and "paused" (it will be "standby").

Updated firetv.py file: https://github.com/JeffLIrion/homeassistant_native_firetv/blob/fix-issue7/media_player/firetv.py

Updated configuration (note the get_source: false entry)

media_player:
  - platform: vizio
    host: 172.20.0.58
    access_token: <redacted>
  - platform: firetv
    name: livingroomfire
    host: 192.168.1.111
    adbkey: "/config/android/adbkey"
    get_source: false
    get_sources: false
  - platform: firetv
    name: masterbedfire
    host: 192.168.1.112
    adbkey: "/config/android/adbkey"

[maddog986]

I was also having the re-establish errors. The update you just posted yesterday seems to have fixed it for me when adding "get_sources: false" to my media_player firetv config.

[JeffLIrion]

I was also having the re-establish errors. The update you just posted yesterday seems to have fixed it for me when adding "get_sources: false" to my media_player plex config.

I assume "plex" is a typo and that you meant firetv...

Is it working error-free now? And what model of Fire TV device are you using?

[maddog986]

Yes sorry, was doing plex configs at the same. Using two FireTV Sticks Gen2 and the newer FireTV. All 3 required adb key authentication.

[JeffLIrion]

Which one(s) were getting those errors?

[maddog986]

All 3 had the errors.

[JeffLIrion]

That's interesting, photinus was only having issues with the pendant style Fire TV, his Fire TV sticks were working just fine.

What's your Home Assistant setup? (HA version, Hass.io / docker / virtual environment / etc., host system and hardware)

[JeffLIrion]

I merged these workarounds into the master branch (https://github.com/JeffLIrion/homeassistant_native_firetv/commit/9db333b3013a02809f9db6c086118b3fcbbd9dba).

[Lippiero]

I have the same error logs which photinus mentioned in his initial post. Setting get_source and get_sources to false don't let the errors come up... so far - so good

I am running HA in a python virtual environment on an arm64 Odroid C2 with Ubuntu 16.04 LTS.

Is there a chance to get the Source and Sources functionality in the future?

[JeffLIrion]

@Lippiero it's not likely. I still don't understand why a seemingly small percentage of people get those errors, so I don't know a better way to fix them. But anyone is welcome to submit a pull request.

[JeffLIrion]

The latest version allows for using an ADB server to send the commands, that should be a stable solution. There's now a community Hass.io ADB addon for this. Furthermore, even when not using the ADB server, I minimized the number of ADB commands sent so that only 1 or2 commands are sent per update, depending on whether get_sources is true/false. So I'm going to (finally!) close this ussue