Netflix (and other apps) on Fire TV breaks media lights sync with 500 error

[jaseonline]

Whenever I load up the Netflix app on my Fire TV, it breaks the media lights sync. Automation reverts back to normal when I close down Netflix and return to the Home screen. Here are the logs:

2021-02-10 01:07:59.417980 INFO media_lights_sync: New picture received from 'media_player.fire_tv_living_room' (entity_picture)
2021-02-10 01:08:05.883659 INFO media_lights_sync: Set 'light.tv_hue_lightstrip' light:
{'rgb_color': [63, 130, 191]}
2021-02-10 01:08:09.390403 INFO media_lights_sync: New picture received from 'media_player.fire_tv_living_room' (entity_picture)
2021-02-10 01:08:09.516680 WARNING media_lights_sync: ------------------------------------------------------------
2021-02-10 01:08:09.517952 WARNING media_lights_sync: Unexpected error in worker for App media_lights_sync:
2021-02-10 01:08:09.519060 WARNING media_lights_sync: Worker Ags: {'id': 'bea241ddaaf644589ad0f516c38753a5', 'name': 'media_lights_sync', 'objectid': 'd7b5cbc4f111482f9331862d20a2f648', 'type': 'state', 'function': <bound method MediaLightsSync.change_lights_color of <media_lights_sync.MediaLightsSync object at 0x7f82c907f0>>, 'attribute': 'entity_picture', 'entity': 'media_player.fire_tv_living_room', 'new_state': '/api/media_player_proxy/media_player.fire_tv_living_room?token=73def4c443ecf1dfd8e0ae3a87336cdc47b1ae9ac2189b4546727db90c353da3&cache=1612919289.360683', 'old_state': '/api/media_player_proxy/media_player.fire_tv_living_room?token=73def4c443ecf1dfd8e0ae3a87336cdc47b1ae9ac2189b4546727db90c353da3&cache=1612919279.388156', 'pin_app': True, 'pin_thread': 1, 'kwargs': {'attribute': 'entity_picture', '__thread_id': 'thread-1'}}
2021-02-10 01:08:09.520043 WARNING media_lights_sync: ------------------------------------------------------------
2021-02-10 01:08:09.521903 WARNING media_lights_sync: Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/appdaemon/threading.py", line 900, in worker
    funcref(
  File "/config/appdaemon/apps/ad-media-lights-sync/media_lights_sync.py", line 52, in change_lights_color
    rgb_colors = self.get_colors(self.format_url(new_url, entity, attribute))
  File "/config/appdaemon/apps/ad-media-lights-sync/media_lights_sync.py", line 103, in get_colors
    fd = urlopen(url)
  File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
    response = self.parent.error(
  File "/usr/lib/python3.8/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 500: Internal Server Error
2021-02-10 01:08:09.522998 WARNING media_lights_sync: ------------------------------------------------------------

You can see it changes to RGB 63, 130, 191 on the Home screen and then breaks as soon as I load up Netflix. Other Apps are fine and I can see the entity_picture does not change when Netflix ids the source.

Home screen: entity_picture: /api/media_player_proxy/media_player.fire_tv_living_room?token=73def4c443ecf1dfd8e0ae3a87336cdc47b1ae9ac2189b4546727db90c353da3&cache=1612919819.383837

Netflix: entity_picture: /api/media_player_proxy/media_player.fire_tv_living_room?token=73def4c443ecf1dfd8e0ae3a87336cdc47b1ae9ac2189b4546727db90c353da3&cache=1612919929.417483

configuration.yaml

media_player:
  - platform: androidtv
    name: Fire TV Living Room
    device_class: firetv
    host: 192.168.1.37
    adb_server_ip: 127.0.0.1
    adb_server_port: 5037
    exclude_unnamed_apps: true

appdaemon.yaml (Located Redacted)

---
secrets: /config/secrets.yaml
appdaemon:
  latitude: 0.0
  longitude: 0.0
  elevation: 0
  time_zone: Europe/London
  plugins:
    HASS:
      type: hass
http:
  url: http://127.0.0.1:5050
admin:
api:
hadashboard:

apps.yaml (HA URL Redacted)

---
media_lights_sync:
  module: media_lights_sync
  class: MediaLightsSync
  media_player: media_player.fire_tv_living_room
  lights:
    - light.tv_hue_lightstrip
  ha_url: http://192.168.1.XX:8123
  reset_lights_after: false
  use_saturated_colors: true
  use_current_brightness: true
  condition:
    entity: media_player.samsung_ue55hu7500
    state: "on"

I also use ADB - Android Debug Bridge to connect my Fire TV (IP Redacted) and this works fine based on the logs:

[08:57:41] INFO: Starting the Android Debug Bridge server...
[08:57:41] INFO: Connecting to device: 192.168.1.37...
connected to 192.168.1.XX:5555
[08:59:11] INFO: Connecting to device: 192.168.1.37...
already connected to 192.168.1.XX:5555

[ericmatte]

HTTP Error 500: Internal Server Error means that the server from where the image is fetched returned an error; which is not currently handled by this app.

Are you able to manually fetch the thumbnail while being on Netflix? and does the media_player in HA show the proper in that case?

[jaseonline]

I was looking at this further last night and it seems that some paid for Apps, such as Netflix, won't allow access to entity_picture with the token provided. This is shown when I try and get the image via a web browser, see attached. Works fine on FireTV Home but then an error is returned when I load Netflix.

Have been looking at python-androidtv to see if there is anything I could repurpose there.

[ericmatte]

You are right, it may be more of an issue related to python-androidtv. Maybe you could open an issue there and link this one.

Also some media_players also send a entity_picture which uses a remote CDN without any token. Do you have something like this?

[jaseonline]

@ericmatte I have not installed python-androidtv, so that won't be an issue. I am just using the standard HA androidtv for my media player. I have a fairly basic HA Install on a Pi4, so no CDN. Do you expect all apps on a Firestick TV 4K Ultra to support 'entity_picture'.

[ericmatte]

Oh ok 👍 I don't have a Firestick, but it is similar when I test with a Chromecast with Google TV: Some time I have an entity_picture available, sometime I don't. Except I did not see any error 500.

That said, I could probably handle the error on this app and silently ignore it, but that won't change the fact that it won't be able to change the lights color for that case.

[jaseonline]

That makes sense, @ericmatte - you can't display what you cannot access!

I will see about options and update here if I find a solution to access Netflix entity_picture with the token provided.

[ericmatte]

@jaseonline great, let me know here if you find something.

In the meantime, I've added a try/except to catch http request errors and send a clearer message in the logs: 5c86d8608d71b07d848955833eb5ceb72db999b9

Cheers

[jaseonline]

Thanks @ericmatte - good timing, I am spending my Sat night seeing if I can find a solution! I really do like what you have done with this integration.

[ericmatte]

I've just released version v0.7.0 with exception handler 🎉

[jaseonline]

@ericmatte I don't think the exception handling in v0.7.0 works correctly as now I get an error on load for my Fire TV. I do not have any other apps running via AppDaemon and is starts without error:

.
.
.
2021-02-22 21:08:51.725851 INFO AppDaemon: App 'media_lights_sync' added
2021-02-22 21:08:51.728707 INFO AppDaemon: Found 1 total apps
2021-02-22 21:08:51.730313 INFO AppDaemon: Starting Apps with 1 workers and 1 pins
2021-02-22 21:08:51.732747 INFO AppDaemon: Running on port 5050
2021-02-22 21:08:51.804776 INFO HASS: Evaluating startup conditions
2021-02-22 21:08:51.830006 INFO HASS: Startup condition met: hass state=RUNNING
2021-02-22 21:08:51.831183 INFO HASS: All startup conditions met
2021-02-22 21:08:51.878401 INFO AppDaemon: Got initial state from namespace default
2021-02-22 21:08:53.756757 INFO AppDaemon: Scheduler running in realtime
2021-02-22 21:08:53.797530 INFO AppDaemon: Adding /config/appdaemon/apps to module import path
2021-02-22 21:08:53.798942 INFO AppDaemon: Adding /config/appdaemon/apps/ad-media-lights-sync to module import path
2021-02-22 21:08:53.803360 INFO AppDaemon: Loading App Module: /config/appdaemon/apps/ad-media-lights-sync/media_lights_sync.py
2021-02-22 21:08:53.862036 INFO AppDaemon: Initializing app media_lights_sync using class MediaLightsSync from module media_lights_sync
2021-02-22 21:08:54.072214 INFO media_lights_sync: Using default (MedianCut) quantization method
2021-02-22 21:08:54.074755 INFO media_lights_sync: Listening for picture changes on 'media_player.fire_tv_living_room'
2021-02-22 21:08:54.083922 INFO AppDaemon: App initialization complete

As soon as I start Fire TV and New picture received from 'media_player.fire_tv_living_room' (entity_picture) is received, the following error message occurs:

2021-02-22 21:09:30.311110 WARNING media_lights_sync: ------------------------------------------------------------
2021-02-22 21:09:30.312142 WARNING media_lights_sync: Unexpected error in worker for App media_lights_sync:
2021-02-22 21:09:30.313061 WARNING media_lights_sync: Worker Ags: {'id': '0bbd3dabb3f3490897699c3c629fa457', 'name': 'media_lights_sync', 'objectid': 'eda18367f1e941829417c751405c73a2', 'type': 'state', 'function': <bound method MediaLightsSync.change_lights_color of <media_lights_sync.MediaLightsSync object at 0x7f7f854b50>>, 'attribute': 'entity_picture', 'entity': 'media_player.fire_tv_living_room', 'new_state': '/api/media_player_proxy/media_player.fire_tv_living_room?token=36acae52cf4967326d22239587fb79432f886c803f5956e217402fa2b186ce65&cache=1614028163.912325', 'old_state': '/api/media_player_proxy/media_player.fire_tv_living_room?token=36acae52cf4967326d22239587fb79432f886c803f5956e217402fa2b186ce65&cache=1614028153.938489', 'pin_app': True, 'pin_thread': 0, 'kwargs': {'attribute': 'entity_picture', '__thread_id': 'thread-0'}}
2021-02-22 21:09:30.313901 WARNING media_lights_sync: ------------------------------------------------------------
2021-02-22 21:09:30.318557 WARNING media_lights_sync: Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/appdaemon/threading.py", line 900, in worker
    funcref(
  File "/config/appdaemon/apps/ad-media-lights-sync/media_lights_sync.py", line 56, in change_lights_color
    rgb_colors = self.get_colors(url)
  File "/config/appdaemon/apps/ad-media-lights-sync/media_lights_sync.py", line 115, in get_colors
    palette = im.quantize(colors=len(self.lights), method=self.quantization_method).getpalette()
  File "/usr/lib/python3.8/site-packages/PIL/Image.py", line 1081, in quantize
    raise ValueError(
ValueError: Fast Octree (method == 2) and libimagequant (method == 3) are the only valid methods for quantizing RGBA images

The image can be viewed fine in my web browser via the api URL.

[ericmatte]

Huh, this is an issue related to a new feature I've also added in v0.7.0.

Thanks for letting me know; I will create a new issue.

[ericmatte]

A little more info on that: it does it for me also using the android_tv integration with screencap: true:

Like this comment https://github.com/home-assistant/core/issues/34018#issuecomment-623038453 says, it seems to be a problem with Home Assistant when the image updates on a media_player at the same time as your fetching it.

It also seems to happen more often when there is multiple thing requesting the picture (ie.: multiple media_player lovelace cards + this app). In my case, if I close all my Home Assistant pages in my browser, this app little to no error 500.