search

RemiRigal / Plex-Auto-Languages

Automated language selection for Plex TV Shows !
MIT License
444 stars 17 forks source link

[Enhancement] Add processing thread #51

Open RemiRigal opened 2 years ago

RemiRigal commented 2 years ago

The websocket thread listening for events is currently blocked during the requests to the Plex server, which can lead to loss of messages when the number of events and/or users is large.

The solution to tackle this issue would be to improve the way messages are handled by creating a new thread dedicated to HTTP requests.

RemiRigal commented 2 years ago

@BEisem I've implemented and merged (#53) that adds a processing thread so that every message is queued and processed sequentially instead of blocking the websocket thread, it should solve your issue of play sessions not being detected.

I will be testing this new feature as well as the user cache (from issue #46) for a few days before releasing a new version. If you want to validate the changes with me you can try the remirigal/plex-auto-languages:master docker image.

BEisem commented 2 years ago

Hi @RemiRigal, thanks for this enhancement! I think it will be a great idea once implemented. I've updated, and so far I'm getting errors like the following, repeatedly in the logs:

2022-07-19 11:23:31,050 [ERROR] Connection to the Plex server failed for user 'username'

BEisem commented 2 years ago

I also just got the following error:

2022-07-19 11:40:17,872 [ERROR] Unable to process status
Traceback (most recent call last):
  File "/app/plex_auto_languages/plex_alert_handler.py", line 61, in _process_alerts
    alert.process(self._plex)
  File "/app/plex_auto_languages/alerts/status.py", line 44, in process
    plex.process_new_or_updated_episode(item.key, EventType.NEW_EPISODE, True)
  File "/app/plex_auto_languages/plex_server.py", line 223, in process_new_or_updated_episode
    user_item.reload()
  File "/usr/local/lib/python3.8/site-packages/plexapi/base.py", line 348, in reload
    return self._reload(key=key, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/plexapi/base.py", line 357, in _reload
    data = self._server.query(key)
  File "/usr/local/lib/python3.8/site-packages/plexapi/server.py", line 704, in query
    raise NotFound(message)
plexapi.exceptions.NotFound: (404) not_found; http://localhost:32400/library/metadata/723815?checkFiles=1&includeAllConcerts=1&includeBandwidths=1&includeChapters=1&includeChildren=1&includeConcerts=1&includeExternalMedia=1&includeExtras=1&includeFields=thumbBlurHash%2CartBlurHash&includeGeolocation=1&includeLoudnessRamps=1&includeMarkers=1&includeOnDeck=1&includePopularLeaves=1&includePreferences=1&includeRelated=1&includeRelatedCount=1&includeReviews=1&includeStations=1 <html><head><title>Not Found</title></head><body><h1>404 Not Found</h1></body></html>
BEisem commented 2 years ago

It seems the new user caching has completely broken PAL's ability to connect with Plex, as of now I am unable to run it because I constantly get the errors shown https://github.com/RemiRigal/Plex-Auto-Languages/issues/51#issuecomment-1189212021. here.

RemiRigal commented 2 years ago

The errors you are getting are indeed related to the user cache.

2022-07-19 11:23:31,050 [ERROR] Connection to the Plex server failed for user 'username'

Even though the user is cached, the authentication still has to be performed for the user. I'll try to cache the user's token as well as its session, this might do the trick. In the mean time you can go back to the latest image if you need something functional.

BEisem commented 2 years ago

Just wanted to check in, any luck with this?

RemiRigal commented 2 years ago

Sorry for the delay, the master version should now be working. I've improved the cache in order to reduce the number of calls to the Plex API. Can you try it and le me know ? Thank you.

BEisem commented 2 years ago

Hey @RemiRigal, thanks for the update! It looks like maybe the new version is only recognizing Plex items played by the master user, but none of the shared users. I've been watching the logs, and I only see language updates for items I watch. But I can also see from the Plex UI that other people are watching TV episodes. Those do not show up in the PAL logs. Is there a setting I need to set to allow for all users? Or did the changes you made somehow only affect the main user?

BEisem commented 2 years ago

Correction, it does seem to be working for other users. However I am still getting errors like this fairly regularly:

2022-08-24 10:57:51,029 [INFO] Parsing config file '/config/config.yaml'
2022-08-24 10:57:51,033 [INFO] The provided configuration has been successfully validated
2022-08-24 10:57:51,038 [INFO] Starting scheduler
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 72, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/local/lib/python3.8/socket.py", line 918, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Try again

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 358, in connect
    self.sock = conn = self._new_conn()
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f6cb935d5e0>: Failed to establish a new connection: [Errno -3] Try again

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='plex.tv', port=443): Max retries exceeded with url: /users/account (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f6cb935d5e0>: Failed to establish a new connection: [Errno -3] Try again'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 116, in <module>
    plex_auto_languages.start()
  File "main.py", line 68, in start
    self.init()
  File "main.py", line 45, in init
    self.plex = PlexServer(self.config.get("plex.url"), self.config.get("plex.token"), self.notifier, self.config)
  File "/app/plex_auto_languages/plex_server.py", line 103, in __init__
    self._user = self._get_logged_user()
  File "/app/plex_auto_languages/plex_server.py", line 137, in _get_logged_user
    plex_username = self._plex.myPlexAccount().username
  File "/usr/local/lib/python3.8/site-packages/plexapi/server.py", line 321, in myPlexAccount
    self._myPlexAccount = MyPlexAccount(token=self._token)
  File "/usr/local/lib/python3.8/site-packages/plexapi/myplex.py", line 93, in __init__
    data, initpath = self._signin(username, password, timeout)
  File "/usr/local/lib/python3.8/site-packages/plexapi/myplex.py", line 98, in _signin
    return self.query(self.key), self.key
  File "/usr/local/lib/python3.8/site-packages/plexapi/myplex.py", line 175, in query
    response = method(url, headers=headers, timeout=timeout, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 600, in get
    return self.request("GET", url, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 565, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='plex.tv', port=443): Max retries exceeded with url: /users/account (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f6cb935d5e0>: Failed to establish a new connection: [Errno -3] Try again'))
BEisem commented 2 years ago

Please, are there any updates on this?