search

haijeploeg / excludarr

Excludarr manages your libraries in Radarr/Sonarr. It keeps track of your library and checks if your movies and series are also available on a configured streaming provider. It can exclude the movies and series that are available on a configured streaming provider. But it can also re-add movies and series if they are not streaming anymore.
MIT License
204 stars 12 forks source link

Error when running sonarr excludarr #65

Open jcandresen opened 2 years ago

jcandresen commented 2 years ago

Hey there,

Excludarr was working great until this morning and all of a sudden. I'm running it on WSL Ubuntu in WSL.

jonas@StealthyPC:~$ excludarr --debug sonarr exclude -a not-monitored -d -e --progress [2022-04-30 13:42:32] - Starting Excludarr v1.0.6 [2022-04-30 13:42:32] - Got sonarr as subcommand [2022-04-30 13:42:32] - Reading configuration file [2022-04-30 13:42:32] - Determining which configfile to use [2022-04-30 13:42:32] - Configfile to use: /etc/excludarr/excludarr.yml [2022-04-30 13:42:32] - Reading configfile: /etc/excludarr/excludarr.yml [2022-04-30 13:42:32] - Read the following configuration: {'general': {'fast_search': True, 'locale': 'en_CA', 'providers': ['netflix', 'amazon prime video', 'disney plus']}, 'radarr': {'url': 'http://192.168.0.20:37878/', 'api_key': '', 'verify_ssl': False, 'exclude': ["Don't Look Up"]}, 'sonarr': {'url': 'http://192.168.0.20:38989/', 'api_key': '', 'verify_ssl': False, 'exclude': ['']}} [2022-04-30 13:42:32] - Got exclude as subcommand [2022-04-30 13:42:32] - Got CLI values for -p, --provider option: [2022-04-30 13:42:32] - Got CLI values for -l, --locale option: None [2022-04-30 13:42:32] - Got CLI values for -a, --action option: not-monitored [2022-04-30 13:42:32] - Got CLI values for -d, --delete option: True [2022-04-30 13:42:32] - Got CLI values for -e, --exclusion option: True [2022-04-30 13:42:32] - Got CLI values for -y, --yes option: False [2022-04-30 13:42:32] - Got CLI values for --progress option: True [2022-04-30 13:42:32] - Initializing PySonarr [2022-04-30 13:42:32] - Initializing JustWatch API with locale: en_CA [2022-04-30 13:42:32] - Getting all the series from Sonarr [2022-04-30 13:43:09] - Got the following providers: Netflix, Disney Plus, Amazon Prime Video [2022-04-30 13:43:09] - Extreme Cheapskates has IMDB ID: tt2152721 and TVDB_ID: 255277 [2022-04-30 13:43:09] - Processing title: Extreme Cheapskates with Sonarr ID: 7 and IMDB ID: tt2152721 [2022-04-30 13:43:09] - Query JustWatch API with title: Extreme Cheapskates [2022-04-30 13:43:10] - Querying JustWatch API with ID: 27930 for title: Extreme Cheapskates [2022-04-30 13:43:10] - Got IMDB ID's: ['tt3128214'] from JustWatch API [2022-04-30 13:43:10] - Got TMDB ID's: [44722] from JustWatch API [2022-04-30 13:43:10] - Could not find Extreme Cheapskates using IMDB ID: tt2152721 [2022-04-30 13:43:10] - Giles and Sue Live the Good Life has IMDB ID: tt1851401 and TVDB_ID: 204291 [2022-04-30 13:43:10] - Processing title: Giles and Sue Live the Good Life with Sonarr ID: 9 and IMDB ID: tt1851401 [2022-04-30 13:43:10] - Query JustWatch API with title: Giles and Sue Live the Good Life [2022-04-30 13:43:10] - Could not find Giles and Sue Live the Good Life using IMDB ID: tt1851401 [2022-04-30 13:43:10] - The Supersizers has IMDB ID: tt1242512 and TVDB_ID: 82151 [2022-04-30 13:43:10] - Processing title: The Supersizers with Sonarr ID: 23 and IMDB ID: tt1242512 [2022-04-30 13:43:10] - Query JustWatch API with title: The Supersizers [2022-04-30 13:43:11] - Querying JustWatch API with ID: 20472 for title: The Supersizers [2022-04-30 13:43:11] - Got IMDB ID's: ['tt1220617'] from JustWatch API [2022-04-30 13:43:11] - Got TMDB ID's: [242022, 7317, 69798] from JustWatch API [2022-04-30 13:43:11] - Querying JustWatch API with ID: 21218 for title: The Supersizers [2022-04-30 13:43:12] - Got IMDB ID's: ['tt1196946'] from JustWatch API [2022-04-30 13:43:12] - Got TMDB ID's: [5920] from JustWatch API [2022-04-30 13:43:12] - Querying JustWatch API with ID: 22099 for title: The Supersizers [2022-04-30 13:43:12] - Got IMDB ID's: ['tt0976192'] from JustWatch API [2022-04-30 13:43:12] - Got TMDB ID's: [3854] from JustWatch API [2022-04-30 13:43:12] - Could not find The Supersizers using IMDB ID: tt1242512 [2022-04-30 13:43:28] - Are You Being Served? has IMDB ID: tt0068040 and TVDB_ID: 77879 [2022-04-30 13:43:28] - Processing title: Are You Being Served? with Sonarr ID: 28 and IMDB ID: tt0068040 [2022-04-30 13:43:28] - Query JustWatch API with title: Are You Being Served? [2022-04-30 13:43:29] - Could not find Are You Being Served? using IMDB ID: tt0068040 [2022-04-30 13:43:29] - Forever Summer With Nigella has IMDB ID: tt0341783 and TVDB_ID: 81192 [2022-04-30 13:43:29] - Processing title: Forever Summer With Nigella with Sonarr ID: 59 and IMDB ID: tt0341783 [2022-04-30 13:43:29] - Query JustWatch API with title: Forever Summer With Nigella [2022-04-30 13:43:29] - Could not find Forever Summer With Nigella using IMDB ID: tt0341783 [2022-04-30 13:43:29] - I May Destroy You has IMDB ID: tt11204260 and TVDB_ID: 371798 [2022-04-30 13:43:29] - Processing title: I May Destroy You with Sonarr ID: 61 and IMDB ID: tt11204260 [2022-04-30 13:43:29] - Query JustWatch API with title: I May Destroy You [2022-04-30 13:43:29] - Querying JustWatch API with ID: 223540 for title: I May Destroy You [2022-04-30 13:43:29] - Got IMDB ID's: ['tt11302324'] from JustWatch API [2022-04-30 13:43:29] - Got TMDB ID's: [96723] from JustWatch API [2022-04-30 13:43:29] - Could not find I May Destroy You using IMDB ID: tt11204260 Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 665, in urlopen httplib_response = self._make_request( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 421, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 416, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.8/http/client.py", line 1348, in getresponse response.begin() File "/usr/lib/python3.8/http/client.py", line 316, in begin version, status, reason = self._read_status() File "/usr/lib/python3.8/http/client.py", line 285, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/jonas/.local/lib/python3.8/site-packages/requests/adapters.py", line 440, in send resp = conn.urlopen( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 719, in urlopen retries = retries.increment( File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 400, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/lib/python3/dist-packages/six.py", line 702, in reraise raise value.with_traceback(tb) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 665, in urlopen httplib_response = self._make_request( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 421, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 416, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.8/http/client.py", line 1348, in getresponse response.begin() File "/usr/lib/python3.8/http/client.py", line 316, in begin version, status, reason = self._read_status() File "/usr/lib/python3.8/http/client.py", line 285, in _read_status raise RemoteDisconnected("Remote end closed connection without" urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/jonas/.local/bin/excludarr", line 8, in sys.exit(cli()) File "/home/jonas/.local/lib/python3.8/site-packages/excludarr/main.py", line 68, in cli app(prog_name="excludarr") File "/home/jonas/.local/lib/python3.8/site-packages/typer/main.py", line 214, in call return get_command(self)(*args, kwargs) File "/home/jonas/.local/lib/python3.8/site-packages/click/core.py", line 1130, in call return self.main(args, kwargs) File "/home/jonas/.local/lib/python3.8/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/home/jonas/.local/lib/python3.8/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/jonas/.local/lib/python3.8/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/jonas/.local/lib/python3.8/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, ctx.params) File "/home/jonas/.local/lib/python3.8/site-packages/click/core.py", line 760, in invoke return __callback(args, kwargs) File "/home/jonas/.local/lib/python3.8/site-packages/typer/main.py", line 500, in wrapper return callback(use_params) # type: ignore File "/home/jonas/.local/lib/python3.8/site-packages/excludarr/commands/sonarr.py", line 69, in exclude series_to_exclude = sonarr.get_series_to_exclude( File "/home/jonas/.local/lib/python3.8/site-packages/excludarr/core/sonarr_actions.py", line 182, in get_series_to_exclude episodes = self.sonarr_client.episode.get_episodes_of_serie(sonarr_id) File "/home/jonas/.local/lib/python3.8/site-packages/excludarr/modules/pysonarr/v3/episode.py", line 14, in get_episodes_of_serie return self.client.http_get(self.episode_path, params=params) File "/home/jonas/.local/lib/python3.8/site-packages/excludarr/modules/pysonarr/init.py", line 74, in http_get return self.http_request("get", path, params=params) File "/home/jonas/.local/lib/python3.8/site-packages/excludarr/modules/pysonarr/init.py", line 62, in http_request result = self.session.send(prepped) File "/home/jonas/.local/lib/python3.8/site-packages/requests/sessions.py", line 645, in send r = adapter.send(request, kwargs) File "/home/jonas/.local/lib/python3.8/site-packages/requests/adapters.py", line 501, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) jonas@StealthyPC:~$

I've tried reinstalling WSL and Excludarr with no luck so I was hoping I could hop on in here and see if there's anything I could do.

haijeploeg commented 2 years ago

Sorry I totally missed this issue!

Looking at the errors it seems that your sonarr instance is dropping the connection somehow. However, the libraries used to talk to Sonarr are replaces by PyArr in version v1.0.7 which is just released now! Can you test this with the new version as well?

jcandresen commented 2 years ago

i accidentally fatfingered the close.

I upgraded excludarr and ran it again, it runs a lot longer than before so that's definitely a positive, I got the following: 

jonas@StealthyPC:~$ excludarr --debug sonarr exclude --action delete --delete-files --exclusion --progress
[2022-05-20 22:24:15] - Starting Excludarr v1.0.7
[2022-05-20 22:24:15] - Got sonarr as subcommand
[2022-05-20 22:24:15] - Reading configuration file
[2022-05-20 22:24:15] - Determining which configfile to use
[2022-05-20 22:24:15] - Configfile to use: /etc/excludarr/excludarr.yml
[2022-05-20 22:24:15] - Reading configfile: /etc/excludarr/excludarr.yml
[2022-05-20 22:24:15] - Read the following configuration: {'general': {'fast_search': False, 'locale': 'en_CA', 'providers': ['netflix', 'amazon prime video', 'disney plus']}, 'radarr': {'url': 'http://192.168.0.20:37878/', 'api_key': '<REDACTED>', 'verify_ssl': False, 'exclude': ["Don't Look Up"]}, 'sonarr': {'url': 'http://192.168.0.20:38989/', 'api_key': '<REDACTED>', 'verify_ssl': False, 'exclude': ['']}}
[2022-05-20 22:24:15] - Got exclude as subcommand
[2022-05-20 22:24:15] - Got CLI values for -p, --provider option:
[2022-05-20 22:24:15] - Got CLI values for -l, --locale option: None
[2022-05-20 22:24:15] - Got CLI values for -a, --action option: delete
[2022-05-20 22:24:15] - Got CLI values for -d, --delete option: True
[2022-05-20 22:24:15] - Got CLI values for -e, --exclusion option: True
[2022-05-20 22:24:15] - Got CLI values for -y, --yes option: False
[2022-05-20 22:24:15] - Got CLI values for --progress option: True
[2022-05-20 22:24:15] - Initializing PySonarr
[2022-05-20 22:24:15] - Initializing JustWatch API with locale: en_CA
[2022-05-20 22:24:16] - Getting all the series from Sonarr
[2022-05-20 22:24:32] - Got the following providers: Netflix, Disney Plus, Amazon Prime Video
[2022-05-20 22:24:32] - Extreme Cheapskates has IMDB ID: tt2152721 and TVDB_ID: 255277
[2022-05-20 22:24:32] - Processing title: Extreme Cheapskates with Sonarr ID: 7 and IMDB ID: tt2152721

<-------- normal stuff just going through 60%-70% of my library--------------> 

[2022-05-20 22:49:07] - Processing season Season 6 of Animal Kingdom (2016)
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 665, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 421, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 416, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1348, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 316, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 285, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jonas/.local/lib/python3.8/site-packages/requests/adapters.py", line 440, in send
    resp = conn.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 719, in urlopen
    retries = retries.increment(
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 400, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/lib/python3/dist-packages/six.py", line 702, in reraise
    raise value.with_traceback(tb)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 665, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 421, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 416, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1348, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 316, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 285, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jonas/.local/bin/excludarr", line 8, in <module>
    sys.exit(cli())
  File "/home/jonas/.local/lib/python3.8/site-packages/excludarr/main.py", line 68, in cli
    app(prog_name="excludarr")
  File "/home/jonas/.local/lib/python3.8/site-packages/typer/main.py", line 214, in __call__
    return get_command(self)(*args, **kwargs)
  File "/home/jonas/.local/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/jonas/.local/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/jonas/.local/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/jonas/.local/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/jonas/.local/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/jonas/.local/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/jonas/.local/lib/python3.8/site-packages/typer/main.py", line 500, in wrapper
    return callback(**use_params)  # type: ignore
  File "/home/jonas/.local/lib/python3.8/site-packages/excludarr/commands/sonarr.py", line 67, in exclude
    series_to_exclude = sonarr.get_series_to_exclude(
  File "/home/jonas/.local/lib/python3.8/site-packages/excludarr/core/sonarr_actions.py", line 191, in get_series_to_exclude
    episodes = self.sonarr_client.get_episodes_by_series_id(sonarr_id)
  File "/home/jonas/.local/lib/python3.8/site-packages/pyarr/sonarr.py", line 120, in get_episodes_by_series_id
    return self.request_get(path, self.ver_uri, params=params)
  File "/home/jonas/.local/lib/python3.8/site-packages/pyarr/request_handler.py", line 70, in request_get
    res = self.session.get(
  File "/home/jonas/.local/lib/python3.8/site-packages/requests/sessions.py", line 542, in get
    return self.request('GET', url, **kwargs)
  File "/home/jonas/.local/lib/python3.8/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/jonas/.local/lib/python3.8/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/home/jonas/.local/lib/python3.8/site-packages/requests/adapters.py", line 501, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
haijeploeg commented 2 years ago

Hi, I tried debugging this, but I cannot reproduce this. It seems that JustWatch is blocking you or cannot handle the requests anymore after a certain amount of time. How big is your library? I have seen this behaviour before at huge libraries.

jcandresen commented 2 years ago

It’s definitely chunky, I think it’s 200 shows but it needs to be summer cleaned.

Is there a way to add a wait parameter to add seconds between just watch queries ? I don’t mind it taking long if it needs to.

On Wed, May 25, 2022 at 00:30 Haije Ploeg @.***> wrote:

Hi, I tried debugging this, but I cannot reproduce this. It seems that JustWatch is blocking you or cannot handle the requests anymore after a certain amount of time. How big is your library? I have seen this behaviour before at huge libraries.

— Reply to this email directly, view it on GitHub https://github.com/haijeploeg/excludarr/issues/65#issuecomment-1136790293, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFQD2BDUFOFDEG3FIFTF3JDVLXCI3ANCNFSM5UYYSD6A . You are receiving this because you modified the open/close state.Message ID: @.***>

jcandresen commented 2 years ago

Trimmed the fat

On Wed, May 25, 2022 at 07:45 Jonas Andresen @.***> wrote:

It’s definitely chunky, I think it’s 200 shows but it needs to be summer cleaned.

Is there a way to add a wait parameter to add seconds between just watch queries ? I don’t mind it taking long if it needs to.

On Wed, May 25, 2022 at 00:30 Haije Ploeg @.***> wrote:

Hi, I tried debugging this, but I cannot reproduce this. It seems that JustWatch is blocking you or cannot handle the requests anymore after a certain amount of time. How big is your library? I have seen this behaviour before at huge libraries.

— Reply to this email directly, view it on GitHub https://github.com/haijeploeg/excludarr/issues/65#issuecomment-1136790293, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFQD2BDUFOFDEG3FIFTF3JDVLXCI3ANCNFSM5UYYSD6A . You are receiving this because you modified the open/close state.Message ID: @.***>