Sonarr Scan fails

[DreadDragoon]

Thought it was because of the pip install of excludarr, but went with the docker image to make sure it wasn't a dependency issue.

When I run the job, I get the following error:

|> excludarr <| / #excludarr sonarr exclude -a not-monitored --progress Working... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00 Traceback (most recent call last): File "/usr/local/bin/excludarr", line 33, in sys.exit(load_entry_point('excludarr==1.0.6', 'console_scripts', 'excludarr')()) File "/usr/local/lib/python3.10/site-packages/excludarr-1.0.6-py3.10.egg/excludarr/main.py", line 68, in cli File "/usr/local/lib/python3.10/site-packages/typer/main.py", line 214, in call return get_command(self)(*args, kwargs) File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1128, in call return self.main(args, kwargs) File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1053, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1395, in invoke return ctx.invoke(self.callback, ctx.params) File "/usr/local/lib/python3.10/site-packages/click/core.py", line 754, in invoke return __callback(args, kwargs) File "/usr/local/lib/python3.10/site-packages/typer/main.py", line 500, in wrapper return callback(**use_params) # type: ignore File "/usr/local/lib/python3.10/site-packages/excludarr-1.0.6-py3.10.egg/excludarr/commands/sonarr.py", line 69, in exclude File "/usr/local/lib/python3.10/site-packages/excludarr-1.0.6-py3.10.egg/excludarr/core/sonarr_actions.py", line 281, in get_series_to_exclude KeyError: 'statistics' |> excludarr <| / #

To Reproduce Steps to reproduce the behavior:

1. The exact command that throws an error excludarr sonarr exclude -a not-monitored --progress

2. The name of the movie or serie that is being processed at the time of the error (if applicable) initial scan, so unknown.

3. The settings file used (strip it from sensitive data)

Expected behavior A clear and concise description of what you expected to happen.

I expect it to complete the scan and present a menus

Debug logging To debug the problem further, we need to know exactly what went wrong. Therefor we ask you to run the command in debug mode. You can achieve this by appending the --debug flag after the base command excludarr. e.g. excludarr --debug sonarr exclude -a delete -d -e.

|> excludarr <| / #excludarr --debug sonarr exclude -a not-monitored --progress [2022-01-25 18:59:13] - Starting Excludarr v1.0.6 [2022-01-25 18:59:13] - Got sonarr as subcommand [2022-01-25 18:59:13] - Reading configuration file [2022-01-25 18:59:13] - Determining which configfile to use [2022-01-25 18:59:13] - Configfile to use: /etc/excludarr/excludarr.yml [2022-01-25 18:59:13] - Reading configfile: /etc/excludarr/excludarr.yml [2022-01-25 18:59:13] - Read the following configuration: {'general': {'fast_search': True, 'locale': 'en_US', 'providers': ['netflix', 'amazon prime video']}, 'radarr': {'url': 'http://radarr:7878', 'api_key': '', 'verify_ssl': False, 'exclude': ['']}, 'sonarr': {'url': 'http://sonarr:8989', 'api_key': '', 'verify_ssl': False, 'exclude': ['']}} [2022-01-25 18:59:13] - Got exclude as subcommand [2022-01-25 18:59:13] - Got CLI values for -p, --provider option: [2022-01-25 18:59:13] - Got CLI values for -l, --locale option: None [2022-01-25 18:59:13] - Got CLI values for -a, --action option: not-monitored [2022-01-25 18:59:13] - Got CLI values for -d, --delete option: False [2022-01-25 18:59:13] - Got CLI values for -e, --exclusion option: False [2022-01-25 18:59:13] - Got CLI values for -y, --yes option: False [2022-01-25 18:59:13] - Got CLI values for --progress option: True [2022-01-25 18:59:13] - Initializing PySonarr [2022-01-25 18:59:13] - Initializing JustWatch API with locale: en_US [2022-01-25 18:59:16] - Getting all the series from Sonarr [2022-01-25 18:59:34] - Got the following providers: Netflix, Amazon Prime Video

Traceback (most recent call last): File "/usr/local/bin/excludarr", line 33, in sys.exit(load_entry_point('excludarr==1.0.6', 'console_scripts', 'excludarr')()) File "/usr/local/lib/python3.10/site-packages/excludarr-1.0.6-py3.10.egg/excludarr/main.py", line 68, in cli File "/usr/local/lib/python3.10/site-packages/typer/main.py", line 214, in __call__ return get_command(self)(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1128, in __call__ return self.main(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1053, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1395, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/local/lib/python3.10/site-packages/click/core.py", line 754, in invoke return __callback(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/typer/main.py", line 500, in wrapper return callback(**use_params) # type: ignore File "/usr/local/lib/python3.10/site-packages/excludarr-1.0.6-py3.10.egg/excludarr/commands/sonarr.py", line 69, in exclude File "/usr/local/lib/python3.10/site-packages/excludarr-1.0.6-py3.10.egg/excludarr/core/sonarr_actions.py", line 281, in get_series_to_exclude KeyError: 'statistics'

[haijeploeg]

Hi! Thanks for your bug report. I'm not sure yet what is causing this. What version of Sonarr are you using? You can check this in the interface at: system -> status (in the about tabblad). Then I can try to reproduce this and release a fix.

[DreadDragoon]

3.0.6.1451 linuxserver/sonarr:develop => https://hub.docker.com/layers/linuxserver/sonarr/develop/images/sha256-226244ea897fab5ae3b74d173c446a4e8be47c5d6d44ae44f85a6b6729cd4e0b?context=explore

[haijeploeg]

Hi, I cannot find a way to reproduce this. Do you get this error immediately after the line: [2022-01-25 18:59:34] - Got the following providers: Netflix, Amazon Prime Video? If not can you also put in the rest of the output? Also what series do you have in your library? I need to run the script with your library as well to figure out at which serie is causing the issues (Sonarr is not always consequent with returning information via the API unfortunately).

Also on which OS are you running this?

[DreadDragoon]

hey!

Ok, so "If not can you also put in the rest of the output?" it just lists shows, i'd rather not share that data publicly :) Also what series do you have in your library: A lot, a few hundred.

Sonarr and excludarr are both running inside docker containers on Ubuntu 21.10

I'll update sonarr/excludarr and retest again

[haijeploeg]

Okay, let me be more specific; I only need the name of the show that logs before the error is thrown. Then I know which show is causing the issues, so I can reproduce it. If I cannot reproduce it, I cannot create a proper fix, without knowing the data that Sonarr returns.

[DreadDragoon]

ALl good, updating containers now and verifying theyre the latest and re-running.

It looks like it scans everything (takes about an hour) then fails, I'll get you some log samples

Message ID: @.***>

[DreadDragoon]

Looks like it times out, but I cant see if its to sonarr, or JustWatch

[2022-02-07 14:46:03] - The Vampire Diaries S8E15 is streaming on Netflix
[2022-02-07 14:46:03] - The Vampire Diaries S8E16 is streaming on Netflix
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.10/http/client.py", line 1374, in getresponse
    response.begin()
  File "/usr/local/lib/python3.10/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.10/http/client.py", line 279, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/local/lib/python3.10/socket.py", line 705, in readinto
    return self._sock.recv_into(b)
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 440, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 785, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.10/site-packages/urllib3/packages/six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.10/http/client.py", line 1374, in getresponse
    response.begin()
  File "/usr/local/lib/python3.10/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.10/http/client.py", line 279, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/local/lib/python3.10/socket.py", line 705, in readinto
    return self._sock.recv_into(b)
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/excludarr", line 33, in <module>
    sys.exit(load_entry_point('excludarr==1.0.6', 'console_scripts', 'excludarr')())
  File "/usr/local/lib/python3.10/site-packages/excludarr-1.0.6-py3.10.egg/excludarr/main.py", line 68, in cli
  File "/usr/local/lib/python3.10/site-packages/typer/main.py", line 214, in __call__
    return get_command(self)(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/typer/main.py", line 500, in wrapper
    return callback(**use_params)  # type: ignore
  File "/usr/local/lib/python3.10/site-packages/excludarr-1.0.6-py3.10.egg/excludarr/commands/sonarr.py", line 69, in exclude
  File "/usr/local/lib/python3.10/site-packages/excludarr-1.0.6-py3.10.egg/excludarr/core/sonarr_actions.py", line 182, in get_series_to_exclude
  File "/usr/local/lib/python3.10/site-packages/excludarr-1.0.6-py3.10.egg/excludarr/modules/pysonarr/v3/episode.py", line 14, in get_episodes_of_serie
  File "/usr/local/lib/python3.10/site-packages/excludarr-1.0.6-py3.10.egg/excludarr/modules/pysonarr/__init__.py", line 74, in http_get
  File "/usr/local/lib/python3.10/site-packages/excludarr-1.0.6-py3.10.egg/excludarr/modules/pysonarr/__init__.py", line 62, in http_request
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 501, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

[DreadDragoon]

trimmed a ton of long running, but no longer watched series to see if it'll fail at a diff point

[DreadDragoon]

So i purged about 20 series, each with 20+ seasons, and it works now. Woo!

Wonder if it was due to library size, or timeout to the justwatch site?

[haijeploeg]

Great! Fine that it worked out. It seems that Excludarr crashed because Sonarr was not responding properly on getting the information about an episode. This was probably caused by overloading Sonarr with API calls. You said you had a few hundred shows in your library? Maybe that was causing issues with requesting all those information in such short time.

I will try to add a delay between the retries it does when encountering an error requested via the API of Sonarr. This will give Sonarr the time to breathe for a while without causing issues in Excludarr.

[DreadDragoon]

Yeah, i nuked 10 shows each with hundreds of episodes, which brought it under 10k episodes and it works fine now.

So the logic holds out :)

[haijeploeg]

Great to hear it all worked out! I will increase the retry delay in the next version!