mhdzumair / MediaFusion

Universal Add-On for Stremio & Kodi
https://mhdzumair.github.io/MediaFusion/
MIT License
349 stars 41 forks source link

Selfhosted Mediafusion Errors out for Series Search - with Custom Torznab based prowlarr Indexer (bitmagnet) #285

Closed kiskey closed 2 months ago

kiskey commented 2 months ago

Hi Team,

Thanks for developing the wonderful addon and sharing it to the users to self-host and use. I really thank and appreciate your efforts.

I have been using the self hosted bitmagnet torrent as my indexer configured with prowlarr. The integration works pretty fast and it works flawlessly for movies.

But whenever series is searched , it looks like mediafusion server errors out in the background with timing out on the front end with no results.

It errors out at line 682 in file Prowlarr.py where there is Year comparison .

ERROR LOG:

**

INFO::12-Sep-24 19:34:33 - Found 15 streams for The Umbrella Academy (4) (4) with IMDb ID tt1312171

INFO::12-Sep-24 19:34:33 - Processing 10 streams for The Umbrella Academy (4) (4) with IMDb ID

ERROR::12-Sep-24 19:34:35 - Unexpected error during batch processing: '<' not supported between instances of 'NoneType' and 'int'

Traceback (most recent call last):

File "/mediafusion/utils/network.py", line 60, in call

raise e  # Reraise the exception to handle it outside

^^^^^^^

File "/mediafusion/utils/network.py", line 47, in call

result = await func(*args, **kwargs)

         ^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/mediafusion/scrapers/prowlarr.py", line 682, in parse_and_store_stream

if max_similarity_ratio < 85 or parsed_data.get("year") < year:

                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

TypeError: '<' not supported between instances of 'NoneType' and 'int'

ERROR::12-Sep-24 19:34:35 - Unexpected error during batch processing: '<' not supported between instances of 'NoneType' and 'int'

Traceback (most recent call last):

File "/mediafusion/utils/network.py", line 60, in call

raise e  # Reraise the exception to handle it outside

^^^^^^^

File "/mediafusion/utils/network.py", line 47, in call

result = await func(*args, **kwargs)

         ^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/mediafusion/scrapers/prowlarr.py", line 682, in parse_and_store_stream

if max_similarity_ratio < 85 or parsed_data.get("year") < year:

                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

TypeError: '<' not supported between instances of 'NoneType' and 'int'

ERROR::12-Sep-24 19:34:35 - Unexpected error during batch processing: '<' not supported between instances of 'NoneType' and 'int'

Traceback (most recent call last):

File "/mediafusion/utils/network.py", line 60, in call

raise e  # Reraise the exception to handle it outside

^^^^^^^

File "/mediafusion/utils/network.py", line 47, in call

result = await func(*args, **kwargs)

         ^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/mediafusion/scrapers/prowlarr.py", line 682, in parse_and_store_stream

if max_similarity_ratio < 85 or parsed_data.get("year") < year:

                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

TypeError: '<' not supported between instances of 'NoneType' and 'int'

ERROR::12-Sep-24 19:34:35 - Unexpected error during batch processing: '<' not supported between instances of 'NoneType' and 'int'

Traceback (most recent call last):

File "/mediafusion/utils/network.py", line 60, in call

raise e  # Reraise the exception to handle it outside

^^^^^^^

File "/mediafusion/utils/network.py", line 47, in call

result = await func(*args, **kwargs)

         ^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/mediafusion/scrapers/prowlarr.py", line 682, in parse_and_store_stream

if max_similarity_ratio < 85 or parsed_data.get("year") < year:

                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

TypeError: '<' not supported between instances of 'NoneType' and 'int'

ERROR::12-Sep-24 19:34:35 - Unexpected error during batch processing: '<' not supported between instances of 'NoneType' and 'int'

Traceback (most recent call last):

File "/mediafusion/utils/network.py", line 60, in call

raise e  # Reraise the exception to handle it outside

^^^^^^^

File "/mediafusion/utils/network.py", line 47, in call

result = await func(*args, **kwargs)

         ^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/mediafusion/scrapers/prowlarr.py", line 682, in parse_and_store_stream

if max_similarity_ratio < 85 or parsed_data.get("year") < year:

                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

TypeError: '<' not supported between instances of 'NoneType' and 'int'

INFO::12-Sep-24 19:34:38 - Retrying 5 items due to circuit breaker.

WARNING::12-Sep-24 19:34:48 - Timeout exceeded for operation: scrap_series_streams_from_prowlarr ('tt1312171', 'The Umbrella Academy', ['雨伞学院', 'Академия Амбрелла', 'Az Esernyő Akadémia', 'Skėčio akademija', 'The Umbrella Academy', 'Η ακαδημία της ομπρέλας', 'A Academia Umbrella', 'Академія Амбрелла', '伞学院'], 2019, 4, 4). Skipping.

INFO::12-Sep-24 19:34:48 - X.X.X.X - "GET http://media.mj.lan/***MASKED***/stream/series/tt1312171:4:4.json HTTP/1.1" 200

Front End with No Results: image

Indexer config: scr1

Mediafusion query to prowlarr with Search Result returned from Indexer: 15 results are shown

scr2

The result xml containing results:

scr3

I have also attached xml file containing results , so you can take a look.

series.results.output.log

mhdzumair commented 2 months ago

please test the new improved scraping, if you're using the docker-compose then update the compose file to use "beta" image for mediafusion and dramatiq worker

kiskey commented 2 months ago

Hi @mhdzumair ,

I just tried the beta image,but

The beta image seems to be crashing with module not found error "Ratelimit" .

ModuleNotFoundError: No module named 'ratelimit'

[2024-09-16 03:36:59,067] [PID 1] [MainThread] [dramatiq.MainProcess] [INFO] Dramatiq '1.17.0' is booting up.

[2024-09-16 03:36:58,064] [PID 8] [MainThread] [dramatiq.WorkerProcess(0)] [ERROR] Failed to import module.

Traceback (most recent call last):

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/dramatiq/cli.py", line 393, in worker_process

module, broker = import_broker(args.broker)

                 ^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/dramatiq/cli.py", line 123, in import_broker

module, broker_or_callable = import_object(value)

                             ^^^^^^^^^^^^^^^^^^^^

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/dramatiq/cli.py", line 112, in import_object

module = importlib.import_module(modname)

         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.12/importlib/init.py", line 90, in import_module

return _bootstrap._gcd_import(name[level:], package, level)

       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "", line 1387, in _gcd_import

File "", line 1360, in _find_and_load

File "", line 1331, in _find_and_load_unlocked

File "", line 935, in _load_unlocked

File "", line 995, in exec_module

File "", line 488, in _call_with_frames_removed

File "/mediafusion/api/task.py", line 10, in

from scrapers import tv, imdb_data, trackers, helpers, prowlarr, prowlarr_feed

File "/mediafusion/scrapers/tv.py", line 10, in

from db import schemas, crud

File "/mediafusion/db/crud.py", line 30, in

from scrapers.utils import run_scrapers

File "/mediafusion/scrapers/utils.py", line 6, in

from scrapers.prowlarr import ProwlarrScraper

File "/mediafusion/scrapers/prowlarr.py", line 20, in

from scrapers.base_scraper import BaseScraper

File "/mediafusion/scrapers/base_scraper.py", line 8, in

from ratelimit import limits, sleep_and_retry

ModuleNotFoundError: No module named 'ratelimit'

Thanks, MJ

mhdzumair commented 2 months ago

pushed the new image, make sure to re-pull the beta image or use full tag as mhdzumair/mediafusion:beta@sha256:e5fa4328c1bd516edb6bc6ddf464d4b23f61c9b4f35046f12c9e5f0d1a9e5b3e

kiskey commented 2 months ago

hi @mhdzumair ,

Thanks for fixing the libraries. It works fine now for the series and movies.

But i see the catalog got some updates.So when setting up the new catalog ,

i get "An error occured while encrypting user data "

Simultaneously on the logs, i see this "encrypt-user-data" 422

INFO::16-Sep-24 06:21:20::middleware.py::49 - 192.168.1.130 - "GET http://1.mjlan.duckdns.org/static/css/styles.css HTTP/1.1" 200

INFO::16-Sep-24 06:28:43::middleware.py::49 - 192.168.1.130 - "POST http://1.mjlan.duckdns.org/encrypt-user-data HTTP/1.1" 422

INFO::16-Sep-24 06:28:50::middleware.py::49 - 192.168.1.130 - "POST http://1.mjlan.duckdns.org/encrypt-user-data HTTP/1.1" 422

INFO::16-Sep-24 06:28:53::middleware.py::49 - 192.168.1.130 - "POST http://1.mjlan.duckdns.org/encrypt-user-data HTTP/1.1" 422

INFO::16-Sep-24 06:28:55::middleware.py::49 - 192.168.1.130 - "POST http://1.mjlan.duckdns.org/encrypt-user-data HTTP/1.1" 422

INFO::16-Sep-24 06:30:50::middleware.py::49 - 192.168.1.130 - "POST http://1.mjlan.duckdns.org/encrypt-user-data HTTP/1.1" 422

INFO::16-Sep-24 06:31:13::middleware.py::49 - 192.168.1.130 - "POST http://1.mjlan.duckdns.org/encrypt-user-data HTTP/1.1" 422

mhdzumair commented 2 months ago

@kiskey could you open the browser network tab and check the post entry and check the response message

kiskey commented 2 months ago

@mhdzumair i now started seeing this in log . First i had the old key from before which i used. Which started through the same error. Then i generated openssl random base 64 key and used it. Then i see the below errors in log .

File "/mediafusion/api/middleware.py", line 90, in dispatch

return await call_next(request)

       ^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/starlette/middleware/base.py", line 163, in call_next

raise app_exc

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/starlette/middleware/base.py", line 149, in coro

await self.app(scope, receive_or_disconnect, send_no_error)

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/starlette/middleware/base.py", line 185, in call

with collapse_excgroups():

     ^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.12/contextlib.py", line 158, in exit

self.gen.throw(value)

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/starlette/_utils.py", line 83, in collapse_excgroups

raise exc

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/starlette/middleware/base.py", line 187, in call

response = await self.dispatch_func(request, call_next)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/mediafusion/api/main.py", line 102, in add_cors_header

response = await call_next(request)

           ^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/starlette/middleware/base.py", line 163, in call_next

raise app_exc

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/starlette/middleware/base.py", line 149, in coro

await self.app(scope, receive_or_disconnect, send_no_error)

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/starlette/middleware/cors.py", line 93, in call

await self.simple_response(scope, receive, send, request_headers=headers)

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/starlette/middleware/cors.py", line 144, in simple_response

await self.app(scope, receive, send)

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in call

await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app

raise exc

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app

await app(scope, receive, sender)

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/starlette/routing.py", line 715, in call

await self.middleware_stack(scope, receive, send)

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/starlette/routing.py", line 735, in app

await route.handle(scope, receive, send)

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/starlette/routing.py", line 288, in handle

await self.app(scope, receive, send)

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/starlette/routing.py", line 76, in app

await wrap_app_handling_exceptions(app, request)(scope, receive, send)

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app

raise exc

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app

await app(scope, receive, sender)

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/starlette/routing.py", line 73, in app

response = await f(request)

           ^^^^^^^^^^^^^^^^

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/fastapi/routing.py", line 301, in app

raw_response = await run_endpoint_function(

               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/fastapi/routing.py", line 212, in run_endpoint_function

return await dependant.call(**values)

       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/mediafusion/utils/wrappers.py", line 23, in wrapper

return await func(*args, **kwargs)

       ^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/mediafusion/api/main.py", line 583, in encrypt_user_data

encrypted_str = crypto.encrypt_user_data(user_data)

                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/mediafusion/utils/crypto.py", line 22, in encrypt_user_data

cipher = AES.new(SECRET_KEY, AES.MODE_CBC, iv)

         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/Crypto/Cipher/AES.py", line 228, in new

return _create_cipher(sys.modules[__name__], key, mode, *args, **kwargs)

       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/Crypto/Cipher/init.py", line 79, in _create_cipher

return modes[mode](factory, **kwargs)

       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/Crypto/Cipher/_mode_cbc.py", line 274, in _create_cbc_cipher

cipher_state = factory._create_base_cipher(kwargs)

               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/mediafusion/.local/share/virtualenvs/mediafusion-ZmKbvzsi/lib/python3.12/site-packages/Crypto/Cipher/AES.py", line 90, in _create_base_cipher

raise ValueError("Incorrect AES key length (%d bytes)" % len(key))

ValueError: Incorrect AES key length (44 bytes)

should i be generating any form of specific key type in lesser length ? Also let me check the response message and let you know

mhdzumair commented 2 months ago

the key length should be 32 characters.

kiskey commented 2 months ago

@kiskey could you open the browser network tab and check the post entry and check the response message

i see 500 Internal server error in the network response for the post request. Let me correct key size and try again and update you

kiskey commented 2 months ago

hi @mhdzumair ,

You are right. The key size was the issue. After inputting 32 character lenth key for secret_key . Now i don't see that issue anymore. So it works Now.

Thanks again for guiding and helping me to figure out.

Though i am wondering why it did not accept the key i have been using before with stable version 3.9.16 .

mhdzumair commented 2 months ago

Though i am wondering why it did not accept the key i have been using before with stable version 3.9.16 .

I didn't quite understand. You can use any characters in the key.

The recent changes affect how user data is generated. Since URLs generally have a character limit of around 2k, the system now tries to reduce the size by encoding alias names in the config, encrypting, then compressing and encoding the data. Previously, the entire config was compressed first, then encrypted and encoded, which resulted in a slightly larger size.

kiskey commented 2 months ago

Though i am wondering why it did not accept the key i have been using before with stable version 3.9.16 .

I didn't quite understand. You can use any characters in the key.

The recent changes affect how user data is generated. Since URLs generally have a character limit of around 2k, the system now tries to reduce the size by encoding alias names in the config, encrypting, then compressing and encoding the data. Previously, the entire config was compressed first, then encrypted and encoded, which resulted in a slightly larger size.

Thanks for explaining it. I figured it also has something to do with my cache /session . When i tried with private tab and test the manifest url generation , it worked with the old key. So session/cache has something to do.not sure.

Thanks again and now its working fine as expected.

kiskey commented 2 months ago

hi @mhdzumair ,

I further tested the search results with series. Sometimes it misses the results returned . For example, the below . Say i selected on UI True detective S4 EP2 , The result seem to be fetching all the seasons and episodes, but when mediafusion filters them, it seem to drop or not matching the title properly. (Note: The search used the imdbid , so the results returned are all for the same titles )

From the result below ,

the row number 3 should have been matched, but mediafusion did not match

as its comparing wrong titles , not sure how it came up with title "Detective" vs the actual imdb title . So the entire results seems to be dropped and returned no streams to ui.

Quick Questions/Suggestions:

    1. When searching the tvseries type, i saw the prowlarr.py was issued the only imdbid , is there any way you can also pass the season and episode in query to prowlarr directly  or the minimum pass the imdbid and season  and then in prowalarr.py can match only the episode or range of episodes  , as sometimes results or within in range ? As this would only fetch the exact results,  so title matches can be ignored whenever the imdbid based search is sent out
    2.  when no imdbid based search query  and search is free text type search , then matching title seems good , so we can filter out potential mismatches

Please forgive my ignorance as i fully did not understand the coding, my suggestions or questions are purely based on what i see on logs and how the returned results are working by observing.

INFO::16-Sep-24 19:16:29::prowlarr.py::394 - Found 15 streams for True Detective (2024) with tvsearch Search, params: {'query': '{IMDbId:tt2356777}', 'categories': [5000], 'type': 'tvsearch'}

WARNING::16-Sep-24 19:16:29::base_scraper.py::181 - Title mismatch: 'Detective' vs. 'True Detective'. Torrent title: 'True.Detective.S03E02.720p.WEB.x265-MiNX[eztv].mkv'

WARNING::16-Sep-24 19:16:29::base_scraper.py::181 - Title mismatch: 'Detective' vs. 'True Detective'. Torrent title: 'True.Detective.S03E02.720p.WEB.x265-MiNX[TGx]'

WARNING::16-Sep-24 19:16:29::base_scraper.py::181 - Title mismatch: 'Detective' vs. 'True Detective'. Torrent title: 'True.Detective.S04E02.Night.Country.Parte.2.ITA.ENG.2160p.MAX.WEB-DL.DD5.1.HDR.DoVi.x265-MeM.GP.mkv'

WARNING::16-Sep-24 19:16:29::base_scraper.py::181 - Title mismatch: 'Detective' vs. 'True Detective'. Torrent title: 'True.Detective.S01E07.720p.HDTV.x264-KILLERS [PublicHD]'

WARNING::16-Sep-24 19:16:29::base_scraper.py::181 - Title mismatch: 'Detective' vs. 'True Detective'. Torrent title: 'True.Detective.S03E06.720p.WEB.H264-METCON[rarbg]'

WARNING::16-Sep-24 19:16:29::base_scraper.py::181 - Title mismatch: 'Detective' vs. 'True Detective'. Torrent title: 'True.Detective.S03E08.720p.WEB.x265-MiNX[TGx]'

WARNING::16-Sep-24 19:16:29::base_scraper.py::181 - Title mismatch: 'Detective' vs. 'True Detective'. Torrent title: 'True.Detective.S04E06.720p.10bit.WEBRip.2CH.x265.HEVC-PSA.mkv'

WARNING::16-Sep-24 19:16:29::base_scraper.py::181 - Title mismatch: 'Detective' vs. 'True Detective'. Torrent title: 'True.Detective.S04E04.1080p.WEB.H264-SuccessfulCrab'

WARNING::16-Sep-24 19:16:29::base_scraper.py::181 - Title mismatch: 'Detective' vs. 'True Detective'. Torrent title: 'True.Detective.S04E05.Night.Country.Part.5.720p.AMZN.WEB-DL.DDP5.1.H.264-NTb.mkv'

WARNING::16-Sep-24 19:16:29::base_scraper.py::181 - Title mismatch: 'Detective' vs. 'True Detective'. Torrent title: 'True.Detective.S03E05.720p.WEB.x265-MiNX[TGx]'

WARNING::16-Sep-24 19:16:29::base_scraper.py::181 - Title mismatch: 'Detective' vs. 'True Detective'. Torrent title: 'True Detective S01E07 720p HDTV ReEnc DeeJayAhmed'

WARNING::16-Sep-24 19:16:29::base_scraper.py::181 - Title mismatch: 'Detective' vs. 'True Detective'. Torrent title: 'True.Detective.S03E05.If.You.Have.Ghosts.1080p.10bit.WEBRip.6CH.x265.HEVC-PSA.mkv'

WARNING::16-Sep-24 19:16:29::base_scraper.py::181 - Title mismatch: 'Detective' vs. 'True Detective'. Torrent title: 'True Detective ( Season 1) 2014 BDRemux 1080p 3xUkr_Eng-Sub Ukr_Eng [Hurtom]'

WARNING::16-Sep-24 19:16:29::base_scraper.py::181 - Title mismatch: 'Detective' vs. 'True Detective'. Torrent title: 'True.Detective.S01E08.720p.rus.LostFilm.TV.mp4'

WARNING::16-Sep-24 19:16:29::base_scraper.py::181 - Title mismatch: 'Detective' vs. 'True Detective'. Torrent title: 'True.Detective.S02E01.PROPER.720p.HDTV.x264-KILLERS'

INFO::16-Sep-24 19:16:29::network.py::138 - Processed 15 items out of 15 total items.

INFO::16-Sep-24 19:16:29::prowlarr.py::275 - Finished processing 0 streams from 1 generators

INFO::16-Sep-24 19:16:29::prowlarr.py::93 - Returning 0 scraped streams for True Detective

INFO::16-Sep-24 19:16:29::utils.py::47 - Scraped 0 streams for True Detective

mhdzumair commented 2 months ago

This was due to an issue in Parsett library https://github.com/dreulavelle/PTT the "True" keyword identified as audio=["TrueHd"]

>>> PTT.parse_title('True.Detective.S03E02.720p.WEB.x265-MiNX[eztv].mkv')

{'resolution': '720p', 'quality': 'WEB', 'codec': 'hevc', 'audio': ['TrueHD'], 'group': 'MiNX', 'container': 'mkv', 'seasons': [3], 'episodes': [2], 'extension': 'mkv', 'languages': [], 'title': 'Detective'}

cc: @dreulavelle

kiskey commented 2 months ago

This was due to an issue in Parsett library https://github.com/dreulavelle/PTT the "True" keyword identified as audio=["TrueHd"]

>>> PTT.parse_title('True.Detective.S03E02.720p.WEB.x265-MiNX[eztv].mkv')

{'resolution': '720p', 'quality': 'WEB', 'codec': 'hevc', 'audio': ['TrueHD'], 'group': 'MiNX', 'container': 'mkv', 'seasons': [3], 'episodes': [2], 'extension': 'mkv', 'languages': [], 'title': 'Detective'}

cc: @dreulavelle

But should we match title when imdbid and season is issued in query unless free text search? ,whereas result is exact match for title and seasons (atleast for sure the indexer bitmagnet i use seems to return exact result not sure about other indexers), episode returned sill could be in range or the whole season of episodes is given in collection as one result.

Not sure if the manifest.json option of parsing torrent vs not parsing torrent has anything to do with the above behaviour or it always only issues imdbid in query and always parses the title and stuff.

mhdzumair commented 2 months ago

But should we match title when imdbid and season is issued in query unless free text search?

This is a bit risky. But I see the benefit in your case. Maybe I can bypass title check based on env config value

kiskey commented 2 months ago

But should we match title when imdbid and season is issued in query unless free text search?

This is a bit risky. But I see the benefit in your case. Maybe I can bypass title check based on env config value

Thanks @mhdzumair ,I see your point ,it could behave differently with other indexers . Also ,I like idea of controlling through environment variable . Once you released the environment variable ,please update me ,I can test that 👍