search

Sonarr / Sonarr

Smart PVR for newsgroup and bittorrent users.
https://sonarr.tv
GNU General Public License v3.0
10.35k stars 1.31k forks source link

Improve Searching For Anime Seasons #6495

Open Ravencentric opened 5 months ago

Ravencentric commented 5 months ago

Is there an existing issue for this?

Is your feature request related to a problem? Please describe

Series Type: Anime makes it so Sonarr searches every episode times every alias for every season of an Anime. This was probably necessary long long ago but in modern anime scene, this does nothing but slow down searches and waste api hits. For a 24 episode anime, it took Sonarr 10 mins on Nyaa and 4 mins on AB, this number usually varies depending on the number of episodes and alias but you get the gist of the problem.

This behavior is based on the philosophy that "anime has no seasons" which isn't true, atleast hasn't been true for years. Anime has seasons, They usually will either use SXX/Season X or more commonly, "Season Titles" to declare seasons. For example, a season pack may use a season title to differentiate itself, i.e, [group] Dr. Stone: Stone Wars [...] (xem has this info, https://github.com/Sonarr/Sonarr/issues/5195) where Stone Wars is the title of the second season.

To better demonstrate my point, consider the two "main" trackers for anime:

  1. AnimeBytes does not allow anything that isn't a pack at all, even airing episodes are all under a single group. Searching every episode of an anime on AnimeBytes is such a massive waste because of their strict grouping and season pack policies. The SysOp of AB himself addressed how AB only allows season packs. Prowlarr already realizes this and strips out the episode number from the search and caches the results so when Sonarr hits AB for the 100th time, it gives the same cached result again and again till Sonarr exhausts all the episode and alias combos but this still ends up adding a weird artificial slowness to the whole process. Sonarr could have had done the same thing by just searching the season aliases in a few seconds.

  2. Nyaa, the biggest anime public tracker (and possibly the only relevant one), is the source of most anime. Nyaa is where most content originates and then gets cross posted to AB. So practically, Nyaa has most of the season packs that AB has, albeit with worse retention. Once again, Sonarr searching every alias of every episode ends up being a massive waste of both API hits and anime (which can sometimes get you 429'd). This takes even longer than AB due to no Prowlarr magic behind it, causing a 24 episode anime to take 10 whole mins to search.

Q. But One Piece/Pokemon/{insert old and long anime here}? A. Although I would say these are exceptions, they still have season packs on both Nyaa and AB. Even if they don't series like these are exceptions and should not be considered for the default behavior any longer.

The anime that don't have batches are a minority and an exception. The old continuously running anime for thousands of episode isn't thing anymore and hasn't been a thing for years. Anime has followed a much more digestable 12-24 episode "seasons" format for years. These get immediately packed as soon as they end airing.

The current behavior favors a now dead system that makes up a minority of anime and ends up making things worse for every thing released in the last decade and later. Most of the time you'll end up having to grab these messy super long shows manually anyway.

Describe the solution you'd like

The solution would be making it behave much more like Series Type: Standard but without the forced season/ep params. Search the season aliases and be done with it. Why not use Standard? because it forces the season param and tacks on the S01, S02, etc when this is mostly "optional" in the anime scene. You'll miss Dr Stone: Stone Wars (S02) if you search Dr Stone S02. The season pack can be either Dr Stone: Stone wars or Dr Stone S02.

EDITED to include https://github.com/Sonarr/Sonarr/issues/6495#issuecomment-1936641651

Ideally, an Anime Type should do the following:

Describe alternatives you've considered

Grabbing a pack manually because it's faster than letting Sonarr search every episode

Anything else?

No response

thezoggy commented 5 months ago

if you have better indexers where they support id searches then you dont deal with the suboptimal nature of string searches². https://wiki.servarr.com/sonarr/faq#how-does-sonarr-find-episodes

now about season packs.. 'anime' series type in sonarr (i hate that sonarr calls it anime) but its really just means absolute numbering. which does not have season packs.. its just episode number. if you want season concept, use 'standard' type. then season packs work fine

²assuming that there is results for the id search. if none, then will fall back to string searches.

mynameisbogdan commented 5 months ago

I already had this talk about searching on AB, to me the solution is either AB starts to support season+episode search or it's added natively into Sonarr and does a single search.

Ravencentric commented 5 months ago

if you have better indexers where they support id searches then you dont deal with the suboptimal nature of string searches². https://wiki.servarr.com/sonarr/faq#how-does-sonarr-find-episodes

Issue isn't really about what Indexer/tracker I have, AB/Nyaa are the home for anime, they get more anime and they get it faster than the top indexers/trackers

now about season packs.. 'anime' series type in sonarr (i hate that sonarr calls it anime) but its really just means absolute numbering. which does not have season packs.. its just episode number. if you want season concept, use 'standard' type. then season packs work fine

Ideally, an Anime Type should do the following:

This will pretty much deal with anime smoothly.

thezoggy commented 5 months ago

if you have better indexers where they support id searches then you dont deal with the suboptimal nature of string searches². https://wiki.servarr.com/sonarr/faq#how-does-sonarr-find-episodes

Issue isn't really about what Indexer/tracker I have, AB/Nyaa are the home for anime, they get more anime and they get it faster than the top indexers/trackers

now about season packs.. 'anime' series type in sonarr (i hate that sonarr calls it anime) but its really just means absolute numbering. which does not have season packs.. its just episode number. if you want season concept, use 'standard' type. then season packs work fine

Ideally, an Anime Type should do the following:

* When user explicitly requests a single episode, the current behavior should continue as is like so:

  * `Dr Stone 13` # S02E01
  * `Dr Stone S2 01` # S02E01
  * `Dr Stone S02E01` # S02E01
  * `Dr Stone Stone wars 01` # S02E01

* When season packs are requested, do NOT search individual episodes. Search the xem aliases:

  * `Dr Stone S02`
  * `Dr Stone S2`
  * `Dr Stone Stone Wars`
    _Only 3 searches get the job done instead of 4x12_

This will pretty much deal with anime smoothly.

this assumption is that all series are in xem (they arent, nor would i want them to be).. then not every release group uses the same season alias or need aliases to begin with. and yes it goes back to if you were using better indexers which supported id searches where the series title part of this equation is removed on searching only for matching.

Ravencentric commented 5 months ago

this assumption is that all series are in xem (they arent, nor would i want them to be).. then not every release group uses the same season alias or need aliases to begin with.

If they are not in xem then they already only search a single alias but 12 episodes (again, useless) and they would get the same thing by just searching the name of the show. There was never an assumption that they must be in xem, i just picked an example from xem to convey the waste sonarr generates. In pretty much every case, Searching Show Name 01, Show Name 02, ..., Show Name 12 (no xem aliases) is a waste that at best gives you the same result as searching Show Name once or at worst, actually filters out season packs because they don't mention the episode number.

yes it goes back to if you were using better indexers which supported id searches where the series title part of this equation is removed on searching only for matching.

I already have the top trackers and secret indexers, the point is they are not the home for anime, they are slower to get it and they have smaller anime library compared to AB (which is obvious, anime tracker vs trackers that happen to have anime)

You can boil down my request to this:

thezoggy commented 5 months ago

But this is not true. Sites return limited results so searching by just the series (or even just the alias) may have to many results to return pushing the one the user wants out of the return results if it's not within that limit. This happens with vaguely named series or very long running series...

Which is why you would want to have the absolute number to limit scope.

There is definitely room for improvement with how sonarr does aliases (wish it only did aliases for the lang the user cares for example) but then there is the dataset problem where people want sub/sub which may not align with the lang of the alias or what the group is known for.. so it gets messy to assume. This is why I do push back on adding aliases on xem to try and limit resources drain that it will bring on everyone because of how things work.

In perfect world anime would use some sensible standards with naming that one could just simply leverage that metadata.. but sadly it doesn't exist. So we have to play this alias game most of the time.

Ravencentric commented 5 months ago

But this is not true. Sites return limited results so searching by just the series (or even just the alias) may have to many results to return pushing the one the user wants out of the return results if it's not within that limit.

Searching every episode is helping sonarr narrow down an episode, which it shouldn't look for to begin with for a completed season. Season packs are uploaded after the airing ends so they'll be top of the result when searched without the episode numbers. Searching season packs with episode numbers is actually shoving season packs down the results or filtering them out entirely!

This happens with vaguely named series or very long running series...

These are issues with the current system too and often require you to manually do it anyway. There are 170 anime in the whole world that cross the 100 episode count, a large portion of these are older than 2000s and exclusive to Japan. Again, a minority of anime should not be considered for the default search behavior causing issues for a majority of anime.

In perfect world anime would use some sensible standards with naming that one could just simply leverage that metadata.. but sadly it doesn't exist.

Yes, and it'll probably never exist either. Best thing that could be done here is improving the current method to work better with modern anime scene instead of following practices from 2 decades ago.

rg9400 commented 5 months ago

But this is not true. Sites return limited results so searching by just the series (or even just the alias) may have to many results to return pushing the one the user wants out of the return results if it's not within that limit. This happens with vaguely named series or very long running series...

Which is why you would want to have the absolute number to limit scope.

There is definitely room for improvement with how sonarr does aliases (wish it only did aliases for the lang the user cares for example) but then there is the dataset problem where people want sub/sub which may not align with the lang of the alias or what the group is known for.. so it gets messy to assume. This is why I do push back on adding aliases on xem to try and limit resources drain that it will bring on everyone because of how things work.

In perfect world anime would use some sensible standards with naming that one could just simply leverage that metadata.. but sadly it doesn't exist. So we have to play this alias game most of the time.

Honestly, the current behavior makes searching anime beyond the episode level a complete no go. It's effectively useless. Because it searches every single episode across multiple aliases, you'll end with 3000+ results for a small show, and yet since the best releases are season packs, you still won't get the one you'd want. If you use Usenet, you'll exhaust your API hits with just a few searches. The advice given to people using Sonarr for anime is to simply not backfill and manually grab releases, and only use the RSS for ongoing anime.

Why? Because RSS actually grabs and parses season packs fine while also still grabbing the absolute episode individual episodes for currently airing stuff.

The behavior for anime would be significantly improved if a show and season search only searched season aliases for completed seasons, and episodes for ongoing seasons. This would actually make it functional. Also, not as important, but it would be nice if anime stopped pegging episodes to absolute episode numbers so that if the order changed, the files themselves got switched across seasons. I think the importance of absolute episode number is significantly less now as Raven mentioned, and while there are a few outliers, most don't need that much weightage on it. The main thing they should be used for is to parse and search episode numbers.

markus101 commented 5 months ago

Thanks for the discussion on this so far, this is something I've been thinking about off and on and it's definitely something that needs to be improved.

* When season packs are requested, do NOT search individual episodes. Search the xem aliases:

  * `Dr Stone S02`
  * `Dr Stone S2`
  * `Dr Stone Stone Wars`
    _Only 3 searches get the job done instead of 4x12_

What about searches for season 1? Is Searching for Dr Stone S1 and Dr Stone S01 the expectation? Searching for just Dr Stone will yield results from every season and alias it'd be virtually impossible to find them, but individual results would be very limited.

Should Sonarr always search for Dr Stone Sxx even on series + season combinations where that's not an explicit alias?

Searching for Dr Stone Stone Wars looks like it will yield results, but Sonarr isn't going to be able to do much with those results because of the lack of season/episode identifiers, though interactive search should be able to handle them (with override and grab).

Ravencentric commented 5 months ago

What about searches for season 1? Is Searching for Dr Stone S1 and Dr Stone S01 the expectation? Searching for just Dr Stone will yield results from every season and alias it'd be virtually impossible to find them, but individual results would be very limited.

Yea, the best sonarr could do here is just search Dr stone, Dr stone s1, Dr stone season 1, and Dr stone s01 to try to get them. Sonarr can only fix bad naming so much so i think trying the season and failing early is still better the current behavior. This will atleast still make single season show searches far better.

Should Sonarr always search for Dr Stone Sxx even on series + season combinations where that's not an explicit alias?

yes

Searching for Dr Stone Stone Wars looks like it will yield results, but Sonarr isn't going to be able to do much with those results because of the lack of season/episode identifiers, though interactive search should be able to handle them (with override and grab).

Prowlarr already handles this by guesstimating the season number and appending it to the title before passing to sonarr. For example: Nyaa title: [MTBB] A Place Further than the Universe (BD 1080p) | Sora yori mo Tooi Basho Prowlarr returns: [MTBB] A Place Further than the Universe (BD 1080p) | Sora yori mo Tooi Basho S01 Prowlarr does the same for AB too

side note: apologies for the late response, been a bit busy

markus101 commented 5 months ago

Thanks for the info.

Internally this will require changes to how aliases are fetched and used for searching for anime, as well as removing the individual episode searching that is currently used.

ziomek64 commented 1 month ago

I agree with many things said here.

Would be nice if it searched for absolute numbers such as S2E1 could be searched as Title 13. Searches for episodes or seasons could be just 1 not 01. 01 removes MANY releases. A lot of groups put just the number without 0. Also, if anime has only one single season just search for the title without any numbers! Today I've tried to add many one season animes and 95% of actually good releases didn't show because it searched for Title 01. This would help.

sdugoten commented 3 weeks ago

In case something can be done, I am not sure if this helps, but here are the most common pattern used on anime title.

The following are all Date A Live season 5, episode 11. As you can see, they all comes in different pattern, close ...but not look exactly the same. That makes searching in Sonarr extremely difficult.

[[爱恋&漫猫字幕组][4月新番][约会大作战][Date A Live V][11][1080p][MP4][简中]]
[[爱恋&漫猫字幕组][4月新番][约会大作战][Date A Live V][11][1080p][MP4][繁中]]
[[澄空学园&华盟字幕社] Date A Live V 约会大作战V 第11话 MP4 720p]
[[Up to 21°C] 约会大作战第五季 / Date A Live V - 11 (ABEMA 1280x720 AVC AAC MP4)]
[[Lilith-Raws] 约会大作战 / Date A Live S05 - 11 [Baha][WebDL 1080p AVC AAC][CHT]]
[[ANi] Date a Live / 约会大作战 DATE A LIVE V - 11 [1080P][Baha][WEB-DL][AAC AVC][CHT][MP4]]
【千夏字幕组】【约会大作战IV_Date A Live IV​】[第10话][1080p_AVC][繁体] ​

For season pack would look something like this for Konosuba Season 2. And again, they look similar...but not exactly the same.


【极影字幕社】★01月新番[为美好的世界献上祝福！2 Kono Subarashii Sekai ni Shukufuku wo! 2][01-10+OVA][1080P][BIG5]

[[VCB-Studio] Kono Subarashii Sekai ni Shukufuku wo! / 为美好的世界献上祝福! / この素晴らしい世界に祝福を! 10-bit 1080p HEVC BDRip [S1+S2 Fin]]

[【DHR動研&茉語星夢】[為美好的世界獻上祝福！第二季_Konosuba 2][全10話+OVA][BDRip][繁體外掛字幕][1080P][HEVC_P10_FLAC_OPUS]Rev](https://share.dmhy.org/topics/view/485280_DHR_Konosuba_2_10_OVA_BDRip_1080P_HEVC_P10_FLAC_OPUS_Rev.html)

[[HoneyGod] 为美好的世界献上祝福! 2 Kono Subarashii Sekai ni Shukufuku o! 2[x265_10bit][BDrip_1080p][简繁外挂]]

Right now, I ues XArr-Rss docker to make searching index a bit easier, it can make matching a bit easier by modifying season number and episode number to a more standard way on the fly when feeding to Sonarr. But then...it's still slow although it does match correctly due to reason people state above