search

elgatito / script.elementum.burst

Development of this addon has been stopped!
MIT License
117 stars 119 forks source link

Questions regarding season search & 1337x #315

Open burekas7 opened 2 years ago

burekas7 commented 2 years ago

@elgatito Hi,

When I'm trying for example to make a search for "Black Mirror S04S02" with 1337x It gives me only results from: https://www.1337x.to/search/black+mirror+s04e02/1/

But when I ran a manual search in the site for season I get additional results (The torrents doesn't work, but the Magnet link does work) https://www.1337x.to/search/black+mirror+s04/1/

So my questions is: 1) Why I don't get also the season results in additional to the episodes results? Does it because the torrents links are broken? (Because the Magnet link is working) 2) Does it should search also season results in addtion or only at specific conditions? 3) Does burst return also Magnet results in additional to Torrents results?

Kodi debug log: https://paste.ubuntu.com/p/7PWbFQ98JZ raw: https://paste.ubuntu.com/p/7PWbFQ98JZ/plain/

Burst 0.0.70 Elementum 0.1.86 Kodi 19.3

antonsoroko commented 2 years ago

hi.

just as into: 1st search is for specific episode so it gives you specific results. 2nd search is for season so it gives you a lot of results but vast majority of then are irrelevant (for different episodes).

With that in mind:

Why I don't get also the season results in additional to the episodes results?

because search for season gives you a lot of results but vast majority of then are irrelevant (for different episodes).

Does it because the torrents links are broken? (Because the Magnet link is working)

No. See answer above. But just as side note: torrents (or magnets, same in this context) can be filtered depending on your settings (so seeds, wrong screen resolution, type of rip, size, etc) - see your settings to learn more.

Does it should search also season results in addtion or only at specific conditions?

Depends on configuration of provider, but usually only when:

  1. you search for season (you can click on season and search a torrent for it)
  2. if nothing was found for episode and provider has "fallback" query (that query usually uses season search).

Does burst return also Magnet results in additional to Torrents results?

Depends on configuration of provider - it is either torrent file or magnet (for 1337x it is magnet iirc).

You can see technical search details about 1337x here - https://github.com/elgatito/script.elementum.burst/blob/f66d4dd1cd4da968f8e2233e0189b6920be9bacf/burst/providers/providers.json#L69

e.g. fallback search for episode (when nothing is found) - is to search for season - https://github.com/elgatito/script.elementum.burst/blob/f66d4dd1cd4da968f8e2233e0189b6920be9bacf/burst/providers/providers.json#L94

burekas7 commented 2 years ago

Thanks. @antonsoroko

But I had a lot of cases that searching for a specific episode still returns a season reasults which contains the specific episode in additoin to others episode results for the same provider. So I don't think like you said in (2) that if nothing was found for episode or standing on a season and choose streaming from the context menu.

By the way even when I tried to make a season search I still don't get this specific result that I found manually, why is that?

This is the missing result I found by myself: https://www.1337xxx.to/torrent/2696524/Black-Mirror-S04-COMPLETE-INTERNAL-1080p-WEB-x264-STRiFE/

antonsoroko commented 2 years ago

But I had a lot of cases that searching for a specific episode still returns a season reasults which contains the specific episode in additoin to others episode results for the same provider.

as i said - depends how provider is configured. some might have 2 queries (for episode and season at the same time), but most have 1 query for episode and then fallback for season. See providers.json for details.

So I don't think like you said in (2) that if nothing was found for episode or standing on a season and choose streaming from the context menu.

I do not understand what you wanted to tell. Anyway - what i wrote is based on burst code and providers.json code.

By the way even when I tried to make a season search I still don't get this specific result that I found manually, why is that?

As i said - maybe it was filtered out by your settings. See log files for details. Also iirc elementum will actually try to resolve number of seeds - so if site says 10 but in reality it is zero - it will be filtered out. You can enable "zero seed" torrents in settings and check if this is the case (also it should be reflected in logs iirc).

burekas7 commented 2 years ago

Understood.

So instead of:

        "tv_keywords": "{title} s{season:2}e{episode:2}",
        "tv_keywords2": null,

I changed it to and it seems to be better now:

        "tv_keywords": "{title} s{season:2}e{episode:2}",
        "tv_keywords2": "{title} s{season:2}",

I will open a PR.

By the way: 1) Why not to update all the providers with their "tv_keywords2" to support season search in addition to the specific episode? (Some of them doesn't have this key at all) 2) Why some of the providers their "tv_keywords" is empty?

And just to be sure, if I select a season result (According to "tv_keywords2"), playing that result should selects automatically the specific episode from it right?

But regarding the other issue about this specific result: https://www.1337xxx.to/torrent/2696524/Black-Mirror-S04-COMPLETE-INTERNAL-1080p-WEB-x264-STRiFE/

In the site it shows there are seeds, and except its torrents links that are broken, the Magnet link is active with seeds (I used it), so it can't be that it is filterred by the zero seeds option. In fact, I disabled the zero seeds filter and this specific result still doesn't show up.

log: https://paste.ubuntu.com/p/4qHkDhRSVK

antonsoroko commented 2 years ago

I will open a PR.

please don't. right now behavior is correct like i described above:

because search for season gives you a lot of results but vast majority of then are irrelevant (for different episodes).

antonsoroko commented 2 years ago

you can make local override if you want such behavior - see https://github.com/elgatito/script.elementum.burst#adding--editing-providers

antonsoroko commented 2 years ago

Why not to update all the providers with their "tv_keywords2" to support season search in addition to the specific episode?

let me answer again :-)

because search for season gives you a lot of results but vast majority of then are irrelevant (for different episodes).

so navigating a lot of irrelevant result will make you mad after some time.

Why some of the providers their "tv_keywords" is empty?

probably they are: anime only? movie only? (or maybe person who added them was lazy :-) )

antonsoroko commented 2 years ago

log: https://paste.ubuntu.com/p/4qHkDhRSVK

open link from log 2022-01-24 18:53:30.111 T:23164 WARNING <general>: [script.elementum.burst] [1337x] > 1337x search URL: https://www.1337x.to/search/black+mirror+s04/1/

also see here https://github.com/elgatito/script.elementum.burst/blob/f66d4dd1cd4da968f8e2233e0189b6920be9bacf/burst/providers/providers.json#L55

it uses only first page. your torrent is on second page.

UPD: iirc burst does not support navigating pages. usually we just use query that indicates big number of results per page if possible (not possible for 1337x). another reason not to use season search for episodes - it can "shadow" direct torrents. but as i mentioned - you can use local override if you want non-standard behavior.

burekas7 commented 2 years ago

So I summarize this issue with another idea, a little fix.

But before my change, just to be sure: It make search also for "tv_keywords" & "tv_keywords2" or only for "tv_keywords" and if there is a problem it's checking the "tv_keywords2"?

Lets take an example for this case: Black Mirror S04E02

My advice is to change from the current provider's configuration is:

        "anime_keywords": "{title} s{season:2}e{episode:2}",
        "anime_keywords_fallback": "{title} s{season:2}",
        "anime_keywords_fallback2": "{title} season {season}",
        "anime_keywords_fallback3": "{title} {season:2}x01",
        "base_url": "https://www.1337x.to/search/QUERYEXTRA/1/",
        "movie_keywords": "{title} {year}",
        "season_keywords": "{title} s{season:2}",
        "season_keywords2": "{title} season {season}",
        "season_keywords_fallback": "{title} {season:2}x01",
        "tv_keywords": "{title} s{season:2}e{episode:2}",
        "tv_keywords2": null,
        "tv_keywords_fallback": "{title} s{season:2}",
        "tv_keywords_fallback2": "{title} season {season}",
        "tv_keywords_fallback3": "{title} {season:2}x01"

To this one:

        "anime_keywords": "{title} s{season:2}e{episode:2}/1",
        "anime_keywords_fallback": "{title} s{season:2}/1",
        "anime_keywords_fallback2": "{title} season {season}/1",
        "anime_keywords_fallback3": "{title} {season:2}x01/1",
        "base_url": "https://www.1337x.to/search/QUERYEXTRA/",
        "movie_keywords": "{title} {year}/1",
        "season_keywords": "{title} s{season:2}/1",
        "season_keywords2": "{title} s{season:2}/2",
        "season_keywords_fallback": "{title} season {season:2}/1",
        "tv_keywords": "{title} s{season:2}e{episode:2}/1",
        "tv_keywords2": "{title} s{season:2}e{episode:2}/2",
        "tv_keywords_fallback": "{title} s{season:2}/1",
        "tv_keywords_fallback2": "{title} season {season}/1",
        "tv_keywords_fallback3": "{title} {season:2}x01/1"

The mainly change is to move the 'page' query to the keywords, Regarding my case that now the season results will be returned with more results from both pages. And also the episode will have the option of looking also for page 2 if there is. Also made some fixes with the queries.

antonsoroko commented 2 years ago

heh, interesting hack :-)

just a reminder - you would also need to change queries for anime and movies since you changed base_url and now they will not work w/o extra /N/.

btw there are up to 5 queries per type (e.g. from tv_keywords up to tv_keywords4).

also it might be more flexible to use _extra keyword, for example https://github.com/elgatito/script.elementum.burst/blob/master/burst/providers/providers.json#L1692-L1694 https://github.com/elgatito/script.elementum.burst/blob/master/burst/providers/providers.json#L2956

so you will make "base_url": "https://www.1337x.to/search/QUERY/EXTRA/", (note / between QUERY and EXTRA) and then set appropriate _extra to 1/2/3/4/5.

also note: all _keywords* executed at the same time so you will create more load to tracker (although it also depend of how many elementum's users uses 1337x), and also not every search will have multiple pages to it will generates some errors in log. as usual in life - evaluate tradeoffs to find "acceptable" solution.

burekas7 commented 2 years ago

@antonsoroko

just a reminder - you would also need to change queries for anime and movies since you changed base_url and now they will not work w/o extra /N/.

You're right, I will fix it.

btw there are up to 5 queries per type (e.g. from tv_keywords up to tv_keywords4).

I think 2 pages is enough :) And regarding you note. This small error won't effect so much (again 2 page is enough. so it's a one error)

so you will make "base_url": "https://www.1337x.to/search/QUERY/EXTRA/", (note / between QUERY and EXTRA) and then set appropriate _extra to 1/2/3/4/5.

What is the difference between 'QUERYEXTRA' to 'QUERY/EXTRA'? I thought it's a one argement. Does it actually two argments? What is the QUERY and what is the EXTRA? But I think I will keep it as I already did.

antonsoroko commented 2 years ago

Does it actually two argments?

yes.

What is the QUERY and what is the EXTRA?

this is a bit confusing :-) for example, season_keywords is QUERY and season_extra is EXTRA. then in base_url (or in season_query - which might be confusing because of its name) burst will replace replace QUERY and EXTRA with appropriate _query and _extra for specific type of search (movie, episode, etc).

So you can use QUERY and EXTRA not only in base_url but also in _query, like in my examples. I used EXTRA to implement search in multiple categories in rutor (e.g. local and foreign shows have different categories), same for torrent.by. So it is also possible to implement "pages" hack with the same logic.

Since _extra is rarely used you can see a lot of QUERYEXTRA - so basically since EXTRA is empty it becomes just QUERY.

antonsoroko commented 2 years ago

you moved "season_keywords2": "{title} season {season}", to "season_keywords_fallback": "{title} season {season:2}/1",

i would test first how many results this query gives - maybe we will lose some good results b/c of this change.

also JFYI - all _keywords* are run at the same time, but _keywords_fallback* are run 1 by 1 until found something (so most likely not all of them will be run).

kovacicjanez commented 2 years ago

Can someone post updated 1337x provider configuration? Thanks :)

burekas7 commented 2 years ago

@kovacicjanez

This is my configuration for now, you can use it: Copy and replace. (If I will change it in the future I will update here)

    "1337x": {
        "anime_extra": "",
        "anime_keywords": "{title} s{season:2}e{episode:2}/1",
        "anime_keywords_fallback": "{title} s{season:2}/1",
        "anime_keywords_fallback2": "{title} season {season}/1",
        "anime_keywords_fallback3": "{title} {season:2}x01/1",
        "anime_query": "",
        "base_url": "https://www.1337x.to/search/QUERYEXTRA/",
        "color": "FFF14E13",
        "enabled": true,
        "general_extra": "",
        "general_keywords": "{title}",
        "general_query": "",
        "language": null,
        "languages": "en",
        "login_failed": "",
        "login_object": "",
        "login_path": null,
        "movie_extra": "",
        "movie_keywords": "{title} {year}/1",
        "movie_query": "",
        "name": "1337x",
        "parser": {
            "infohash": "",
            "name": "item('a', order=2)",
            "peers": "item(tag='td', order=3)",
            "row": "find_once(tag='tbody').find_all('tr')",
            "seeds": "item(tag='td', order=2)",
            "size": "item(tag='td', order=5)",
            "torrent": "'https://www.1337x.to%s' % item(tag='a', attribute='href', order=2)"
        },
        "predefined": true,
        "private": false,
        "season_extra": "",
        "season_extra2": "",
        "season_keywords": "{title} s{season:2}/1",
        "season_keywords2": "{title} s{season:2}/2",
        "season_keywords_fallback": "{title} season {season:2}/1",
        "season_query": "",
        "separator": "+",
        "show_query": "",
        "subpage": true,
        "tv_extra": "",
        "tv_extra2": "",
        "tv_keywords": "{title} s{season:2}e{episode:2}/1",
        "tv_keywords2": "{title} s{season:2}e{episode:2}/2",
        "tv_keywords_fallback": "{title} s{season:2}/1",
        "tv_keywords_fallback2": "{title} season {season}/1",
        "tv_keywords_fallback3": "{title} {season:2}x01/1"
    },
SubEdict commented 1 year ago

Been a long time, and I'm still never see any correct search results from 1337x, it always looks like spam. Sample 1: And Just Like that s02e06 https://i.imgur.com/0SWGoFa.png

Sample 2: Star Trek s02e06 https://i.imgur.com/caOtdgu.png

@burekas7 Did you manage to overcome this specific "spam" issue?

elgatito commented 1 year ago

@SubEdict 1337 definition does not use specific language, which means it would use your Kodi's language in request, which results in random output if you use non-english request...

SubEdict commented 1 year ago

@elgatito Sooo simple, bingo! I changed the interface to English and... Boom! All 1337x results are now correct. So embarrassing, didn't think of this almost 2 years. 🙈

antonsoroko commented 1 year ago

@SubEdict for reference https://github.com/elgatito/script.elementum.burst/commit/8799e2eaebdb34b3ae03dd8fc0893e22f0767587

antonsoroko commented 1 year ago

Although I never had this issue because I used this option: https://github.com/elgatito/script.elementum.burst/blob/8799e2eaebdb34b3ae03dd8fc0893e22f0767587/resources/language/messages.pot#L55

SubEdict commented 1 year ago

Yep, yep, great! all good now. I changed back to my default interface language and 1337x finally works as it should. Thanks again @elgatito, @antonsoroko for kipping everything up and running.

BTW, why would you use "Use language preference...." for? Mine is turned off (default) and all links on the web are English anyway.

antonsoroko commented 1 year ago

Oh, I meant that I do not use that option. But since it is disabled by default - it just means that I do not remember how things were, and I did have issues with 1337 when i searched something non-english. About what it does: https://github.com/elgatito/script.elementum.burst/blob/8799e2eaebdb34b3ae03dd8fc0893e22f0767587/burst/filtering.py#L463-L465

here is some bad-ass logic about what to use as title for search - language defined in tracker (providers.json) or language of kodi or original language of video.

SubEdict commented 1 year ago

The thing that confuse me was why in the first place if provider not in self.language_exceptions ...I mean all my providers are English, and movies/shows are also English. So why search algorithm care about GUI language preference? I always thought GUI translation is just visual, and search commands look for the original ID_English_Title-Number (like IMDB, tvdb, Trakt, etc.).

I am a simple man I don't like complications Those 3 lines of code make me think that for the best result maybe it's better to use English after all. 🤔

antonsoroko commented 1 year ago

@SubEdict

I mean all my providers are English, and movies/shows are also English. So why search algorithm care about GUI language preference?

Hah, you just extrapolated your preferences to everybody else :-D I use my language as GUI and I watch sometimes TV shows and movies in my language and in other languages (with subtitles). That's why burst also uses original title, which is not english for those cases. And with english title search will fail in many cases, because releases often use original language.

language_exceptions meaning is https://github.com/elgatito/script.elementum.burst/blob/8799e2eaebdb34b3ae03dd8fc0893e22f0767587/resources/language/messages.pot#L95

so you can search all trackers in your language (maybe to find releases with dubbing audio) except those trackers that do not understand/support other languages.

I guess this is just a "legacy" idea from original code. You can use "git blame" feature in github UI to see history of any line of code, like: https://github.com/elgatito/script.elementum.burst/commit/93bbc867c45efa26ee0477b2d0ce9522395a6e2b https://github.com/elgatito/script.elementum.burst/commit/04ceee429d68f5dcb41964c301abacdf38bfb7c4

this is 6.5 years old code, probably back then it was used to fix some issues with local (per-country) trackers.

Anyway, we also can specify language(s) in tracker definition (json) - if this information is correct then you will get proper results for queries in your language and will not get garbage results. But not all of them have language(s) defined, which also can be a reason why this option exists.

Also, language is used to prepare correct search query, like: https://github.com/elgatito/script.elementum.burst/blob/8799e2eaebdb34b3ae03dd8fc0893e22f0767587/burst/filtering.py#L478

So, now you know that searching is more complicated than it sounds :-)

SubEdict commented 1 year ago

@antonsoroko Thanks for all the logical explanations. live long and prosper 🖖