search

Jackett / Jackett

API Support for your favorite torrent trackers
GNU General Public License v2.0
12.21k stars 1.3k forks source link

UNIT3D API #8682

Closed HDVinnie closed 2 years ago

HDVinnie commented 4 years ago

Im wondering if it would be better to authenticate users and search torrents in Jackett via UNIT3D's API?

https://hdinnovations.github.io/UNIT3D-Community-Edition-Docs/api_endpoints.html

Im not sure a I have never really looked into the core of Jackett. Just throwing it out there.

HDVinnie commented 4 years ago

@garfield69 thoughts?

garfield69 commented 4 years ago

out of my league. I don't do c# @cadatoiva @ngosang this is your territory.

ngosang commented 4 years ago

I think we can move to the C# + API but there are around 25 trackers using UNIT3D. Testing all of them will take a lot of time. Since those trackers are working this is low priority for me.

HDVinnie commented 3 years ago

Any update on this? @ilike2burnthing ?

ilike2burnthing commented 3 years ago

No, we'd need people with the accounts, knowhow, time, and desire to rewrite them. You could try asking in https://github.com/Jackett/Jackett/issues/8180

garfield69 commented 3 years ago

@HDVinnie You may be interested to know that with the pending introduction of JSON support from within the Jackett cardigann processor #12424, it will soon be possible to code a yml indexer to access the unit3d API, allowing me to create yml indexers that can process JSON responses from unit3d sites.

[edit] I've already created and tested one for the Blutopia site.

Are you planning to make available a poster link to the API ? Also, do the sort_field and sort_direction query parms apply to the API? Thanks.

HDVinnie commented 3 years ago

I can add posters, sort_field and sort_direction query parms and whatever else you need/want. @garfield69

Just lmk.

garfield69 commented 3 years ago

I can add posters, sort_field and sort_direction query parms and whatever else you need/want. Just lmk.

Oh good. Yes please, these would be great additions to the unit3d API. Do you want me to formally make a ticket on https://github.com/HDInnovations/UNIT3D-Community-Edition or is this request here good enough to get the ball rolling? :-)

HDVinnie commented 3 years ago

Ill start it tonight.

HDVinnie commented 3 years ago

@garfield69 what size poster is recommended for the Jackett UI? And is anything else useful like genres, etc?

HDVinnie commented 3 years ago

@garfield69 poster has been added.

Also added sortField, sortDirection and perPage

Example data from demo site which gets reset ever 72 hours

https://unit3d.site/api/torrents/filter?perPage=1&api_token=X614ypqEq6xoojMdE2SwLs8gzmdInlRdg0PExASX3qUSHzprH1FAPHDTBSz7PmenleCHOlvhWaYfrQQhvxPHsKUjijXa6BYSKyex

{"data":[{"type":"torrent","id":"279","attributes":{"poster":"https://image.tmdb.org/t/p/w342/33VdppGbeNxICrFUtW2WpGHvfYc.jpg","name":"The Call of the Wild (2020)","release_year":2020,"category":"Movies","type":"Full Disc","resolution":"1080i","size":3888,"num_file":233,"freeleech":0,"double_upload":1,"internal":1,"uploader":"kbeier","seeders":673592,"leechers":565224862,"times_completed":75633,"tmdb_id":"481848","imdb_id":"81561","tvdb_id":"77203947","mal_id":"14","igdb_id":"978412469","created_at":"2021-10-21T03:10:27.000000Z","download_link":"https://unit3d.site/torrent/download/279.d152c0cfe31c12fb988f105ddaa5b146","details_link":"https://unit3d.site/torrents/279"}}],"links":{"first":"https://unit3d.site/api/torrents/filter?page=1","last":"https://unit3d.site/api/torrents/filter?page=279","prev":null,"next":"https://unit3d.site/api/torrents/filter?page=2","self":"https://unit3d.site/api/torrents"},"meta":{"current_page":1,"from":1,"last_page":279,"links":[{"url":null,"label":"« Previous","active":false},{"url":"https://unit3d.site/api/torrents/filter?page=1","label":"1","active":true},{"url":"https://unit3d.site/api/torrents/filter?page=2","label":"2","active":false},{"url":"https://unit3d.site/api/torrents/filter?page=3","label":"3","active":false},{"url":"https://unit3d.site/api/torrents/filter?page=4","label":"4","active":false},{"url":"https://unit3d.site/api/torrents/filter?page=5","label":"5","active":false},{"url":"https://unit3d.site/api/torrents/filter?page=6","label":"6","active":false},{"url":"https://unit3d.site/api/torrents/filter?page=7","label":"7","active":false},{"url":"https://unit3d.site/api/torrents/filter?page=8","label":"8","active":false},{"url":"https://unit3d.site/api/torrents/filter?page=9","label":"9","active":false},{"url":"https://unit3d.site/api/torrents/filter?page=10","label":"10","active":false},{"url":null,"label":"...","active":false},{"url":"https://unit3d.site/api/torrents/filter?page=278","label":"278","active":false},{"url":"https://unit3d.site/api/torrents/filter?page=279","label":"279","active":false},{"url":"https://unit3d.site/api/torrents/filter?page=2","label":"Next »","active":false}],"path":"https://unit3d.site/api/torrents/filter","per_page":"1","to":1,"total":279}}
garfield69 commented 3 years ago

what size poster is recommended for the Jackett UI?

Up to now we have been using what ever size poster the web site returns. The current w92 posters from Blutopia are fine, and anything up to w180 is acceptable. Anything much larger than that starts to take up too much real estate on the Jackett dashboard search results page ;-)

And is anything else useful like genres, etc?

The response example you just provided in https://github.com/Jackett/Jackett/issues/8682#issuecomment-948212414 covers all the torznab attributes we need to return to apps. So at this stage I think we can call it good. Thank you.

HDVinnie commented 3 years ago

@garfield69 ok cool. 

            'poster_big'   => 'w500',
            'poster_mid'   => 'w342',
            'poster_small' => 'w92',

Thats what I have to offer for poster sizes.....API is currently returning mid. Want me to update it too small....BLU is live with this. Once confirmed good ill publish it to UNIT3D core so other sites can update.

garfield69 commented 3 years ago

poster_small is best of the three choices thank you.

HDVinnie commented 3 years ago

Done. Thanks @garfield69 and @mikeoscar2006 for adding this.

garfield69 commented 3 years ago

@HDVinnie the BrSociety site (currently on unit3d 5.3.0) have magnet alongside download_link on their results page. Would it be possible to add magnet to the API when it is available at a site, and leave it out when its not?

HDVinnie commented 3 years ago

@HDVinnie the BrSociety site (currently on unit3d 5.3.0) have magnet alongside download_link on their results page. Would it be possible to add magnet to the API when it is available at a site, and leave it out when its not?

https://github.com/HDInnovations/UNIT3D-Community-Edition/commit/a62fe5d4dc6cc02195e7ce06f277929aeaa74b15

garfield69 commented 3 years ago

@HDVinnie Awesome. Thank you.

ilike2burnthing commented 3 years ago

sortField, sortDirection, and perPage don't appear to be having any affect on the ones I've tested.

DesiTorrents is returning 500 | SERVER ERROR for the keywordless search. % seems to be accepted as a wildcard.

garfield69 commented 3 years ago

that is correct. the new query parms will only take effect when the sites upgrade their engines. same applies to the result field Poster. in the mean time the indexer is ready to use them when they do upgrade and are ignored by the current engines.

garfield69 commented 3 years ago

desitorrents are running a custom flavour of 5.3.0 so it does not surprise me if the api indexer for that site needs finessing.

As I don't have an account, I created the api solely based on the 5.3.0 model (blutopia) that I use for all base tests on the assumption that is ought to work ;-)

Are you able to update and test the desitorrents api indexer or would you prefer to let me borrow a/c while I adjust/test ?

ilike2burnthing commented 3 years ago

I think the only thing needing changed is the wildcard, but I'll double-check.

ilike2burnthing commented 3 years ago

aee0f2b5e3898481fcdf066f61a3abda6e678b4a

ilike2burnthing commented 2 years ago

Can this now be closed?

garfield69 commented 2 years ago

not yet. I still have several unit3d api indexers on hold pending completion of PR #12464 after which I will close.

HDVinnie commented 2 years ago

Hello @garfield69,

Please drop support for the scraper implementation of jackett on Blutopia. We have a fully functioning api that replicates the scraper functionality that can be used instead. Allowing the scraper causes unnecessary load as well increased potential for the user's account to be compromised.

Thanks, Blutopia Staff

PS: This should really go for any UNIT3D tracker running v5.3.0 or newer.

garfield69 commented 2 years ago

Please drop support for the scraper implementation of jackett [...] for any UNIT3D tracker running v5.3.0 or newer.

Okely Dokely. Will roll this out for the next Jackett release in about 5 hours.

garfield69 commented 2 years ago

list of HTML indexers that are due to be dropped in favour of the API

HDVinnie commented 2 years ago

Please drop support for the scraper implementation of jackett [...] for any UNIT3D tracker running v5.3.0 or newer.

Okely Dokely. Will roll this out for the next Jackett release in about 5 hours.

Thanks for understanding

ngosang commented 2 years ago

@HDVinnie could you include the "category id" in the API response? We are using the "category" field but parsing strings is harder than numbers. Also the numbers change less frequently.

HDVinnie commented 2 years ago

@HDVinnie could you include the "category id" in the API response? We are using the "category" field but parsing strings is harder than numbers. Also the numbers change less frequently.

Just seeing this.

Its as follows now:

{
    "type": "torrent",
    "id": "122101",
    "attributes": {
        "poster": "https://image.tmdb.org/t/p/w92/sav0jxhqiH0bPr2vZFU0Kjt2nZL.jpg",
        "name": "Citizen Kane 1941 1080p Criterion Blu-ray AVC LPCM 1.0",
        "release_year": "1941",
        "category": "Movie",
        "type": "Full Disc",
        "resolution": "1080p",
        "distributor": "Criterion",
        "region": "USA",
        "media_info": null,
        "bd_info": "Disc Title: CITIZEN KANE - DISC 1\r\nDisc Label: CITIZEN_KANE_DISC01\r\nDisc Size: 36,805,390,694 bytes\r\nProtection: AACS\r\nPlaylist: 00001.MPLS\r\nSize: 36,425,717,760 bytes\r\nLength: 1:59:33.124\r\nTotal Bitrate: 40.62 Mbps\r\nVideo: MPEG-4 AVC Video / 35743 kbps / 1080p / 23.976 fps / 16:9 / High Profile 4.1\r\nAudio: English / LPCM Audio / 1.0 / 48 kHz /  1152 kbps / 24-bit\r\nAudio: English / Dolby Digital Audio / 1.0 / 48 kHz /   192 kbps / DN -31dB\r\nAudio: English / Dolby Digital Audio / 1.0 / 48 kHz /   192 kbps / DN -31dB\r\nAudio: English / Dolby Digital Audio / 1.0 / 48 kHz /   192 kbps / DN -31dB\r\nSubtitle: English / 71.606 kbps\r\n\r\nDisc Title: CITIZEN KANE - DISC 2\r\nDisc Label: CITIZEN_KANE_DISC02\r\nDisc Size: 44,788,521,541 bytes\r\nProtection: AACS\r\nPlaylist: 00800.MPLS\r\nSize: 18,117,101,568 bytes\r\nLength: 1:35:16.001\r\nTotal Bitrate: 25.36 Mbps\r\nVideo: MPEG-4 AVC Video / 24000 kbps / 1080p / 23.976 fps / 16:9 / High Profile 4.1\r\nAudio: English / Dolby Digital Audio / 1.0 / 48 kHz /   192 kbps / DN -31dB\r\n\r\nDisc Title: CITIZEN KANE - DISC 3\r\nDisc Label: CITIZEN_KANE_DISC03\r\nDisc Size: 44,582,887,473 bytes\r\nProtection: AACS\r\nPlaylist: 00807.MPLS\r\nSize: 2,203,490,304 bytes\r\nLength: 0:53:13.815\r\nTotal Bitrate: 5.52 Mbps\r\nVideo: MPEG-4 AVC Video / 5000 kbps / 1080p / 23.976 fps / 16:9 / High Profile 4.1\r\nAudio: English / Dolby Digital Audio / 1.0 / 48 kHz /   192 kbps / DN -31dB",
        "description": "SPECIAL FEATURES\r\n\r\n- New 4K digital restoration, with uncompressed monaural soundtrack\r\n- Three audio commentaries: from 2021 featuring Orson Welles scholars James Naremore and Jonathan Rosenbaum; from 2002 featuring filmmaker Peter Bogdanovich; and from 2002 featuring film critic Roger Ebert\r\n- The Complete \u201cCitizen Kane,\u201d (1991), a rarely seen feature-length BBC documentary\r\n- New interviews with critic Farran Smith Nehme and film scholar Racquel J. Gates\r\n- New video essay by Orson Welles scholar Robert Carringer\r\n- New program on the film\u2019s special effects by film scholars and effects experts Craig Barron and Ben Burtt\r\n- Interviews from 1990 with editor Robert Wise; actor Ruth Warrick; optical-effects designer Linwood Dunn; Bogdanovich; filmmakers Martin Scorsese, Henry Jaglom, Martin Ritt, and Frank Marshall; and cinematographers Allen Daviau, Gary Graver, and Vilmos Zsigmond\r\n- New documentary featuring archival interviews with Welles\r\n- Interviews with actor Joseph Cotten from 1966 and 1975\r\n- The Hearts of Age, a brief silent film made by Welles as a student in 1934\r\n- Television programs from 1979 and 1988 featuring appearances by Welles and Mercury Theatre producer John Houseman\r\n- Program featuring a 1996 interview with actor William Alland on his collaborations with Welles\r\n- Selection of The Mercury Theatre on the Air radio plays featuring many of the actors from Citizen Kane\r\n- Trailer\r\n\r\n\r\n[url=https://ibb.co/xFMKLws][img]https://i.ibb.co/xFMKLws/00274-m2ts-snapshot-00-07-55-464.png[/img][/url] [url=https://ibb.co/CQQCfGP][img]https://i.ibb.co/CQQCfGP/00274-m2ts-snapshot-00-14-02-358.png[/img][/url] [url=https://ibb.co/kXhFCVp][img]https://i.ibb.co/kXhFCVp/00274-m2ts-snapshot-00-22-11-417.png[/img][/url] [url=https://ibb.co/djVnZFg][img]https://i.ibb.co/djVnZFg/00274-m2ts-snapshot-00-34-47-842.png[/img][/url] [url=https://ibb.co/wBnYzz0][img]https://i.ibb.co/wBnYzz0/00274-m2ts-snapshot-00-43-24-415.png[/img][/url]\r\n\r\n[url=https://ibb.co/QvbL0DF][img]https://i.ibb.co/QvbL0DF/00274-m2ts-snapshot-00-47-56-023.png[/img][/url] [url=https://ibb.co/bzHC03G][img]https://i.ibb.co/bzHC03G/00274-m2ts-snapshot-00-58-41-773.png[/img][/url] [url=https://ibb.co/6r475jS][img]https://i.ibb.co/6r475jS/00274-m2ts-snapshot-01-11-55-339.png[/img][/url] [url=https://ibb.co/K9JFh4R][img]https://i.ibb.co/K9JFh4R/00274-m2ts-snapshot-01-38-20-279.png[/img][/url] [url=https://ibb.co/1X03pJZ][img]https://i.ibb.co/1X03pJZ/00274-m2ts-snapshot-01-56-39-729.png[/img][/url]",
        "info_hash": "d86816f8a05085714395145a81277e00cc6a34ea",
        "size": 126176799708,
        "num_file": 406,
        "freeleech": "0%",
        "double_upload": 0,
        "internal": 0,
        "uploader": "xxx",
        "seeders": 15,
        "leechers": 0,
        "times_completed": 56,
        "tmdb_id": "15",
        "imdb_id": "0033467",
        "tvdb_id": "0",
        "mal_id": "0",
        "igdb_id": "0",
        "category_id": 1,
        "type_id": 1,
        "resolution_id": 2,
        "distributor_id": 218,
        "region_id": 229,
        "created_at": "2022-02-01T20:07:32.000000Z",
        "download_link": "xxxx",
        "details_link": "xxxx"
    }
}
ngosang commented 2 years ago

Thank you! We will update the definitions when the sites upgrade the API.

ilike2burnthing commented 2 years ago

@HDVinnie can you include a genre name (not ID) attribute?

Is it possible to search by genre name? Thanks.

HDVinnie commented 2 years ago

I can include genre name....however you can only search by genre ids which would be the same across all UNIT3D sites.

     all: [
       App\Models\Genre {#4041
         id: 28,
         name: "Action",
       },
       App\Models\Genre {#4042
         id: 10759,
         name: "Action & Adventure",
       },
       App\Models\Genre {#4043
         id: 12,
         name: "Adventure",
       },
       App\Models\Genre {#4044
         id: 16,
         name: "Animation",
       },
       App\Models\Genre {#4045
         id: 35,
         name: "Comedy",
       },
       App\Models\Genre {#4046
         id: 80,
         name: "Crime",
       },
       App\Models\Genre {#4047
         id: 99,
         name: "Documentary",
       },
       App\Models\Genre {#4048
         id: 18,
         name: "Drama",
       },
       App\Models\Genre {#4049
         id: 10751,
         name: "Family",
       },
       App\Models\Genre {#4050
         id: 14,
         name: "Fantasy",
       },
       App\Models\Genre {#4051
         id: 36,
         name: "History",
       },
       App\Models\Genre {#4052
         id: 27,
         name: "Horror",
       },
       App\Models\Genre {#4053
         id: 10762,
         name: "Kids",
       },
       App\Models\Genre {#4054
         id: 10402,
         name: "Music",
       },
       App\Models\Genre {#4055
         id: 9648,
         name: "Mystery",
       },
       App\Models\Genre {#4056
         id: 10763,
         name: "News",
       },
       App\Models\Genre {#4057
         id: 10764,
         name: "Reality",
       },
       App\Models\Genre {#4058
         id: 10749,
         name: "Romance",
       },
       App\Models\Genre {#4059
         id: 10765,
         name: "Sci-Fi & Fantasy",
       },
       App\Models\Genre {#4060
         id: 878,
         name: "Science Fiction",
       },
       App\Models\Genre {#4061
         id: 10766,
         name: "Soap",
       },
       App\Models\Genre {#4062
         id: 10767,
         name: "Talk",
       },
       App\Models\Genre {#4063
         id: 53,
         name: "Thriller",
       },
       App\Models\Genre {#4064
         id: 10770,
         name: "TV Movie",
       },
       App\Models\Genre {#4065
         id: 10752,
         name: "War",
       },
       App\Models\Genre {#4066
         id: 10768,
         name: "War & Politics",
       },
       App\Models\Genre {#4067
         id: 37,
         name: "Western",
       },
     ],

https://unit3d.site/api/torrents/filter?api_token=xxx&genres[]=10749 would return all Romance torrents.

ilike2burnthing commented 2 years ago

Thanks. That should work too.

Let me know when the genre attribute gets rolled out to trackers, may as well do the search capabilities and results fields at the same time. No rush.

HDVinnie commented 2 years ago

@garfield69 @ilike2burnthing lmk if whats now live on BLU works for you. 

...
"meta": {
    "type": "Movie",
    "poster": "https://image.tmdb.org/t/p/w92/9eFULnzgoLpO7lvg6FMutGRuNFg.jpg",
    "genres": "Drama, Romance"
 },
...

Also if any other meta is needed lmk

ilike2burnthing commented 2 years ago

Many thanks.

https://github.com/Jackett/Jackett/pull/13406

garfield69 commented 2 years ago

@HDVinnie yes, I can pull meta.genres from the results, although I am confused about meta.type since that iseems to be returning a category not a type as defined by the web site's torrents page advanced search selection?

garfield69 commented 2 years ago

never mind, I see that attributes.type is still there so all good.