Gks Tracker plugin development

[Torf]

Hello,

I (try to) develop a Gks Indexer plugin in order to search in the Gks tracker database for movies, games & music but I haven't success to use my plugin yet.

I wrote a plugin similar to Newznab but for"torrent" instead of "nzb". May you say me what i'm doing wrong, please ?

My plugin : https://github.com/Torf/XDM-Gks

Edit: I have just realized that XDM needs a provider in order to search in an indexer. So my problem is solved. Another point, I have multiples Gks categories for each XDM category, have I to do a trick (link each XDM category to the right GKS categories) ?

Thanks, Torf.

[lad1337]

i just tested you provider and i noticed that the torrent files are empty since the "url" is the link to the page and not to the file ... which is needed for the black hole plugin at least

what do you mean with XDM category ? such as "Category for Books" if so yes i would use some kind of list "," separation if i where you if you want to allow multiple GKS categories

edit: for the download links i think you have to create these "https://gks.gs/private-get/" links with the id and authkey

[Torf]

For the torrent link, i saw that and i will to use the right link to search and get it.

And for the categories, i finally use the coma separation, like you said, thanks.

I'm wondering if XDM is made for advance filtering (with a filter plugin), after the search, in order to choose the right torrent according to parameters that are linked to torrents info (seeders count, Freeleech, file size...) ?

[lad1337]

only these fields are given right now

name = CharField() # name of the "post"
url = CharField(unique=True) # url to get the fiel / container
size = IntegerField(True) # size in kb
status = ForeignKeyField(Status) # XDM internal
type = CharField() # set by the indexer
indexer = CharField(True) # automatically set from XDM
indexer_instance = CharField(True) # automatically set from XDM
external_id = CharField(True) # the external id
pp_log = TextField(True) # the post process log from an post processor plugin

but i planed and implemented an "extra_data" field (at leas i could have sworn i did) that can hold any information that is given and the filter plugins can look / use this additional information but this is not yet implemented!

[Torf]

but i planed and implemented an "extra_data" field (at leas i could have sworn i did) that can hold any information that is given and the filter plugins can look / use this additional information but this is not yet implemented!

It is a good idea :) I'm waiting it.

But.. wait.. I saw in source that download size is in bytes. In what file is this code ?

I get the URL that i set in the download.url. This URL works fine if I do a HTTPS GET REQUEST of it. But your blackhole plugin still creating a blank file.

[lad1337]

I get the URL that i set in the download.url. This URL works fine if I do a HTTPS GET REQUEST of it. But your blackhole plugin still creating a blank file.

that is weird the blackhole plugin does a simple write from the data it gets from that url https://github.com/lad1337/XDM-main-plugin-repo/blob/master/Blackhole/Blackhole.py#L47 i will try it myself again and have a deeper look at the blackhole plugin

But.. wait.. I saw in source that download size is in bytes. In what file is this code ?

you are right it is in bytes, not kb! https://github.com/lad1337/XDM/blob/master/xdm/classes.py#L814

[Torf]

that is weird the blackhole plugin does a simple write from the data it gets from that url

Yes really weird, i saw some similars issues in stackoverflow but I've tried some scripts with the same result. (I am sure my link is OK, I have checked and test with wfetch) I don't understand because the script receive the begin of the response (HTTP 1.1 200 OK) but it seems that it doesn't read the content. Gks give in the header the correct content size and the script create a file at the correct size. I think the probleme is when you read the response content. There is a "unicode" parameter for iter_content... I will test some others scripts..

[lad1337]

it works for me now, files are created and some online tool accepts them as valid torrent files

but i noticed that the filename is not cleaned from the blackhole plugin changed that in version 0.4

what i also noticed is that the gks plugin returns a lot of results that don't really match the searched movie this will create a lot of false positives since XDM itself does not validate the results

[Torf]

what i also noticed is that the gks plugin returns a lot of results that don't really match the searched movie this will create a lot of false positives since XDM itself does not validate the results

I also notice that, XDM (or a filter plugin ?) will handle that or it's at the indexer to select only downloads that match with the search ?

changed that in version 0.4

Ok thx, I will test

Edit: Still empty .torrent for me (w/ Ubuntu) :/

[lad1337]

I also notice that, XDM (or a filter plugin ?) will handle that or it's at the indexer to select only downloads that match with the search ?

technically this is entirely your choice but i would refine the search terms / search arguments as much as possible to get only results that make sense especially because there is no filter plugin that would evaluate the downloads as of yet

i saw you use the getSearchTerms() which is a good choice to start with but if you notice that this does not help with better results you can use the available fields of that element

e.g. a movie from de.lad1337.movies and de.lad1337.tmdb

element.name # returns the title / name of the movie
element.year # returns the year the movie was released in

now while a "." call like will return any value of a field with that name if you want the information from a specific provider you will have to use

element.getField('name', 'tmdb') # returns the value if the name field from the provider with the tag tmdb or None

this gets interesting for external ids

element.getField("id", "tmdb") # get the tmdb movie id
element.getField("id", "imdb") # get the IMDB movie id

yes the tmdb provider also adds the IMDB id to a movie https://github.com/lad1337/XDM-main-plugin-repo/blob/master/TheMovieDB/TheMovieDB.py#L88

[Torf]

as much as possible to get only results that make sense

Moreover, i notice if a downloader give, for example 200 results, XDM takes a lot of time to analyse them. I will add, at least, a filter on the upload name (that have to contains all words of the movie/game/ebook/music title).

e.g. a movie from de.lad1337.movies and de.lad1337.tmdb

Thanks you for this information, but I think, in this case, using tmdb informations is complicated. And if an user doesn't install the TMDB plugin but another one ? But I would like to search others language titles (french by example) to add more results.

Another probleme with movies is how to choose if I download VOST or only good audio language ?

And do you have any idea about this please : Still empty .torrent for me (w/ Ubuntu) :/ ?