qBT does not call the "torrent_download" method for any search plugin

[aleqx]

I'm writing a custom search plugin and I've successfully implemented the search method and can get the search results correctly into qBittorrent. So that part is fine.

The site I'm getting results from requires me to go to a 2nd html page to get the .torrent file or magnet: link, which I must parse.

The wiki says I can use the download_torrent method for that, but when I double click on a search result in qBittorrent for this plugin, the download_torrent method is not called at all.

EDIT: In fact, nova2dl.py is not invoked at all by qBT, which is where the download_torrent method is called ... looks like qBT downloads the 1st entry in the info string directly, instead of via nova2dl.py.

I'm using Windows,Python 2.7.11, qBT 3.3.3

What am I missing?

[nbusseneau]

I understand your reasoning, sadly I fear this will only work with a couple of trackers. In my case, I will wait the DownloadManager upgrade to see if I can make use of it for rutracker.org :)

[glassez]

@Skymirrh, I have another question. What to do with cookies set from JavaScript code? The issue author has also highlighted the need to deal with such cookies.

[sledgehammer999]

This is a bit out of scope: The search subsystem doesn't strive to be a solution for every possible search engine. If the search engine does various tricks to disallow automatic parsing then it effectively says "leave my site alone" and we somewhat need to respect that. PS: What the above means? Yes, each plugin author can write whatever he wants in the plugins to parse the site. But we don't have to extend DownloadManager(C++) in a such a way to facilitate this. If it cannot be done within the confines of python plugins then leave it be.

[nbusseneau]

I agree with @sledgehammer999, if Python does the job just fine for specific use cases (such as JavaScript parsing) then there is no reason to try and bundle a bunch of behaviors in the DownloadManager.

In that way, and when the DownloadManager new scripting system is ready, qBittorrent users could make use of the official plugins without needing to install Python, and would only need to do so if they wish to use specific engines for which Python plugins are required.

[glassez]

Guys, I think we are talking about different things. More precisely, about the different sides of the same thing. I suggest to end this here, because the solution for this problem ready. As for my suggestions, let's discuss them after the fact (when I publish my PR).

[aleqx]

@sledgehammer999 Yes I did unsubscribe but somehow I started receiving notifications again today ... weird.

I love qBT and I respect your coding, but I just have to say that your proposal doesn't make sense to me. Why maintain two separate pieces of code to do the same thing when you can (and already do!) have a single one that does it.

I simply don't understand this obsession to move away from Python - especially after having developed plugins in it which work just fine and having nobody complained about it. This whole discussion is just surreal to me.

@Skymirrh, I have another question. What to do with cookies set from JavaScript code? The issue author has also highlighted the need to deal with such cookies.

This is a bit out of scope: The search subsystem doesn't strive to be a solution for every possible search engine. If the search engine does various tricks to disallow automatic parsing then it effectively says "leave my site alone" and we somewhat need to respect that.

With all due respect, that's just wrong, principle wise and technically. Give the plugin author the freedom to design his plugin as he sees fit and allow him to make it work an any site he wants. Plugins should be allowed to do EVERYTHING that a user can do in a browser. Limiting that, especially after allow it for so long, is odd, to say the very least. This is taking a sad turn.

You're wrong about rarbg: it does not disallow automatic parsing (they would have to ban browsers too if that was the case) but disallow search bots from issuing and indexing hundreds of pages at once. The search from qBT is a manual search, and is no different than what the user would do in browser. In fact, it's lighter than a browser since images, js, css are not fetched.

Currently you guys just broke existing and useful functionality and so far the reasons for not bringing it back are lacking. Why was nova2dl removed in the first place and why it's not simply brought back is beyond me. Why not just embed a small python distro and be done with all this?

Anyway, I said my piece. I see nova2dl is not going to be brought back and you are against parsing js cookies (another decision that is beyond me) so I don't see a reason to continue discussing. I may just do a fork when I have the time and re-implement calling nova2dl myself, hoping to automate a patch from the main qBT repo.

[zeule]

@aleqx there is PR #5085 which reverts nova2dl removing. To encourage contributions projects need to accept not only code which is strictly needed, but also the code which keeps the contributors interested in the project development (provided that it does not break anything :) ). Thus if @glassez wants to implement another search plugins system why stop him? Speaking of old python plugins I would even lift restrictions for such plugins and consider as search plugin just any executable in a specific directory. nova.py code gives a framework for wrinting python plugins, but anything that can print to stdout correctly formatted text could work just as fine.

[nbusseneau]

@aleqx I think the goal of separating plugins into two systems (one embedded and one using Python) is to ensure anybody downloading qBittorrent will be able to at least make use of the official search engines, without needing to download and install Python. Of course a lot of people will have Python installed separately anyway (notably Linux users), but for regular Windows users who just need a torrent client and won't make use of Python, it will be an improvement.

I actually agree with your idea of embedding Python and think this is the most simple idea, however this is not the path that was chosen for qBittorrent. So I will settle with having on one side official and simple plugins using some sort of inhouse scripting system, and advanced Python plugins on the other side, as it still allows for flexibility when absolutely needed.

[glassez]

Well, the existence of two plugin subsystems is overkill. But we really can transform an existing system into a certain "Convention on cooperation" (a kind of simple Protocol) to call any executable as a search engine. Just to add to the menu something like "Run external search engine".

[sledgehammer999]

@aleqx nova2dl will be added back. PR is ready, I just need to review and merge it.

With all due respect, that's just wrong, principle wise and technically. Give the plugin author the freedom to design his plugin as he sees fit and allow him to make it work an any site he wants. Plugins should be allowed to do EVERYTHING that a user can do in a browser. Limiting that, especially after allow it for so long, is odd, to say the very least. This is taking a sad turn.

I was talking about something else. If the parsing can't be done with Python then we don't care about that site. Above my comment people were talking about transfering some logic into qbittorrent and I was pointing out that I don't want to bloat the C++ code to facilitate the search plugin system. I was saying that if more special parsing is needed then do it in Python. Don't expect the C++ downloadmanager to have great capabilities.

To all others: Search has been working great so far. I don't consider having Python as a dynamic dependency a bad thing. Also I don't want to implement a half-assed scritpting language just to get rid of that dependency. qBittorrent needs developer time focused on other things. Search works fine.

I don't think we have something else to discuss here. I am going to review the pending PR soon and merge it.

[aleqx]

I was just about to fork qBT to get nova2dl back into it and wanted to check the status of this issue that I started.

I'm sorry for jumping the gun in some of the messages above - I don't usually feel that strongly about such issues. You are right. It's nice to have a basic search that works even without Python, and to keep the python capability for anything more fancy.

So is nova2dl now back in 3.3.4? I haven't tried 3.3.4 yet.

Also, I just noticed someone submitted a rarbg search plugin on April 18, 2016 (one of you guys?) but it uses the API from rarbg which doesn't provide any information about size, seeds, etc. I wrote a fully working rarbg search plugin for nova2dl that uses the main website and returns all info, and correctly handles all cookies and page redirects etc. I can add it to the list but I don't know if you guys have a policy for duplicates? I don't see any duplicates in there.

EDIT: it looks like the rarbg api now returns size and seeds as well (limited to 100 results).

[glassez]

I was just about to fork qBT to get nova2dl back into it and wanted to check the status of this issue that I started.

This issue is closed (fixed) after v3.3.4 was released.

[sledgehammer999]

The fix is in git master and will be included in v3.3.5, which I might release this week. There isn't a rarbg in the official release(or in the git repo). I suspect it is in the wiki page of unofficial plugins. Then you can post your own as an additional entry there.