Feature Request: Add "Referer" url info in requests

[Micdu70]

Since ruTorrent can handle URL links itself (via "php/addtorrent.php") to download torrents, I really would like an option to send, without any checking, the URL link to download torrents.

[bogenpirat]

yeah, adding just a URL would've made writing this extension a lot easier. but getting the torrent and uploading it directly was the primary focus of the extension, because that way, it works as an interface between a website serving a torrent, which may require special credentials that the browser knows how to handle, and a website accepting a torrent that is served by a torrent client, which may not necessarily be able to even access the torrent link in question.

a quick example: you're getting a torrent from a private tracker that requires a login (username/password; a resulting session is stored in cookies in the browser). you're trying to add a .torrent URL from this tracker to a qbittorrent client, but the client cannot access it because it doesn't (1) have a valid session to access that url and (2) doesn't know how to log into the tracker and (3) doesn't even have credentials to it.

out of curiosity, what's the issue with the way it works now?

[Micdu70]

I speak about ruTorrent WebUI only because you can set cookies (or login for many private trackers) in ruTorrent settings.

The issue is from one specific french private tracker: To be able to download .torrent files, you need to also send a "referer" url (domain name of the tracker is enough to make it working) in the request. For now, your extension can't get the .torrent file on this tracker.

[bogenpirat]

alright. generally speaking, adding a "send url" method just for rutorrent would imo disrupt the ease of use, so i'd rather not do that. you'd have to change torrent-getting-behavior based on the client, and those are control flows that i'd prefer to keep separate.

it does however make sense to fix the referrer issue. let me look into this.

[bogenpirat]

this worked fine in my tests. mind trying it out for me? instructions: https://github.com/bogenpirat/remote-torrent-adder/wiki/Try-new-code

[Micdu70]

It's not working on this tracker:

[bogenpirat]

chrome://extensions/ <- type that into your address bar. what does the background page view of the unpacked RTA extension say? any errors?

[Micdu70]

No error, everything looks fine =>

Your extension:

== Request ==

GET /actions.php?torrent=Name HTTP/1.1
Host: www.tracker.fr
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36
Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: fr-FR,fr;q=0.9,en-US;q=0.8,en;q=0.7
Cookie: PHPSESSID=123456789
referer: http://www.tracker.fr/actions.php?torrent=Name
origin: http://www.tracker.fr/actions.php?torrent=Name

== Answer ==

HTTP/1.1 200 OK
Date: Tue, 18 Aug 2020 21:33:30 GMT
Server: Apache
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Disposition: attachment; filename="Name.torrent"
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: application/x-bittorrent

Without extension:

== Request ==

GET /actions.php?torrent=Name HTTP/1.1
Host: www.tracker.fr
Connection: keep-alive
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Referer: http://www.tracker.fr/home.php
Accept-Encoding: gzip, deflate
Accept-Language: fr-FR,fr;q=0.9,en-US;q=0.8,en;q=0.7
Cookie: PHPSESSID=123456789

== Answer ==

HTTP/1.1 200 OK
Date: Tue, 18 Aug 2020 21:37:34 GMT
Server: Apache
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Disposition: attachment; filename="Name.torrent"
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: application/x-bittorrent
X-DNS-Prefetch-Control: off

EDIT: For latest release in Chrome Web Store:

== Answer ==

HTTP/1.1 200 OK
Date: Tue, 18 Aug 2020 22:02:26 GMT
Server: Apache
Vary: Accept-Encoding
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked

[bogenpirat]

alright, so the new code does work; both with the extension disabled, as well as with the unpacked extension enabled, you're receiving a torrent file. only the chrome web store version gives you an error message. that's all assuming that the response Content-Type header is accurate.

i'm not sure where exactly in the code the error is thrown. adding torrents works from other sites for you?

in devtools -> network -> (try adding a torrent) -> check the request's response tab: does the data begin with a line similar to d8:announce, followed by some tracker url? if so, the data should be correct.

if the answer to these questions is "yes", we'll need to dig a little deeper into what exactly is going wrong. i suspect it has something to do with these three lines: https://github.com/bogenpirat/remote-torrent-adder/blob/issue-285/miscapis/functions.js#L77.

since getting/adding torrents works fine for me, i suspect this is a quirk with this particular server. debugging without access to it, and thus being unable to replicate the error is going to be very difficult.

[Micdu70]

Yes, other sites are still working.

Yes, data is correct.

Anyway thanks for trying. ^^

[Micdu70]

To me, you can merge "issue-285" branch in "master" and close this issue if you want. ;)

EDIT: I added an option to always send URLs to ruTorrent WebUI, it's fine for me ^^ => https://github.com/bogenpirat/remote-torrent-adder/commit/0bb96c81165edbafee56a0803f57dd524b08b480

[bogenpirat]

hmm. i think i will. your fix seems good, care to open a pull request? i'll wait a bit before releasing it to the chrome web store, though.

[Micdu70]

Done. ;)

[bogenpirat]

sorry for the delay, merged.

closing this one since it appears to be working (albeit not for you specifically).

for historical context: i added referer rewriting for torrent links, micdu70 added an option to just send torrent links to the UI.