403 error when processing result from what

[bruut]

I needed to reinstall my system (including windows) so I started with a fresh install of python 2.7 (32bit) and headphones (latest from source) On my previous system, everything worked correctly, could download without problems.

Now, when I try to download an album using what, it finds the correct torrent, but gives my the following error during rpeprocessing:

21-Jan-2014 19:52:14 - INFO :: Thread-12 : Found 2 matching releases from What.c
d for Amateur Transplants - Unfit to Practise after filtering
21-Jan-2014 19:52:14 - INFO :: Thread-12 : Sorting torrents by times snatched an
d preferred bitrate None...
21-Jan-2014 19:52:14 - INFO :: Thread-12 : New order: Torrent: Unfit To Practise
 - 320 - ID: 29576904, Torrent: Unfit To Practise - V0 (VBR) - ID: 467548
21-Jan-2014 19:52:20 - INFO :: Thread-12 : Pre-processing result
Exception in thread Thread-12:
Traceback (most recent call last):
  File "H:\Python27\lib\threading.py", line 810, in __bootstrap_inner
    self.run()
  File "H:\Python27\lib\threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "H:\Python programs\headphones\headphones\searcher.py", line 132, in sear
chforalbum
    searchTorrent(result['AlbumID'], new)
  File "H:\Python programs\headphones\headphones\searcher.py", line 1467, in sea
rchTorrent
    (data, bestqual) = preprocesstorrent(torrentlist, pre_sorted_results)
  File "H:\Python programs\headphones\headphones\searcher.py", line 1556, in pre
processtorrent
    response = urllib2.urlopen(request)
  File "H:\Python27\lib\urllib2.py", line 127, in urlopen
    return _opener.open(url, data, timeout)
  File "H:\Python27\lib\urllib2.py", line 410, in open
    response = meth(req, response)
  File "H:\Python27\lib\urllib2.py", line 523, in http_response
    'http', request, response, code, msg, hdrs)
  File "H:\Python27\lib\urllib2.py", line 448, in error
    return self._call_chain(*args)
  File "H:\Python27\lib\urllib2.py", line 382, in _call_chain
    result = func(*args)
  File "H:\Python27\lib\urllib2.py", line 531, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 403: Forbidden

Any ideas? Login to what is OK, I can manually download the torrent without problems.

[bruut]

Reading the forums on What, this is probably an effect of the new cloudflare anti ddos protection system... No solution yet over there

[bruut]

Still having the same issue. Logging in is fine, searching the torrents is fine, but preprocessing/downloading the torrent is where it fails.

[evanjd]

Apologies, didn't read your problem correctly. Deleted my original comment.

I think I can see why it works for me, but fails for you. When Headphones.py uses Transmission or uTorrent (I use the former), it drops out of the preprocesstorrent function immediately, it instead forwards the URL to transmission for it to download. If you're using another client/black hole, headphones will attempt to download the .torrent file for you.

From a glance, it looks like headphones.py needs to include the special cloudflare cookie (__cfduid) and/or the what.cd session cookie for the download to succeed.

I'll take a look at this later, this might be an easy fix.

[evanjd]

OK, fixed. I've added the revision to my pull request: https://github.com/rembo10/headphones/pull/1407, or you can view the commit here: https://github.com/evanjd/headphones/commit/afe9a0259e13106b5111ec9cb537ff691a7835c2

I was wrong about the problem. Cookies aren't an issue, but cloudflare/what rejects requests without a user-agent.

[bruut]

Yes! Thank you for that. It runs nicely now. I am indeed using uTorrent via black hole method. Ty!