Saved filename is wrong when feed item points to a redirecting URL

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Add a feed that consists of items of redirecting URL:s instead of the final 
URL (in my case: podcast from sr.se).
2. When the file is downloaded, it will be saved as the filename of the initial 
URL.

In this case this gives a downloaded filename like "2853847.mp3" instead of the 
more descriptive "SR_p3_tankesmedjan_110118121110.mp3". In worse cases (other 
sites) this can give the filename "download.php" instead of "file.mp3", which 
makes the "noClobber" option necessary, but the resulting filenames will just 
be "download.php.1" and so on.

The filename should be decided from the final URL instead of the initial.

I attach a patch where I implemented this. I wrote it in as a boolean 
global+thread option "nameFromRedirect". "True" makes rssdler decide the 
filename from the final redirected URL, "False" leaves the behaviour as before. 
I let the default value be global:"False"/thread:"None" to not alter the 
standard behaviour.

The option could very well be implemented as standard without a choice, but 
since it results in a few extra HTTP requests and there very well might be bugs 
in the functionality that I haven't encountered yet, it should be more well 
tested before that should happen.

Using rssdler 0.42 on Debian Sid.

Original issue reported on code.google.com by daniel.r...@gmail.com on 18 Jan 2011 at 4:18

Attachments:

• rssdler.py.patch-nameFromRedirect

[GoogleCodeExporter]

I havnt tested the patch but I'm running into this a lot on http://www.kat.ph 
their rss feeds dont point straight to the torrent file example 
http://www.kat.ph/torrents/family-guy-s10e04-hdtv-xvid-lol-t5960718/ it 
redirects to the actual torrent file but rssdler doesnt seem to understand this 
behavior and saves .html file instead of the .torrent file

Original comment by lsell...@gmail.com on 7 Nov 2011 at 8:24

[GoogleCodeExporter]

Tried out the patch, but it didn't help. Perhaps we have two different issues.

Original comment by lsell...@gmail.com on 7 Nov 2011 at 9:50

[GoogleCodeExporter]

Did you also enable the configuration option "nameFromRedirect" as indicated in 
my post for the RSS feed in question? Are you sure your new configuration was 
read?

Original comment by daniel.r...@gmail.com on 7 Nov 2011 at 9:54

[GoogleCodeExporter]

Sure enough I had made a temp config file for testing the patch, but ran my 
test with out specifying that config. Retested and it works perfect. Thanks for 
the fix I hope it gets merged. 

Original comment by lsell...@gmail.com on 7 Nov 2011 at 10:05

[GoogleCodeExporter]

one thing I did notice is even with the patch it will not fetch the correct 
file size example 
http://www.ezrss.it/search/index.php?show_name=Family+Guy&show_name_exact=true&m
ode=rss episodes 10x2 and 10x4 come from kat.ph and will not download with my 
file size limit set to maxsize = 1024 says they are over 7000

Original comment by lsell...@gmail.com on 7 Nov 2011 at 10:36

[GoogleCodeExporter]

I haven't used the file size limiting functionality, so I haven't made any 
special adjustments to my patch if that is needed. RSSDler seems to do some 
special magic concerning torrent files to figure out the size of the material 
hashed in the torrent, not the torrent file itself, by reading the torrent file.

RSSDler might get confused of the filetype when it tries to guess it by URL. 
Since the suffix of the original URL doesn't end in .torrent, maybe it treats 
it like a regular file, and confusion occurs.

From a quick glance at the code, this doesn't obviously seem to be the case, 
though. The filetype guessing from the filename occurs after nameFromRedirect 
(through getUrlFromRedirect()) has worked its magic, so it should be able to 
determine that it's a torrent file.

Also (I'm thinking as I'm writing :-) ), if RSSDler thought it was a regular 
HTML file say, then it should interpret its size as someting in the kB range, 
so the problem is probably elsewhere.

See if the problem occurs in every setting, or if it can be isolated to that 
specific tracker.

Original comment by daniel.r...@gmail.com on 7 Nov 2011 at 11:27