proxyDownloader

[GoogleCodeExporter]

proxyDownloader enables downloading/streaming of files from smb-shares via a 
http-server (the proxy).

Current version is working (including range request etc.)

But there's still work to be done:
Bugs to be fixed:
- Header Content-Length is not sent
- Logging via print-statements not usable in production (error logging)

Improvements:
- Code cleanup (and documentation)
- Test for speed improvement by increasing buffer-/chunk-size
- Speed improvement by using the read-ahead-technique (n*chunk_size)
- proxyDownloader will only be available if pysmbc is available (try: import 
smbc ...)

Original issue reported on code.google.com by moschlar@metalabs.de on 8 Sep 2010 at 10:47

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Nr. 1 fixed!
Content-length is now being sent!

The problem was as easy as that: If you set res.app_iter, res.content_length is 
erased! So content_lenth has to be set after app_iter!

Original comment by moschlar@metalabs.de on 9 Sep 2010 at 8:18

[GoogleCodeExporter]

In r35 all the bugs are fixed. (And development-logging goes to file and stdout)

Selective availability of proxyDownloader is done (Error message on Page, when 
smbfileapp (or psymbc) could not be imported).

Speed Tests with different chunk_sizes done, with paster, maximum is ~150k with 
chunk_size = 512KiB.

Original comment by moschlar@metalabs.de on 10 Sep 2010 at 8:19

[GoogleCodeExporter]

Btw., when testing the proxyDownloader with curl, the -I option, that performs 
an HTTP HEAD request (instead of a regular GET request) should *not* be used!!!
Especially when making a range request ("curl -I -r 0-499 
http://noodle/proxyDownload?id=123"), curl sends an "Content-Range: 0-499" 
header. But Content-Range is not a valid HTTP request header. It should be only 
"Range: 0-499". The only way to achieve this with curl is to *not* use -I.
To see the headers anyway, use "-D filename", which will write the headers to 
filename.
e.g.: "curl -D headers.log -r 0-499 http://noodle/proxyDownload?id=123"

Original comment by moschlar@metalabs.de on 10 Sep 2010 at 1:44

[GoogleCodeExporter]

It's important to use an up-to-date version of WebOb!
Most of our remaining bugs have been fixed then... ;)

Original comment by moschlar@metalabs.de on 11 Sep 2010 at 1:04

[GoogleCodeExporter]

Remaining bugs don't affect general "Finished" status

Original comment by moschlar@metalabs.de on 7 Jan 2011 at 8:05

• Changed state: Fixed