Out of Memory while trying to serve large Files

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Have a large enough file to exhaust your memory
2. start the server
3. try to get the file

I'm using the last release Version 0.9.4, I read in the code, that the file 
handler .read()'s the complete file into a string. I saw there is another 
implementation, but i dont know how to force it to be used.

The Client is OSX 10.6.

Thanks for the nice Tool, its especially awesome because no other small server 
is as compatible as this one.

I'm getting flooded with following exception:
----------------------------------------
Exception happened during processing of request from ('10.10.10.102', 56758)
Traceback (most recent call last):
  File "/usr/lib/python2.6/SocketServer.py", line 560, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib/python2.6/SocketServer.py", line 322, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python2.6/SocketServer.py", line 617, in __init__
    self.handle()
  File "/usr/lib/pymodules/python2.6/DAV/AuthServer.py", line 229, in handle
    AuthRequestHandler.handle(self)
  File "/usr/lib/pymodules/python2.6/DAV/AuthServer.py", line 149, in handle
    method()
  File "/usr/lib/pymodules/python2.6/DAV/WebDAVServer.py", line 293, in do_GET
    status_code = self._HEAD_GET(with_body=True)
  File "/usr/lib/pymodules/python2.6/DAV/WebDAVServer.py", line 277, in _HEAD_GET
    content_type, headers)
  File "/usr/lib/pymodules/python2.6/DAV/WebDAVServer.py", line 136, in send_body_chunks_if_http11
    self.send_body_chunks(DATA, code, msg, desc, ctype, headers)
  File "/usr/lib/pymodules/python2.6/DAV/WebDAVServer.py", line 201, in send_body_chunks
    self._append(DATA.read())
  File "/usr/lib/pymodules/python2.6/DAVServer/fshandler.py", line 66, in read
    data = self.__fp.read(length)
MemoryError
----------------------------------------

Original issue reported on code.google.com by army.of....@gmail.com on 20 Sep 2010 at 10:19

[GoogleCodeExporter]

Hacked it to work as follows:

http_response_use_iterator = 1
 into the config

and added self._buffer="" twice to WebDAVServer.py to ensure ._buffer is 
defined.
Because else following exception is thrown:
----------------------------------------
Exception happened during processing of request from ('10.10.10.102', 54656)
Traceback (most recent call last):
  File "/usr/lib/python2.6/SocketServer.py", line 560, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib/python2.6/SocketServer.py", line 322, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python2.6/SocketServer.py", line 617, in __init__
    self.handle()
  File "/usr/lib/pymodules/python2.6/DAV/AuthServer.py", line 229, in handle
    AuthRequestHandler.handle(self)
  File "/usr/lib/pymodules/python2.6/DAV/AuthServer.py", line 149, in handle
    method()
  File "/usr/lib/pymodules/python2.6/DAV/WebDAVServer.py", line 293, in do_GET
    status_code = self._HEAD_GET(with_body=True)
  File "/usr/lib/pymodules/python2.6/DAV/WebDAVServer.py", line 277, in _HEAD_GET
    content_type, headers)
  File "/usr/lib/pymodules/python2.6/DAV/WebDAVServer.py", line 136, in send_body_chunks_if_http11
    self.send_body_chunks(DATA, code, msg, desc, ctype, headers)
  File "/usr/lib/pymodules/python2.6/DAV/WebDAVServer.py", line 187, in send_body_chunks
    self.wfile.write(self._buffer)
AttributeError: DAVAuthHandler instance has no attribute '_buffer'
----------------------------------------

DAVAuthHandler being the last parent class.  I saw, that there is a .__buffer 
attribute in BufferedHTTPRequestHandler via ._init_buffer() but I dont wanna 
break something.

It seems that the downloaded files are corrupted tough.

Could you give me some hints how to fix that ? - I'm not in the mood reading 
the whole source tree.

:) have a nice day

Original comment by army.of....@gmail.com on 24 Sep 2010 at 2:26

[GoogleCodeExporter]

the  send_body_chunks seems to be responsible for the data corruption.
i hardcoded send_body_chunks_if_http11 to always use send_body(...) and now it 
works without data corruption.

Original comment by army.of....@gmail.com on 24 Sep 2010 at 2:41

[GoogleCodeExporter]

Fixed in next release - thanks for the report

Original comment by spamsch@gmail.com on 17 Feb 2011 at 10:56

[GoogleCodeExporter]

Original comment by spamsch@gmail.com on 17 Feb 2011 at 10:57

• Changed state: Fixed