Does haproxy "trim" POST request that are too large?

[peterlauri]

We are having some problems in production where images are uploaded. We are on Docker Cloud, with this configuration. We are getting the exceptions in Django (python), where it looks like the stream closes at some point. Removing haproxy makes it work again.

Is there any trimming or max post size setting/env that I'm missing?

docker-cloud.yml:

web:
  image: dolphinkiss/ballongen:latest
  tags:
    - ballongen
  environment:
    # ssl and hosts definitions, that HAproxy is using
    FORCE_SSL: yes
    VIRTUAL_HOST: "http://www.example.com,https://www.example.com"
    SSL_CERT: "-----BEGIN PRIVATE KEY----- XXXX -----END CERTIFICATE-----\n"
lb:
  image: dockercloud/haproxy:1.4
  tags:
    - ballongen
  links:
    - web
  ports:
    - "80:80"
    - "443:443"
  restart: on-failure
  roles:
    - global

Django exception:

Traceback:
File "/venv/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
1. response = wrapped_callback(request, *callback_args, **callback_kwargs)
    File "/venv/local/lib/python2.7/site-packages/django/views/generic/base.py" in view
2. return self.dispatch(request, *args, **kwargs)
    File "/venv/local/lib/python2.7/site-packages/django/utils/decorators.py" in _wrapper
3. return bound_func(*args, **kwargs)
    File "/venv/local/lib/python2.7/site-packages/django/views/decorators/csrf.py" in wrapped_view
4. return view_func(*args, **kwargs)
    File "/venv/local/lib/python2.7/site-packages/django/utils/decorators.py" in bound_func
5. return func.*get*(self, type(self))(*args2, **kwargs2)
    File "./blog/views.py" in dispatch
6. return super(ImageBulkUpload, self).dispatch(request, *args, **kwargs)
    File "/venv/local/lib/python2.7/site-packages/braces/views/_access.py" in dispatch
7. request, *args, **kwargs)
    File "/venv/local/lib/python2.7/site-packages/braces/views/_access.py" in dispatch
8. request, *args, **kwargs)
    File "/venv/local/lib/python2.7/site-packages/django/views/generic/base.py" in dispatch
9. return handler(request, *args, **kwargs)
    File "./blog/views.py" in post
10. image = BlogImage(blog=self.blog, image=request.FILES['file'])
    File "/venv/local/lib/python2.7/site-packages/django/core/handlers/wsgi.py" in _get_files
11. self._load_post_and_files()
    File "/venv/local/lib/python2.7/site-packages/django/http/request.py" in _load_post_and_files
12. self._post, self._files = self.parse_file_upload(self.META, data)
    File "/venv/local/lib/python2.7/site-packages/django/http/request.py" in parse_file_upload
13. return parser.parse()
    File "/venv/local/lib/python2.7/site-packages/django/http/multipartparser.py" in parse
14. for chunk in field_stream:
    File "/venv/local/lib/python2.7/site-packages/django/utils/six.py" in next
15. return type(self).*next*(self)
    File "/venv/local/lib/python2.7/site-packages/django/http/multipartparser.py" in *next*
16. output = next(self._producer)
    File "/venv/local/lib/python2.7/site-packages/django/utils/six.py" in next
17. return type(self).*next*(self)
    File "/venv/local/lib/python2.7/site-packages/django/http/multipartparser.py" in *next*
18. for bytes in stream:
    File "/venv/local/lib/python2.7/site-packages/django/utils/six.py" in next
19. return type(self).*next*(self)
    File "/venv/local/lib/python2.7/site-packages/django/http/multipartparser.py" in *next*
20. output = next(self._producer)
    File "/venv/local/lib/python2.7/site-packages/django/utils/six.py" in next
21. return type(self).*next*(self)
    File "/venv/local/lib/python2.7/site-packages/django/http/multipartparser.py" in *next*
22. data = self.flo.read(self.chunk_size)
    File "/venv/local/lib/python2.7/site-packages/django/http/request.py" in read
23. six.reraise(UnreadablePostError, UnreadablePostError(*e.args), sys.exc_info()[2])
    File "/venv/local/lib/python2.7/site-packages/django/http/request.py" in read
24. return self._stream.read(*args, **kwargs)
    File "/venv/local/lib/python2.7/site-packages/django/core/handlers/wsgi.py" in read
25. result = self.buffer + self._read_limited(size - len(self.buffer))
    File "/venv/local/lib/python2.7/site-packages/django/core/handlers/wsgi.py" in _read_limited
26. result = self.stream.read(size)

Exception Type: UnreadablePostError at /inside/blog/252/image_upload/
Exception Value: timeout during read(65536) on wsgi.input

[tifayuki]

Not sure, but I don't think haproxy trims the request. From your Django exception, it looks like it is a timeout error. Maybe you have to adjust the timeout settings instead of using the default ones.