search

Pylons / waitress

Waitress - A WSGI server for Python 3
https://docs.pylonsproject.org/projects/waitress/en/latest/
Other
1.45k stars 175 forks source link

Сan't upload large file #351

Closed StpMax closed 2 years ago

StpMax commented 3 years ago

Hello. I use waitress==2.0.0 as server for flask app. It works on ubuntu 18.04. I can not upload a big file, because i get memory exceed issue during that. File size is about 1.1Gb, so i add to serve arguments: max_request_body_size=1073741824 * 10, inbuf_overflow=1073741824 * 10 I have 16Gb of memory. I tried upload with default flask server and with gunicorn. In both cases memory usage raises from 30% to 55% and go back to 30%. With waitress memory usage grow to 100% and start eat swap file.

digitalresistor commented 3 years ago

inbuf_overflow is in bytes. So your argument sets it to 10 GiB. That is being stored in a BytesIO object. Not sure how efficient that is to be honest.

A large upload should however flow into a temp file without issue, and once its written to the temporary file it should not continue growing in size.

I would recommend just changing the max_request_body_size and leaving the default for inbuf_overflow or setting it to something a little smaller than 10 GiB.

digitalresistor commented 2 years ago

Please let me know if you can provide more information, until then I am going to close this request.