Open iczero opened 8 months ago
All read and write events of Tengine operate in Edge-Triggered mode. It is efficient and safe. The directive proxy_buffering is enabled by default. You can disable the proxy_buffering specifically. Then, the response is passed to a client synchronously.
proxy_buffering off;
proxy_buffer_size 64k;
proxy_buffers 256 64k;
BTW, if the upstream response with the header "X-Accel-Buffering: no" specifically, even if the Tengine directive "proxy_buffering on" by default, the response will be passed to the client synchronously.
Thanks for the response. In my test case, I was serving a static file from the disk directly with Tengine (using the root
directive). I was not using the reverse proxy functionality, so I do not believe that directive will fix the problem. I will try setting proxy_buffering
in the test environment.
Additionally, the issue only occurs when using XQUIC. Using the same configuration, I can serve the same file with HTTP/2 and HTTP/1.1 without significant memory increase. I traced the buffering behavior with XQUIC to the function ngx_http_xquic_send_chain
in gdb, and I do not believe it is possible to disable it with configuration.
I can provide the full configuration if needed.
Ⅰ. Issue Description
Tengine buffers the entire response body into RAM before sending it.
Ⅱ. Describe what happened
If serving a file, Tengine will read the entire file into RAM (visible from
htop
), and after that is done, it will send the file to the client.Ⅲ. Describe what you expected to happen
Tengine should not buffer the entire file in RAM. It should stream the file as it is read from the disk. Memory usage should not increase significantly.
Ⅳ. How to reproduce it (as minimally and precisely as possible)
ymuski/curl-http3
for local testing, but Firefox also triggers this)truncate -s 2G 2.bin
curl --http3 -v -k https://localhost/2.bin
. There is a substantial delay before curl begins to receive the response. It is visible fromhtop
that nginx will rapidly consume memory until it buffers the entire file in RAM.Ⅴ. Anything else we need to know?
It appears the issue is caused in part by
ngx_http_xquic_send_chain
reading the entire response into another chain. This prevents normal backpressure from happening, which results in the entire response being read into the buffer. The patch below makes it so that Tengine will no longer wait until the entire file is in RAM to begin sending the response. However, it does not fully fix the issue.With the patch, Tengine will still consume the same amount of RAM, but it will also busy-wait on
epoll_wait
. I am not sure how to fix this.Ⅵ. Environment:
sbin/nginx -V
): Commit 04baff4 on branchmaster
(latest commit as of issue creation)uname -a
):6.5.12-300.fc39.x86_64