Closed qjavax closed 3 years ago
bachue
commented
3 years ago I can reproduce this issue, and it's definitely a bug. I tracked it and found if ngx_lookup_udp_connection() could find an existed udp_connection and reuse it, then this bug would happen.
junhochoi
commented
3 years ago Which curl version are you using? It fixed a similar issue weeks ago https://github.com/curl/curl/issues/6213
qjavax
commented
3 years ago Im using
curl 7.74.0-DEV (x86_64-pc-linux-gnu) libcurl/7.74.0-DEV BoringSSL zlib/1.2.11 brotli/1.0.7 zstd/1.4.3 libidn2/2.2.0 libpsl/0.21.0 (+libidn2/2.2.0) nghttp2/1.39.2 quiche/0.6.0 Release-Date: [unreleased] Protocols: http https mqtt Features: alt-svc AsynchDNS brotli HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Largefile libz NTLM NTLM_WB PSL SSL UnixSockets zstd
As I mentioned while opening issue I belive it is not case of curl because this problem is not reproducable by curling to https://cloudflare-quic.com/ . This issue is also possible to reproduce with browser.
qjavax
commented
3 years ago @junhochoi Just to provide evidence it's not issue of curl here's screenshot of this issue replicated with Firefox. How you can achieve that: just refresh page several times to get request without any response
junhochoi
commented
3 years ago @qjavax we recently updated our nginx patch. Can you try with latest? Thanks.
qjavax
commented
3 years ago @junhochoi yes I did rebuild my nginx + quiche yesterday with commit: 1e439773dcb96c64071d6b85496bd33cc306aad7 . Unfortunately problem didn't change. I also tried your docker file with nginx+quiche
docker run -d -v $(pwd):/usr/local/nginx/html -p 127.0.0.1:8443:8443/udp -p 127.0.0.1:8443:8443/tcp -p 127.0.0.1:8080:8080/tcp -it ef6cc77f47b0
for x in {1..1000}; do echo " ===> $x"; curl3 -k -vvI --http3 https://127.0.0.1:8443; done
You can also replicate this issue like that.
qjavax
commented
3 years ago Hi, i got focused too much on nginx side of this issue. But after rebuilding nginx AND curl with newest quiche 0.7.0 this issue do not occur anymore so I'm closing this issue. Thank you for your time.
Hi, I'm having some issues with HTTP/3 nginx + quiche. After n-times requests to server it will hang and then timeout. Problem is replicatible with curl:
for x in {1..1000}; do echo " ===> $x"; curl3 -vvI --http3 https://<domain>/ -X GET; doneor with browser (in my case Firefox) with http3 enabled and refreshing page many times. Number of successful request may vary in range from ~5 to ~350. I'm sure thats not case of curl (curling to https://cloudflare-quic.com/ will not cause any problems) or problem of nginx or network configuration (http/2 and http/1 will not cause problems as well). I also tried changing nginx configuration of keepalive or http3_ configuration directives. Nginx works as a reverse proxy to Apache (but I belive Apache is not causing any issues here)All build on arch linux with pkgbuild from quiche commit: 9d0c677ef1411b24d720b5c8b73bcc94b5535c29 (latest right now) and nginx version: nginx/1.16.1.
Log from successful curl request:
With curl in for loop request will hang to timeout then proceed next request and aften n-times hang again.
Nginx debug log:
And after other connections cleanup waits for curl timeout then process other requests.
Strace from nginx process:
I would like to point out this (from other strace of nginx proccess):
Parts of server configuration (http3 section): (changes in keepalive and http3 won't fix issue)
This type of flooding does not block nginx server (other connection work well). I have an idea of quiche socket getting close too early in some cases (but I'm not sure)