search

alibaba / tengine

A distribution of Nginx with some advanced features
https://tengine.taobao.org
BSD 2-Clause "Simplified" License
12.84k stars 2.52k forks source link

tengine上传大文件支持问题 #1003

Open xianshi122 opened 6 years ago

xianshi122 commented 6 years ago

从nginx1.10.1升级为tengin2.1.2个后， 用户通过multipart/form-data 上传文件时 如果文件大于1M就会上传失败 抓包分析后，发现在用户在上传大文件时，http body内容不完整 导致 java MultipartHttpServletRequest request 拿到的 request.getFileNames() size 为0

抓取了 tengin与tomcat之间的报文，异常报文中 http body内容异常 正常报文 ` POST /attachment/upload HTTP/1.0 Host: .com X-Real-IP: 10.1.1.15 X-Forwarded-For: 10.1.1.15 Connection: close Content-Length: 396752 accept: application/json, text/javascript, /; q=0.01 origin: https://.com x-requested-with: XMLHttpRequest user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36 content-type: multipart/form-data; boundary=----WebKitFormBoundaryn1FCXAuaQn8567yc referer: https://qa-manage.******/104407 accept-encoding: gzip, deflate, br accept-language: zh-CN,zh;q=0.9,en;q=0.8,en-US;q=0.7 cookie: gr_user_id=6f4a05ba-9acc-438e-b904-fc580cfdf390; =2f854935-21d9-4f9e-90c8-cfbcd9d1c735; _ga=GA1.2.699820353.1517203602; _gid=GA1.2.485202220.1517203602; Hm_lvt_5d14949ada92c16ee007021e4d7234db=1516938388,1517073337,1517195006,1517202090; Hm_lpvt_5d14949ada92c16ee007021e4d7234db=1517208531; =48fd5991-f24d-424a-9f54-4245d28bfb6f; langmanage=en_US

------WebKitFormBoundaryn1FCXAuaQn8567yc Content-Disposition: form-data; name="saveToUserKey"

false ------WebKitFormBoundaryn1FCXAuaQn8567yc Content-Disposition: form-data; name="userAttachOnly"

undefined ------WebKitFormBoundaryn1FCXAuaQn8567yc Content-Disposition: form-data; name="titles"

.............................. ------WebKitFormBoundaryn1FCXAuaQn8567yc Content-Disposition: form-data; name="files[]"; filename="...............................pptx" Content-Type: application/vnd.openxmlformats-officedocument.presentationml.presentation .....more 异常报文 POST /attachment/upload HTTP/1.0 Host: qa-**.com X-Real-IP: 10.1.1.15 X-Forwarded-For: 10.1.1.15 Connection: close Content-Length: 5342347 accept: application/json, text/javascript, /; q=0.01 origin: https://.com x-requested-with: XMLHttpRequest user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36 content-type: multipart/form-data; boundary=----WebKitFormBoundaryWM42hdx6CVJtNPxz referer: https://qa-***/detail/104407 accept-encoding: gzip, deflate, br accept-language: zh-CN,zh;q=0.9,en;q=0.8,en-US;q=0.7 cookie: gr_user_id=6f4a05ba-9acc-438e-b904-fc580cfdf390; =2f854935-21d9-4f9e-90c8-cfbcd9d1c735; _ga=GA1.2.699820353.1517203602; _gid=GA1.2.485202220.1517203602; Hm_lvt_5d14949ada92c16ee007021e4d7234db=1516938388,1517073337,1517195006,1517202090; Hm_lpvt_5d14949ada92c16ee007021e4d7234db=1517208531; =48fd5991-f24d-424a-9f54-4245d28bfb6f; langmanage=en_US

0.................DN.ppt/media/image28.pngPK..-. .......!...NaP=..P=................fuN.ppt/media/image32.pngPK..-. .......!._F^QJ=..J=..................N.ppt/media/image31.pngPK..-. .......!.e6V..:...:................f.N.ppt/media/image30.pngPK..-. .......!." *..6...6................F+O.ppt/media/image29.pngPK..-. .......!..) .Ne..Ne.................aO.ppt/media/image27.pngPK..-. .......!....h.4...4................y.O.ppt/media/image33.pngPK..-. .......!...juuG..uG..................O.ppt/media/image35.pngPK..-. .......!..&0.q<..q<................iCP.ppt/media/image38.pngPK..-. .......!.....b1..b1................ more `

taoyuanyuan commented 6 years ago

错误日志有啥信息吗？ 麻烦贴一下配置文件

yourchanges commented 6 years ago

类似的问题： 版本最新的2.2.2 centos 6.7 64位 只有在https+http2 启用后，上传1MB以上的文件，就会卡住，后端java收到为空

验证：

lee@lee-Inspiron-7577 ~/curl-7.58.0 $ /usr/local/bin/curl --http2 -vvv https://x.xx.com/api/rent/productImage/uploadImage    --form  "file=@/home/lee/1.png" 
*   Trying x.x.x.x...
* TCP_NODELAY set
* Connected to x.x.com (x) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=CN; L=\U6606\U660E; O=\U4E91\U5357\U53C1\U7396\U7F51\U7EDC\U79D1\U6280\U6709\U9650\U516C\U53F8; OU=\U6280\U672F\U90E8; CN=*.x.com
*  start date: Jan 26 00:00:00 2018 GMT
*  expire date: Sep 13 12:00:00 2020 GMT
*  subjectAltName: host "x.x.com" matched cert's "*.x.com"
*  issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=GeoTrust RSA CA 2018
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x231bf90)
> POST /api/rent/productImage/uploadImage HTTP/2
> Host: x.x.com
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Length: 1436542
> Content-Type: multipart/form-data; boundary=------------------------7d5aff34511dd99a
> 
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!

会在这里卡住，死掉

如果换用http1.1 请求一点问题没有

lee@lee-Inspiron-7577 ~/curl-7.58.0 $ /usr/local/bin/curl --http1.1 -vvv https://x.x.com/api/rent/productImage/uploadImage   --form  "file=@/home/lee/1.png" 
*   Trying x.x.x.x...
* TCP_NODELAY set
* Connected to x.x.com (x) port 443 (#0)
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=CN; L=\U6606\U660E; O=\U4E91\U5357\U53C1\U7396\U7F51\U7EDC\U79D1\U6280\U6709\U9650\U516C\U53F8; OU=\U6280\U672F\U90E8; CN=*.x.com
*  start date: Jan 26 00:00:00 2018 GMT
*  expire date: Sep 13 12:00:00 2020 GMT
*  subjectAltName: host "x.x.com" matched cert's "*.x.com"
*  issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=GeoTrust RSA CA 2018
*  SSL certificate verify ok.
> POST /api/rent/productImage/uploadImage HTTP/1.1
> Host: x.x.com
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Length: 1436542
> Content-Type: multipart/form-data; boundary=------------------------6e340c4b050537bb
> Expect: 100-continue
> 
< HTTP/1.1 100 Continue
< HTTP/1.1 200 
< Server: ch999ws19
< Date: Tue, 06 Mar 2018 08:34:42 GMT
< Content-Type: application/json;charset=UTF-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< X-Application-Context: rent:prod:9987
< Set-Cookie: buyCartCookieId=970940723212963842; Max-Age=86400; Expires=Wed, 07-Mar-2018 08:34:42 GMT; Path=/
< 
* Connection #0 to host x.x.com left intact
{"code":0,"msg":"SUCCESS","userMsg":"上传成功!","data":"https://x.x.com/newstatic/794/3a6fc89460638a.png","area":{}}

配置文件：


server {

       server_name  zu.xxx.com;

listen  443 ssl http2;
ssl_certificate conf.d/cerNew/xxx_com.crt;
ssl_certificate_key conf.d/cerNew/xxx_com.key;

    root /var/www/xxx/;
    index index.html;

location /api {
           proxy_pass http://zuji;
           proxy_set_header X-Real-IP       $remote_addr;
           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
           proxy_set_header XHttpScheme "$scheme";
           #proxy_buffering off;
        }

nginx.conf

http {
    ssl_session_cache    shared:SSL:1000m;
    ssl_session_timeout  10m;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
    ssl_ciphers EECDH+CHACHA20:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5;
    ssl_prefer_server_ciphers   on;

    client_max_body_size    1024m;

nginx 模块

[root@linux2 ~]# /usr/local/nginx/sbin/nginx -V
Tengine version: Tengine/2.2.2 (nginx/1.8.1)
built by gcc 4.4.7 20120313 (Red Hat 4.4.7-18) (GCC) 
TLS SNI support enabled
configure arguments: --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_gzip_static_module --with-http_concat_module --with-http_ssl_module --with-http_v2_module --with-openssl=../openssl-1.0.2n --with-http_lua_module --with-luajit-lib=/usr/local/lib/ --with-luajit-inc=/usr/local/include/luajit-2.0/ --with-lua-inc=/usr/local/include/luajit-2.0/ --with-lua-lib=/usr/local/lib/ --with-ld-opt=-Wl,-rpath,
nginx: loaded modules:
nginx:     ngx_core_module (static)
nginx:     ngx_errlog_module (static)
nginx:     ngx_conf_module (static)
nginx:     ngx_dso_module (static)
nginx:     ngx_events_module (static)
nginx:     ngx_event_core_module (static)
nginx:     ngx_epoll_module (static)
nginx:     ngx_procs_module (static)
nginx:     ngx_proc_core_module (static)
nginx:     ngx_openssl_module (static)
nginx:     ngx_regex_module (static)
nginx:     ngx_http_module (static)
nginx:     ngx_http_core_module (static)
nginx:     ngx_http_log_module (static)
nginx:     ngx_http_upstream_module (static)
nginx:     ngx_http_v2_module (static)
nginx:     ngx_http_static_module (static)
nginx:     ngx_http_gzip_static_module (static)
nginx:     ngx_http_autoindex_module (static)
nginx:     ngx_http_index_module (static)
nginx:     ngx_http_concat_module (static)
nginx:     ngx_http_auth_request_module (static)
nginx:     ngx_http_auth_basic_module (static)
nginx:     ngx_http_access_module (static)
nginx:     ngx_http_limit_conn_module (static)
nginx:     ngx_http_limit_req_module (static)
nginx:     ngx_http_geo_module (static)
nginx:     ngx_http_map_module (static)
nginx:     ngx_http_split_clients_module (static)
nginx:     ngx_http_referer_module (static)
nginx:     ngx_http_rewrite_module (static)
nginx:     ngx_http_ssl_module (static)
nginx:     ngx_http_proxy_module (static)
nginx:     ngx_http_fastcgi_module (static)
nginx:     ngx_http_uwsgi_module (static)
nginx:     ngx_http_scgi_module (static)
nginx:     ngx_http_memcached_module (static)
nginx:     ngx_http_empty_gif_module (static)
nginx:     ngx_http_browser_module (static)
nginx:     ngx_http_user_agent_module (static)
nginx:     ngx_http_upstream_hash_module (static)
nginx:     ngx_http_upstream_ip_hash_module (static)
nginx:     ngx_http_upstream_consistent_hash_module (static)
nginx:     ngx_http_upstream_check_module (static)
nginx:     ngx_http_upstream_least_conn_module (static)
nginx:     ngx_http_upstream_keepalive_module (static)
nginx:     ngx_http_upstream_dynamic_module (static)
nginx:     ngx_http_stub_status_module (static)
nginx:     ngx_http_write_filter_module (static)
nginx:     ngx_http_header_filter_module (static)
nginx:     ngx_http_chunked_filter_module (static)
nginx:     ngx_http_v2_filter_module (static)
nginx:     ngx_http_range_header_filter_module (static)
nginx:     ngx_http_gzip_filter_module (static)
nginx:     ngx_http_postpone_filter_module (static)
nginx:     ngx_http_ssi_filter_module (static)
nginx:     ngx_http_charset_filter_module (static)
nginx:     ngx_http_userid_filter_module (static)
nginx:     ngx_http_footer_filter_module (static)
nginx:     ngx_http_trim_filter_module (static)
nginx:     ngx_http_headers_filter_module (static)
nginx:     ngx_http_upstream_session_sticky_module (static)
nginx:     ngx_http_reqstat_module (static)
nginx:     ngx_http_lua_module (static)
nginx:     ngx_http_copy_filter_module (static)
nginx:     ngx_http_range_body_filter_module (static)
nginx:     ngx_http_not_modified_filter_module (static)
[root@linux2 ~]#
yourchanges commented 6 years ago

上述同样的配置，在2.2.0 版本没问题

[root@linux2 sbin]# /usr/local/nginx/sbin/nginx -V
Tengine version: Tengine/2.2.0 (nginx/1.8.1)
built by gcc 4.4.7 20120313 (Red Hat 4.4.7-17) (GCC) 
TLS SNI support enabled
configure arguments: --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_gzip_static_module --with-http_concat_module --with-http_ssl_module --with-http_v2_module --with-openssl=../openssl-1.0.2j --add-module=../ngx_cache_purge-2.3 --with-http_lua_module --with-luajit-lib=/usr/local/lib/ --with-luajit-inc=/usr/local/include/luajit-2.0/ --with-lua-inc=/usr/local/include/luajit-2.0/ --with-lua-lib=/usr/local/lib/ --with-ld-opt=-Wl,-rpath,
nginx: loaded modules:
nginx:     ngx_core_module (static)
nginx:     ngx_errlog_module (static)
nginx:     ngx_conf_module (static)
nginx:     ngx_dso_module (static)
nginx:     ngx_events_module (static)
nginx:     ngx_event_core_module (static)
nginx:     ngx_epoll_module (static)
nginx:     ngx_procs_module (static)
nginx:     ngx_proc_core_module (static)
nginx:     ngx_openssl_module (static)
nginx:     ngx_regex_module (static)
nginx:     ngx_http_module (static)
nginx:     ngx_http_core_module (static)
nginx:     ngx_http_log_module (static)
nginx:     ngx_http_upstream_module (static)
nginx:     ngx_http_v2_module (static)
nginx:     ngx_http_static_module (static)
nginx:     ngx_http_gzip_static_module (static)
nginx:     ngx_http_autoindex_module (static)
nginx:     ngx_http_index_module (static)
nginx:     ngx_http_concat_module (static)
nginx:     ngx_http_auth_request_module (static)
nginx:     ngx_http_auth_basic_module (static)
nginx:     ngx_http_access_module (static)
nginx:     ngx_http_limit_conn_module (static)
nginx:     ngx_http_limit_req_module (static)
nginx:     ngx_http_geo_module (static)
nginx:     ngx_http_map_module (static)
nginx:     ngx_http_split_clients_module (static)
nginx:     ngx_http_referer_module (static)
nginx:     ngx_http_rewrite_module (static)
nginx:     ngx_http_ssl_module (static)
nginx:     ngx_http_proxy_module (static)
nginx:     ngx_http_fastcgi_module (static)
nginx:     ngx_http_uwsgi_module (static)
nginx:     ngx_http_scgi_module (static)
nginx:     ngx_http_memcached_module (static)
nginx:     ngx_http_empty_gif_module (static)
nginx:     ngx_http_browser_module (static)
nginx:     ngx_http_user_agent_module (static)
nginx:     ngx_http_upstream_hash_module (static)
nginx:     ngx_http_upstream_ip_hash_module (static)
nginx:     ngx_http_upstream_consistent_hash_module (static)
nginx:     ngx_http_upstream_check_module (static)
nginx:     ngx_http_upstream_least_conn_module (static)
nginx:     ngx_http_upstream_keepalive_module (static)
nginx:     ngx_http_upstream_dynamic_module (static)
nginx:     ngx_http_stub_status_module (static)
nginx:     ngx_http_cache_purge_module (static)
nginx:     ngx_http_write_filter_module (static)
nginx:     ngx_http_header_filter_module (static)
nginx:     ngx_http_chunked_filter_module (static)
nginx:     ngx_http_v2_filter_module (static)
nginx:     ngx_http_range_header_filter_module (static)
nginx:     ngx_http_gzip_filter_module (static)
nginx:     ngx_http_postpone_filter_module (static)
nginx:     ngx_http_ssi_filter_module (static)
nginx:     ngx_http_charset_filter_module (static)
nginx:     ngx_http_userid_filter_module (static)
nginx:     ngx_http_footer_filter_module (static)
nginx:     ngx_http_trim_filter_module (static)
nginx:     ngx_http_headers_filter_module (static)
nginx:     ngx_http_upstream_session_sticky_module (static)
nginx:     ngx_http_reqstat_module (static)
nginx:     ngx_http_lua_module (static)
nginx:     ngx_http_copy_filter_module (static)
nginx:     ngx_http_range_body_filter_module (static)
nginx:     ngx_http_not_modified_filter_module (static)
[root@linux2 sbin]#

请求记录：

lee@lee-Inspiron-7577 ~/curl-7.58.0 $ /usr/local/bin/curl --http2 -vvv https://x.x.com/api/rent/productImage/uploadImage    --form  "file=@/home/lee/1.png" 
*   Trying x.x.x.x...
* TCP_NODELAY set
* Connected to x.x.com (x) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=CN; L=\U6606\U660E; O=\U4E91\U5357\U53C1\U7396\U7F51\U7EDC\U79D1\U6280\U6709\U9650\U516C\U53F8; OU=\U6280\U672F\U90E8; CN=*.x.com
*  start date: Jan 26 00:00:00 2018 GMT
*  expire date: Sep 13 12:00:00 2020 GMT
*  subjectAltName: host "x.x.com" matched cert's "*.x.com"
*  issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=GeoTrust RSA CA 2018
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x100df90)
> POST /api/rent/productImage/uploadImage HTTP/2
> Host: x.x.com
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Length: 1436542
> Content-Type: multipart/form-data; boundary=------------------------355178d905bb1f52
> 
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 200 
< server: nginx
< date: Tue, 06 Mar 2018 08:49:38 GMT
< content-type: application/json;charset=UTF-8
< x-application-context: rent:prod:9987
< set-cookie: buyCartCookieId=970944482521501697; Max-Age=86400; Expires=Wed, 07-Mar-2018 08:49:38 GMT; Path=/
< 
* Connection #0 to host x.x.com left intact
{"code":0,"msg":"SUCCESS","userMsg":"上传成功!","data":"https://x.x.com/newstatic/797/3a719e59ef8a69.png","area":{}}
yourchanges commented 6 years ago

不用curl 用最新的chrome 64 也是一样的 Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36

yourchanges commented 6 years ago

有没有任何提示，代码在相关代码在哪个模块，我好自己去看了尝试修复，生产环境，急

chobits commented 6 years ago

proxy_request_buffering这个指令你有配置吗？这个指令在2.1.2里要设置成off才可以，因为这个版本基于的ngx还不支持http2的流式上传

yourchanges commented 6 years ago

非常感谢，的确目前的2.1.2 和2.2.2 里面的http2 还不支持stream下的请求缓存，设置 proxy_request_buffering off; 即可解决问题

yourchanges commented 6 years ago

那么小于1MB的文件，怎么就可以正常上传？是否我调大buffer大小（如果有这个参数，可以调整的话）也可以避免这个问题。 另外，http2的上传支持实现，有计划表了么？

chobits commented 6 years ago
  1. tengine-2.2.2里的支持流式上传
  2. tengine-2.1.2里，上传数据过大（超过client_body_buffer_size）会写磁盘的，不超过就放在内存里。所以对于上传来说可用性没啥问题，但是会把完整上传数据收完才发给后端java
yourchanges commented 6 years ago

那就有bug 我用的2.2.2有问题 2.2.0没问题

Xiaochen Wang notifications@github.com于2018年3月7日 周三21:14写道：

  1. tengine-2.2.2里的支持流式上传

  2. 2.1.2里，上传数据过大（超过client_body_buffer_size）会写磁盘的，不超过就放在内存里。所以对于上传来说可用性没啥问题

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/alibaba/tengine/issues/1003#issuecomment-371133860, or mute the thread https://github.com/notifications/unsubscribe-auth/AAeFkiyimRmOU-LUrZMM49Iv6xzRUb1yks5tb9zsgaJpZM4RwfxN .

chobits commented 6 years ago

那就有bug 我用的2.2.2有问题 2.2.0没问题

你这里都是设置 'proxy_request_buffering off;'测试的吗？

这周我会看下，因为h2还有一些代码更新下没merge进来

yourchanges commented 6 years ago

@chobits 是的，在location 段加的proxy_request_buffering off;

lvai125 commented 6 years ago 
                    client_header_timeout 3600;
                    keepalive_timeout     3600;
                    client_body_timeout 3600;
                    send_timeout          3600;
                    client_max_body_size 200m;  
                    proxy_request_buffering off;
                    proxy_connect_timeout   3600; 
                    proxy_read_timeout 3600;
                    proxy_send_timeout 3600;
                    client_header_buffer_size 12800k;
                    large_client_header_buffers 4 12800k;
                    proxy_ignore_client_abort on;
                    proxy_temp_file_write_size 12800k;
lvai125 commented 6 years ago

上面是我的配置，在http里上传图片没问题1m左右。而在https里上传超过10k就报400。加了proxy_request_buffering off;这个参数解决了。

lazyfighter commented 5 years ago

这个已经修复了吗

chobits commented 5 years ago

Hi all

Please try our latest master branch, we have merged all HTTP/2 bugfix.

Hi all, we have merged nginx 1.15.9 into latest master. Please check it! We'll release new version 2.3.x (based on nginx 1.15.x) as soon. Please check WARNING list before upgrade. If have any questions, please give us feedback in time. Thank you.

WARNING

Tengine has been upgraded core files from the Nginx 1.15.9 version. Note that this version is slightly not backwards compatible, some tengine features has been replaced by nginx offical, check the following list:

  1. Removed reuse_port directive implemented by Tengine and use the official reuseport of Nginx, detailed reference document .
  2. Removed dso_tool tool and dso directive, If before using Tengine dso feature, you could switch to the Nginx official load_module directive, detailed reference document1 、document2 .
  3. Moved the slice module into modules dir, use the Nginx official slice feature by default and use --add-module=modules/ngx_http_slice_module way to compile if need Tengine's slice module, otherwise use --with-http_slice_module compilation parameter.
  4. Extract the tengine modules to modules dir, if you need to use that module then use --add-module=modules/<module_name> way to compile.
  5. The request counting logic for limit_req is consistent with the Nginx official, removed the logic that skip counting with any empty value variable in limit_req_zone.

各位Tengine使用者你们好！我们近期合并了Nginx官方1.15.9版本代码到Tengine master分支，大家可以先尝鲜使用master分支代码做测试（若上生产使用请一定要做好灰度验证、并阅读相关注意列表）。同时我们最近也会正式发布Tengine-2.3.x版本，欢迎大家使用，如有任何问题请随时反馈，谢谢。

警告

本次Tengine升级core代码到官方Nginx 1.15.9版本，由于其部分功能Nginx官方当前已经具备、所以本次直接弃用Tengine自身实现的部分配置指令，具体不兼容列表如下：

  1. 废弃Tengine自身实现的reuse_port指令，使用Nginx官方的reuseport。升级方法：将events配置块里面的reuse_port on|off注释掉，在对应的监听端口后面加reuseport参数、详细参考文档
  2. 废弃Tengine的dso_tool工具以及dso配置指令，若之前有使用Tengine的dso功能、则可以切换到Nginx官方的load_module指令,详细文档参考1 和 参考2
  3. 移除Tengine加强版slice模块到modules、默认使用Nginx官方slice功能，如果依然需要使用Tengine的slice则编译slice时请使用 --add-module=modules/ngx_http_slice_module，否则使用 --with-http_slice_module 编译参数
  4. Tengine自身实现的模块，当前全部剥离到modules目录下，如果需要使用那个模块、则使用--add-module=modules/<module_name>的方式进行编译
  5. limit_req的请求计数逻辑和官方保持一致，去除limit_req_zone中任何一个变量值为空跳过请求计数的逻辑