help request: a client request body is buffered to a temporary file

saiyoofan commented 2 years ago

Description

get some error log:

a client request body is buffered to a temporary file

but set the apisix.config.yaml client_body_buffer_size: 2048k It doesn't seem to be working

Environment

APISIX version (run apisix version):
Operating system (run uname -a):
OpenResty / Nginx version (run openresty -V or nginx -V):
etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info):
APISIX Dashboard version, if relevant:
Plugin runner version, for issues related to plugin runners:
LuaRocks version, for installation issues (run luarocks --version):

soulbird commented 2 years ago

Maybe the size of your request body has exceeded 2048k. You should show more infomation about your request first.

saiyoofan commented 2 years ago

when I set client_body_buffer_size: 30m,The same error

soulbird commented 2 years ago

when I set client_body_buffer_size: 30m,The same error

Are you sure your configuration is working? When Apache APISIX was running, check the config item client_body_buffer_size in nginx.conf. And could you provide your specific configuration about apisix.config.yaml

soulbird commented 2 years ago

when I set client_body_buffer_size: 30m,The same error

Could you provide steps to reproduce?

sandy420 commented 2 years ago

me too!Please advise! A(J6T%3LP_XJUO_GKR`2~1F

tokers commented 2 years ago

when I set client_body_buffer_size: 30m,The same error

Are you sure your configuration is working? When Apache APISIX was running, check the config item client_body_buffer_size in nginx.conf. And could you provide your specific configuration about apisix.config.yaml

@sandy420 Try this.

sandy420 commented 2 years ago

client_body_buffer_size I have configuration in config file and it takes effect in nginx.conf

tokers commented 2 years ago

client_body_buffer_size I have configuration in config file and it takes effect in nginx.conf

So did you still see the related log entries?

sandy420 commented 2 years ago

So did you still see the related log entries?

Yes,can you give me some ideal?

tokers commented 2 years ago

Strange. Could you share the APISIX configuration and the steps to reproduce this issue?

sandy420 commented 2 years ago

Strange. Could you share the APISIX configuration and the steps to reproduce this issue?

etcd:
    host:
  tls:
      verify: false
apisix:
  node_listen: 80
  enable_server_tokens: false
  admin_key:
    - name: "admin"
  role: admin

  router:
    http: 'radixtree_host_uri'
  ssl:
    enable_http2: true
    listen_port: 443
  disable_sync_configuration_during_start: true
  proxy_cache:
    cache_ttl: 10s
    zones:
      - name: disk_cache_one
        memory_size: 5G
        disk_size: 5G
        disk_path: /tmp/disk_cache_one
        cache_levels: 1:2
plugins:
  \- prometheus
  \- log-rotate
  \- proxy-rewrite
  \- redirect
  \- error-log-logger
  \- node-status
  \- ip-restriction
  \- echo
  \- basic-auth
  \- server-info
  \- proxy-mirror
  \- kafka-logger
  \- response-rewrite
  \- serverless-pre-function
plugin_attr:
  prometheus:
    export_uri: /apisix/prometheus/metrics
    enable_export_server: false
    export_addr:
      ip: 0.0.0.0
      port: 9091
  log-rotate:
    interval: 21600
    max_kept: 10
nginx_config:
  http_end_configuration_snippet: |
    reset_timedout_connection off;
        client_header_buffer_size 1024k;
        client_body_buffer_size 20m;
        proxy_intercept_errors on;
        large_client_header_buffers 8 128k;
        gzip on;
        proxy_next_upstream off;
        proxy_connect_timeout 60;
        proxy_send_timeout 300s;
        proxy_read_timeout 300s;
        proxy_buffers 16 2048k; # getconf PAGESIZE
        proxy_buffer_size 2048k;
        proxy_busy_buffers_size 2048k;
        proxy_temp_file_write_size 2048K;
        proxy_max_temp_file_size 0;
        proxy_ignore_client_abort on;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header traffic-source "INTERNAL";
  http:
    client_max_body_size: 3g
    enable_access_log: true
    lua_shared_dict:
      prometheus-metrics: 100m                       
    real_ip_header: "X-Forwarded-For"
    real_ip_from:
      - 0.0.0.0/0
    keepalive_timeout: 600s
    client_header_timeout: 600s
    client_body_timeout: 600s
    send_timeout: 100s
    access_log_format: "$time_iso8601|$msec|$status|$request_completion|$bytes_sent|$body_bytes_sent|$realip_remote_addr|$remote_addr|$http_x_forwarded_for|$remote_user|$host|$server_name|$server_port|$server_protocol|$scheme|$request_method|$request_length|$request_time|$request_uri|$uri|$content_length|$content_type|$http_referer|$http_user_agent|$http_app_jb|$http_client_info|$upstream_addr|$upstream_connect_time|$upstream_header_time|$upstream_response_time|$upstream_status|$upstream_bytes_received|$upstream_cache_status|$upstream_http_content_type|$upstream_http_content_length|$upstream_http_content_disposition|$http_x_cat_parent_id|$upstream_scheme://$upstream_host$upstream_uri|$hostname"`

@tokers

tokers commented 2 years ago

Strange. Could you share the APISIX configuration and the steps to reproduce this issue?

`etcd: host: tls: verify: false apisix: node_listen: 80 enable_server_tokens: false admin_key: - name: "admin" role: admin

router: http: 'radixtree_host_uri' ssl: enable_http2: true listen_port: 443 disable_sync_configuration_during_start: true proxy_cache: cache_ttl: 10s zones: - name: disk_cache_one memory_size: 5G disk_size: 5G disk_path: /tmp/disk_cache_one cache_levels: 1:2 plugins: - prometheus - log-rotate - proxy-rewrite - redirect - error-log-logger - node-status - ip-restriction - echo - basic-auth - server-info - proxy-mirror - kafka-logger - response-rewrite - serverless-pre-function plugin_attr: prometheus: export_uri: /apisix/prometheus/metrics enable_export_server: false export_addr: ip: 0.0.0.0 port: 9091 log-rotate: interval: 21600 max_kept: 10 nginx_config: http_end_configuration_snippet: | reset_timedout_connection off; client_header_buffer_size 1024k; client_body_buffer_size 20m; proxy_intercept_errors on; large_client_header_buffers 8 128k; gzip on; proxy_next_upstream off; proxy_connect_timeout 60; proxy_send_timeout 300s; proxy_read_timeout 300s; proxy_buffers 16 2048k; # getconf PAGESIZE proxy_buffer_size 2048k; proxy_busy_buffers_size 2048k; proxy_temp_file_write_size 2048K; proxy_max_temp_file_size 0; proxy_ignore_client_abort on; proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header traffic-source "INTERNAL"; http: client_max_body_size: 3g enable_access_log: true lua_shared_dict: prometheus-metrics: 100m real_ip_header: "X-Forwarded-For" real_ip_from: - 0.0.0.0/0 keepalive_timeout: 600s client_header_timeout: 600s client_body_timeout: 600s send_timeout: 100s access_log_format: "$time_iso8601|$msec|$status|$request_completion|$bytes_sent|$body_bytes_sent|$realip_remote_addr|$remote_addr|$http_x_forwarded_for|$remote_user|$host|$server_name|$server_port|$server_protocol|$scheme|$request_method|$request_length|$request_time|$request_uri|$uri|$content_length|$content_type|$http_referer|$http_user_agent|$http_app_jb|$http_client_info|$upstream_addr|$upstream_connect_time|$upstream_header_time|$upstream_response_time|$upstream_status|$upstream_bytes_received|$upstream_cache_status|$upstream_http_content_type|$upstream_http_content_length|$upstream_http_content_disposition|$http_x_cat_parent_id|$upstream_scheme://$upstream_host$upstream_uri|$hostname"`

@tokers

I also need the reproduce steps and the APISIX version.

tzssangglass commented 2 years ago

@sandy420 can you try move client_body_buffer_size form http_end_configuration_snippet to http_configuration_snippet?

xuminwlt commented 2 years ago

Same question to me.

tzssangglass commented 2 years ago

Same question to me.

Try the suggestions I gave above

zjd950828 commented 2 years ago

get error of "nginx: [emerg] unknown directive "client_body_buffer_size:" in /usr/local/apisix/conf/nginx.conf:131"

tzssangglass commented 2 years ago

get error of "nginx: [emerg] unknown directive "client_body_buffer_size:" in /usr/local/apisix/conf/nginx.conf:131"

If you are trying to give feedback on a problem, you need to describe the version information and reproduce the steps, not just give a line of error log.

startzhuzhu commented 2 years ago

apisix version (cmd: apisix version): 2.14.1 OS (cmd: uname -a): Linux 3.10.0-1160.6.1.el7.x86_64 OpenResty / Nginx version (cmd: nginx -V or openresty -V): nginx version: openresty/1.21.4.1 etcd version, if have (cmd: run etcd --version):3.4.18 apisix-dashboard version, if have: 2.13.0 Browser version, if have:

tzssangglass commented 2 years ago

apisix version (cmd: apisix version): 2.14.1 OS (cmd: uname -a): Linux 3.10.0-1160.6.1.el7.x86_64 OpenResty / Nginx version (cmd: nginx -V or openresty -V): nginx version: openresty/1.21.4.1 etcd version, if have (cmd: run etcd --version):3.4.18 apisix-dashboard version, if have: 2.13.0 Browser version, if have:

hi @startzhuzhu I don't know what you are trying to say

believe1499 commented 2 years ago

@sandy420 can you try move client_body_buffer_size form http_end_configuration_snippet to http_configuration_snippet?

it works for me

zengzzzzz commented 1 year ago

thakns

qshuai commented 1 year ago

@tzssangglass moving client_body_buffer_size to the http_configuration_snippet scope not work to me.

ENV:

api version 2.15.0
uname -a: Linux apisix-89664c8cc-j4ltv 4.18.0-348.7.1.el8_5.x86_64 #1 SMP Wed Dec 22 13:25:12 UTC 2021 x86_64 Linux

openresty -V:
nginx version: openresty/1.21.4.1
built by gcc 10.3.1 20210424 (Alpine 10.3.1_git20210424)
built with OpenSSL 1.1.1g  21 Apr 2020
TLS SNI support enabled
configure arguments: --prefix=/usr/local/openresty/nginx --with-cc-opt='-O2 -DAPISIX_BASE_VER=1.21.4.1.1 -DNGX_LUA_ABORT_AT_PANIC -I/usr/local/openresty/zlib/include -I/usr/local/openresty/pcre/include -I/usr/local/openresty/openssl111/include' --add-module=../ngx_devel_kit-0.3.1 --add-module=../echo-nginx-module-0.62 --add-module=../xss-nginx-module-0.06 --add-module=../ngx_coolkit-0.2 --add-module=../set-misc-nginx-module-0.33 --add-module=../form-input-nginx-module-0.12 --add-module=../encrypted-session-nginx-module-0.09 --add-module=../srcache-nginx-module-0.32 --add-module=../ngx_lua-0.10.21 --add-module=../ngx_lua_upstream-0.07 --add-module=../headers-more-nginx-module-0.33 --add-module=../array-var-nginx-module-0.05 --add-module=../memc-nginx-module-0.19 --add-module=../redis2-nginx-module-0.15 --add-module=../redis-nginx-module-0.3.9 --add-module=../ngx_stream_lua-0.0.11 --with-ld-opt='-Wl,-rpath,/usr/local/openresty/luajit/lib -Wl,-rpath,/usr/local/openresty/wasmtime-c-api/lib -L/usr/local/openresty/zlib/lib -L/usr/local/openresty/pcre/lib -L/usr/local/openresty/openssl111/lib -Wl,-rpath,/usr/local/openresty/zlib/lib:/usr/local/openresty/pcre/lib:/usr/local/openresty/openssl111/lib' --add-module=/tmp/tmp.M4fAebVwPS/openresty-1.21.4.1/../mod_dubbo-1.0.2 --add-module=/tmp/tmp.M4fAebVwPS/openresty-1.21.4.1/../ngx_multi_upstream_module-1.1.1 --add-module=/tmp/tmp.M4fAebVwPS/openresty-1.21.4.1/../apisix-nginx-module-1.9.0 --add-module=/tmp/tmp.M4fAebVwPS/openresty-1.21.4.1/../apisix-nginx-module-1.9.0/src/stream --add-module=/tmp/tmp.M4fAebVwPS/openresty-1.21.4.1/../apisix-nginx-module-1.9.0/src/meta --add-module=/tmp/tmp.M4fAebVwPS/openresty-1.21.4.1/../wasm-nginx-module-0.6.2 --add-module=/tmp/tmp.M4fAebVwPS/openresty-1.21.4.1/../lua-var-nginx-module-v0.5.3 --with-poll_module --with-pcre-jit --with-stream --with-stream_ssl_module --with-stream_ssl_preread_module --with-http_v2_module --without-mail_pop3_module --without-mail_imap_module --without-mail_smtp_module --with-http_stub_status_module --with-http_realip_module --with-http_addition_module --with-http_auth_request_module --with-http_secure_link_module --with-http_random_index_module --with-http_gzip_static_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-threads --with-compat --with-stream --with-http_ssl_module

My config:

nginx_config:
  http_configuration_snippet: |
    client_body_buffer_size   10m;

There is a warn log like "a client request body is buffered to a temporary file xxx" in error.log when I upload a file with size 1m.

Aaron199 commented 1 year ago

it‘s not work to me My config:

http_configuration_snippet:|
    proxy_buffering on;
    proxy_buffer_size 128k;
    proxy_buffers 8 128k;
    client_body_buffer_size 10240k;

ENV:

apisix version (cmd: apisix version): 2.15.2
OS (cmd: uname -a): Linux apisix-658cfb87f-qbvbx 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47
OpenResty / Nginx version (cmd: nginx -V or openresty -V): nginx version: openresty/1.21.4.1
                  built by gcc 9.3.1 20200408 (Red Hat 9.3.1-2) (GCC)
                  built with OpenSSL 1.1.1s  1 Nov 2022
                  TLS SNI support enabled
etcd version, if have (cmd: run etcd --version): 3.5.4

XXMY commented 1 year ago

see https://github.com/apache/apisix/issues/5301

apache / apisix

help request: a client request body is buffered to a temporary file #6741

Description

Environment