openresty / lua-resty-redis

Lua redis client driver for the ngx_lua based on the cosocket API
1.91k stars 448 forks source link

set_keepalive() is failing during small load, 10 req/s #282

Open nikhilrakuten opened 1 month ago

nikhilrakuten commented 1 month ago

We are using this modules with nginx 1.16 , its working perfectly fine. Recently we are planning to upgrade the nginx to 1.24 or 1.25.1 version. Its failing there

System Information Os : Ubuntu 22.04 Nginx version : 1.25.1 lua-nginx-module: 0.10.26 / 0.10.26rc1 resty redis : https://github.com/openresty/lua-resty-redis

We are building this nginx from scratch and not using openresty bundle. we are passing "--add-module=/build/lua-nginx-module-0.10.26" during the build phase and nginx -V to confirm the installation.

Installation Steps ./configure --prefix=/opt/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -ffile-prefix-map=/data/builder/debuild/nginx-1.24.0/debian/debuild-base/nginx-1.24.0=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-Wl,-rpath,/usr/local/lib,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie' --add-module=/build/ngx_devel_kit-0.3.3 --add-module=/build/lua-nginx-module-0.10.26 --add-module=/build/ngx_http_auth_pam_module-1.5.5 --add-module=/build/headers-more-nginx-module-0.36

Error Details If one user is accessing the webapge, keepalive is working fine. But if we do small load test with 10 virutal users, keepalive() starts failing. But not all requests are failing. lua entry thread aborted: runtime error: /etc/nginx/lua/file_name.lua:81: bad request

Few sample logs:

2024/09/20 13:47:53 [crit] 845145#845145: 57313 [lua] File_Name.lua:82: keepalive_release(): keepalive-configured:1, client: 133.237.1.2, server: HOSTNAME, request: "GET /testing HTTP/1.1", host: "HOSTNAME" 2024/09/20 13:47:53 [crit] 845145#845145: 57313 [lua] File_Name.lua:82: keepalive_release(): keepalive-configured:1, client: 133.237.1.2, server: HOSTNAME, request: "GET /testing HTTP/1.1", host: "HOSTNAME" 2024/09/20 13:47:53 [error] 845144#845144: 57361 lua entry thread aborted: runtime error: /etc/nginx/lua/File_Name.lua:81: bad request stack traceback: coroutine 0: [C]: in function 'set_keepalive' /etc/nginx/lua/File_Name.lua:81: in function 'keepalive_release' /etc/nginx/lua/File_Name.lua:136: in main chunk, client: 133.237.1.2, server: HOSTNAME, request: "GET /testing HTTP/1.1", host: "HOSTNAME" 2024/09/20 13:47:53 [crit] 845145#845145: 57313 [lua] File_Name.lua:82: keepalive_release(): keepalive-configured:1, client: 133.237.1.2, server: HOSTNAME, request: "GET /testing HTTP/1.1", host: "HOSTNAME"

Testcase configure a path /test , which will call lua file. It will use lua resty redis to create new object / connection. Do redis operation call set_keepalive() method. [ client:set_keepalive(10000, 200) ] Using apache bench : ab -k -c 50 -n 500 https://host/test

nikhilrakuten commented 1 month ago

I have raised this as well, still facing the problem https://github.com/openresty/lua-nginx-module/issues/2364

nikhilrakuten commented 1 month ago

I have tried with openresty as well. problem remains same.