help request: Questions Regarding the `least_conn` Load Balancer

[diaosj]

Description

Hi,

I have some questions about the least_conn load balancing algorithm and would appreciate some clarification:

1. I want to confirm my understanding: When using least_conn, the metric used is the number of connections in Nginx rather than the actual number of requests to the upstream servers. To achieve load balancing based on the number of requests, would it be necessary to configure Nginx to use a single worker and disable keepalive? Specifically, are these the configurations to adjust?

   nginx_config:
   worker_processes: 1
   enable_cpu_affinity: false
   worker_rlimit_nofile: 20480
   worker_shutdown_timeout: 240s
   event:
   worker_connections: 10620
   http:
   keepalive_timeout: 0s
   upstream:
     keepalive: 320
     keepalive_requests: 1000
     keepalive_timeout: 0s

   By doing so, would this effectively balance the load based on the number of requests received by the upstream servers?

2. Upon reviewing the code for the least_conn algorithm, I noticed that the weight is updated in the after_balance() method, which is called during the http_log_phase. init.lua

   if api_ctx.server_picker and api_ctx.server_picker.after_balance then
       api_ctx.server_picker.after_balance(api_ctx, false)
   end

least_conn.lua

            local info = servers_heap:valueByPayload(server)
            info.score = info.score - info.effect_weight
            servers_heap:update(server, info)

At this stage, has the request/connection already been released? Could updating at this time cause a brief inaccuracy in the connection count? Might this lead to a slight imbalance in the distribution of a small number of requests?

Thank you for your time and assistance.

Environment

• APISIX version (run apisix version): 2.7
• Operating system (run uname -a): Linux ti-ai-gateway-yuewen-84f46c6c74-2s5qt 5.4.119-1-tlinux4-0009-eks #1 SMP Wed Jan 3 17:12:20 CST 2024 x86_64 x86_64 x86_64 GNU/Linux
• OpenResty / Nginx version (run openresty -V or nginx -V):

  
  nginx version: openresty/1.19.3.2
  built by gcc 9.3.1 20200408 (Red Hat 9.3.1-2) (GCC) 
  built with OpenSSL 1.1.1k  25 Mar 2021
  TLS SNI support enabled
  configure arguments: --prefix=/usr/local/openresty/nginx --with-cc-opt='-O2 -DNGX_LUA_ABORT_AT_PANIC -I/usr/local/openresty/zlib/include -I/usr/local/openresty/pcre/include -I/usr/local/openresty/openssl111/include' --add-module=../ngx_devel_kit-0.3.1 --add-module=../echo-nginx-module-0.62 --add-module=../xss-nginx-module-0.06 --add-module=../ngx_coolkit-0.2 --add-module=../set-misc-nginx-module-0.32 --add-module=../form-input-nginx-module-0.12 --add-module=../encrypted-session-nginx-module-0.08 --add-module=../srcache-nginx-module-0.32 --add-module=../ngx_lua-0.10.19 --add-module=../ngx_lua_upstream-0.07 --add-module=../headers-more-nginx-module-0.33 --add-module=../array-var-nginx-module-0.05 --add-module=../memc-nginx-module-0.19 --add-module=../redis2-nginx-module-0.15 --add-module=../redis-nginx-module-0.3.7 --add-module=../ngx_stream_lua-0.0.9 --with-ld-opt='-Wl,-rpath,/usr/local/openresty/luajit/lib -L/usr/local/openresty/zlib/lib -L/usr/local/openresty/pcre/lib -L/usr/local/openresty/openssl111/lib -Wl,-rpath,/usr/local/openresty/zlib/lib:/usr/local/openresty/pcre/lib:/usr/local/openresty/openssl111/lib' --with-cc='ccache gcc -fdiagnostics-color=always' --with-pcre-jit --with-stream --with-stream_ssl_module --with-stream_ssl_preread_module --with-http_v2_module --without-mail_pop3_module --without-mail_imap_module --without-mail_smtp_module --with-http_stub_status_module --with-http_realip_module --with-http_addition_module --with-http_auth_request_module --with-http_secure_link_module --with-http_random_index_module --with-http_gzip_static_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-threads --with-compat --with-stream --with-http_ssl_module```

• etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info):
• APISIX Dashboard version, if relevant:
• Plugin runner version, for issues related to plugin runners:
• LuaRocks version, for installation issues (run luarocks --version):

[theweakgod]

1. To achieve load balancing based on the number of requests, would it be necessary to configure Nginx to use a single worker and disable keepalive

least active connection.

[shreemaan-abhishek]

To achieve load balancing based on the number of requests, would it be necessary to configure Nginx to use a single worker and disable keepalive? Specifically, are these the configurations to adjust?

I'm saying this by my obvious reasoning that you don't have to do this to make it work. If least-conn was supposed to be used this way, it would make no sense.

[diaosj]

To achieve load balancing based on the number of requests, would it be necessary to configure Nginx to use a single worker and disable keepalive? Specifically, are these the configurations to adjust?

I'm saying this by my obvious reasoning that you don't have to do this to make it work. If least-conn was supposed to be used this way, it would make no sense.

Thank you for your response.

May I clarify if this implies that the least_conn algorithm is independent of the keepalive setting? In my understanding, if keepalive connections are reused, wouldn't this result in an inaccurate count of connections for each node in the upstream? Greatly appreciate your insight on this matter.

[theweakgod]

wouldn't this result in an inaccurate count of connections for each node in the upstream

least active connection. When the pool has an idle connection, the least active connection policy will be satisfied. Worker will count the upstream with the smallest number of active connections. Connection will not close if keepalive and http request with header connection: keep-alive. Worker will monitor the status of this connection in pool.