openresty / lua-resty-memcached

Lua memcached client driver for the ngx_lua based on the cosocket API
211 stars 63 forks source link

idle connections to memcached #47

Open RussianCatYakov opened 10 months ago

RussianCatYakov commented 10 months ago

Hi, I have the following code executed on each request:

    access_by_lua_block {
      local memcached = require "resty.memcached"
      local memc = memcached:new()
      memc:set_timeout(3000) -- 3 sec
      local ok = memc:connect("127.0.0.1", 11211)
      local myvar = memc:get(othervar)
      ### do something
      local ok, err = memc:close()
    }

put operation is happening in other internal location in case proxy origin receives 302 redirect

    access_by_lua_block {
      local memcached = require "resty.memcached"
      local memc = memcached:new()
      memc:set_timeout(1000) -- 1 sec
      local ok = memc:connect("127.0.0.1", 11211)
      ### do something
      local ok, err = memc:set(stringone, stringtwo, 604800)
      local ok, err = memc:close()
    }

Recently I noticed that connections are not really closed but stay idle for some time. And it's count is pretty high. I have memcached running in docker contrainer like this

memcached -c 10000 -t 8

nginx itself handles 4k rps nonstop

And netstat shows this

netstat -an | grep 11211 
tcp        0      0 0.0.0.0:11211           0.0.0.0:*               LISTEN     
tcp        0      0 127.0.0.1:62334         127.0.0.1:11211         TIME_WAIT  
tcp        0      0 127.0.0.1:59430         127.0.0.1:11211         TIME_WAIT  
tcp        0      0 127.0.0.1:61444         127.0.0.1:11211         TIME_WAIT  
tcp        0      0 172.17.0.1:11638        172.17.0.2:11211        TIME_WAIT  
tcp        0      0 127.0.0.1:45352         127.0.0.1:11211         TIME_WAIT  
tcp        0      0 127.0.0.1:62010         127.0.0.1:11211         TIME_WAIT  
tcp        0      0 127.0.0.1:56390         127.0.0.1:11211         TIME_WAIT  
tcp        0      0 172.17.0.1:31884        172.17.0.2:11211        TIME_WAIT  
tcp        0      0 172.17.0.1:64364        172.17.0.2:11211        TIME_WAIT  
...

netstat -an | grep -c 11211 
31836

lua module version is 0.16

Please help me solve this issue, thank you in advance.

zhuizhuhaomeng commented 10 months ago

please don't call memc:close() to close the connection after the put operation. Instead, you should use memc:set_keepalive to reuse the connection.

 local ok, err = memc:set_keepalive(10000, 100)
                if not ok then
                    ngx.say("cannot set keepalive: ", err)
                    return
                end