Open junos opened 6 years ago
@junos Your example is not self-contained nor minimal so I cannot really run your case directly on our side and try to reproduce the problem.
My hunch is that your nginx config has internal redirects that bypass the log_by_lua*
handler which decrements the counter by calling lim:leaving()
. This is the common reason for an ever incrementing counter.
@junos BTW, you can (temporarily) enable the nginx debugging logs in your OpenResty to confirm the internal redirects or other important details of your request. If you are using the binary pre-built packages provided by OpenResty, then you can simply switch to the openresty-debug
package temporarily and configure error_log logs/error.log debug;
in your nginx.conf
(for all the error_log
directives there). If you are compiling OpenResty from source, then you should pass the --with-debug
option to the ./configure
command and still always use error_log logs/error.log debug;
in your nginx.conf
consistently.
@agentzh, I enable the debug log, and I didn't find any internal redirect. I have no clue why this happened. On the other hand, For our production code, concurrent counting working well at some concurrent level, but when the concurrent level goes really high like > 1000, counting keep growing, Is it possible for this? Out-of-Sync Counter Prevention
https://github.com/openresty/lua-resty-limit-traffic/blob/master/lib/resty/limit/conn.md#out-of-sync-counter-prevention, Is there any way when this happened, we could reset the counting number?
@junos You should check if your nginx workers ever crashed. Crashes must always be fixed.
Here are steps to re-produce the issue I have:
1.11.2.5
on an EC2 instance withconcurrent connections counting
, I install npm packageartillery
at my local machine, https://www.npmjs.com/package/artillery bynpm install -g artillery
I expect concurrent connections counting return to 1 after I stop make concurrent request, but that counting number keep increasing without reset
My question is: Did I do something wrong or this is an issue for
lua-resty-limit-traffic
This is my config:
Here is partial of the result: ............ 2017/10/31 20:28:52 [error] 589#0: 25 [lua] log_by_lua(default:86):20:
concurrent connections =49
while logging request, client: 10.1.254.13, server: , request: "GET / HTTP/1.1", host: "10.1.17.130" 2017/10/31 20:28:52 [error] 589#0: 25 [lua] log_by_lua(default:86):20:concurrent connections =50
while logging request, client: 10.1.254.13, server: , request: "GET / HTTP/1.1", host: "10.1.17.130" .............