Open Cibifang opened 5 years ago
@Cibifang Try removing any 3rd-party nginx C modules or 3rd-party Lua modules. The crash site is apparently not the culprit, but the consequence of earlier memory corruptions.
Another way to debug such things is to use valgrind to run your app as a whole. But make sure you've applied the "no-pool" patch for the nginx core and enforce the system allocator in your LuaJIT (if you use the OpenResty pre-built packages, then openresty-valgrind package can do the trick). See below for more details:
https://openresty.gitbooks.io/programming-openresty/content/testing/test-modes.html#_valgrind_mode
Sorry for the late reply BTW.
BTW2: please upgrade to the latest OpenResty to avoid running into any known bugs that have already been fixed. OpenResty 1.11.2.2 is ancient.
The latest version is below:
We have very complex and large-scale deployment of OpenResty for long and we're not seeing even a single core dump file for quite a long time.
Hi.
Recently I met some nginx carshes(not frequent, but appear). I checked the error log and coredump, found that it appeared when ngx.exit with HTTP_NOT_FOUND.
The coredump:
I checked the code, it's in below:
Then I checked the
u
andr
, found these are garbled as below.I also went up and checked
request
inngx_http_free_request
, found the request is right. So the only problem seems that thengx_http_lua_socket_tcp_upstream_t
is wrong?But the problem is that I can't reproduce it and it doesn't appear frequently(about one in a million). Some details: It only appears on specific machines and specific time. I think it is related to high concurrency.
Solutions and further debug methods are both helpful. Thanks!
openresty version: 1.11.2.2 operating system version: Linux version 3.16.0-4-amd64 (debian-kernel@lists.debian.org)
ps: I only found two related issues(#110 and #192). and these both seem not same with this case.