Not insert the same request at ngx_http_auth_ldap_get_connection()

eramoto commented 7 years ago

When inserting the same request to waiting_requests queue twice, the queue will be broken. In addition, the following segmentation fault occurs at the second ngx_http_auth_ldap_return_connection() if the nginx binary was compiled with --with-debug. Related issues: #104 #132 #143

By this fix, ngx_http_auth_ldap_get_connection() will not insert the same request.

debug messages at the moment nginx dumped core

2016/07/26 13:19:09 [debug] 4299#0: *2 http_auth_ldap: Authentication loop (phase=0, iteration=0)
2016/07/26 13:19:09 [debug] 4299#0: *2 event timer add: 3: 10000:1469506759827
2016/07/26 13:19:09 [debug] 4299#0: *2 http_auth_ldap: request_timeout=10000
2016/07/26 13:19:09 [debug] 4299#0: *2 http_auth_ldap: Authentication loop (phase=1, iteration=0)
2016/07/26 13:19:09 [debug] 4299#0: *2 http_auth_ldap: Wants a free connection to "test_ldap"
2016/07/26 13:19:09 [debug] 4299#0: *2 http_auth_ldap: No connection available at the moment, waiting...
2016/07/26 13:19:09 [debug] 4299#0: *2 http run request: "/portal/Image?"
2016/07/26 13:19:09 [debug] 4299#0: *2 access phase: 6
2016/07/26 13:19:09 [debug] 4299#0: *2 http_auth_ldap: Authentication loop (phase=1, iteration=0)
2016/07/26 13:19:09 [debug] 4299#0: *2 http_auth_ldap: Wants a free connection to "test_ldap"
2016/07/26 13:19:09 [debug] 4299#0: *2 http_auth_ldap: No connection available at the moment, waiting...
<snip>
2016/07/26 13:19:09 [debug] 4299#0: *5 http_auth_ldap: Authentication loop (phase=6, iteration=1)
2016/07/26 13:19:09 [debug] 4299#0: *5 event timer del: 12: 1469506759826
2016/07/26 13:19:09 [debug] 4299#0: http_auth_ldap: Marking the connection to "test_ldap" as free
<snip>
2016/07/26 13:19:09 [debug] 4299#0: *2 http_auth_ldap: Authentication loop (phase=6, iteration=1)
2016/07/26 13:19:09 [debug] 4299#0: *2 event timer del: 3: 1469506759827
2016/07/26 13:19:09 [debug] 4299#0: http_auth_ldap: Marking the connection to "test_ldap" as free
2016/07/26 13:19:09 [notice] 4298#0: signal 17 (SIGCHLD) received
2016/07/26 13:19:09 [alert] 4298#0: worker process 4299 exited on signal 11 (core dumped)

backtrace

Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000000000491b73 in ngx_http_auth_ldap_return_connection (c=0x26a84e8) at /tmp/test/nginx-1.11.2/../nginx-auth-ldap/ngx_http_auth_ldap_module.c:1117
1117                ngx_queue_remove(q);
(gdb) bt
#0  0x0000000000491b73 in ngx_http_auth_ldap_return_connection (c=0x26a84e8) at /tmp/test/nginx-1.11.2/../nginx-auth-ldap/ngx_http_auth_ldap_module.c:1117
#1  0x000000000049496a in ngx_http_auth_ldap_authenticate (conf=<optimized out>, ctx=0x26e4698, r=0x26e35a0)
  at /tmp/test/nginx-1.11.2/../nginx-auth-ldap/ngx_http_auth_ldap_module.c:1902
#2  ngx_http_auth_ldap_handler (r=0x26e35a0) at /tmp/test/nginx-1.11.2/../nginx-auth-ldap/ngx_http_auth_ldap_module.c:1717
#3  0x0000000000446c38 in ngx_http_core_access_phase (r=<optimized out>, ph=0x26a7f48) at src/http/ngx_http_core_module.c:1071
#4  0x00000000004428a3 in ngx_http_core_run_phases (r=r@entry=0x26e35a0) at src/http/ngx_http_core_module.c:845
#5  0x0000000000491ab7 in ngx_http_auth_ldap_wake_request (r=0x26e35a0) at /tmp/test/nginx-1.11.2/../nginx-auth-ldap/ngx_http_auth_ldap_module.c:1063
#6  0x0000000000491c44 in ngx_http_auth_ldap_reply_connection (c=c@entry=0x26a84e8, error_code=0, error_msg=<optimized out>)
  at /tmp/test/nginx-1.11.2/../nginx-auth-ldap/ngx_http_auth_ldap_module.c:1141
#7  0x00000000004956b8 in ngx_http_auth_ldap_read_handler (rev=0x265d500) at /tmp/test/nginx-1.11.2/../nginx-auth-ldap/ngx_http_auth_ldap_module.c:1486
#8  0x00000000004384fe in ngx_epoll_process_events (cycle=0x26594e0, timer=<optimized out>, flags=<optimized out>) at src/event/modules/ngx_epoll_module.c:900
#9  0x000000000042ea85 in ngx_process_events_and_timers (cycle=cycle@entry=0x26594e0) at src/event/ngx_event.c:242
#10 0x0000000000435fe0 in ngx_worker_process_cycle (cycle=0x26594e0, data=<optimized out>) at src/os/unix/ngx_process_cycle.c:753
#11 0x0000000000434942 in ngx_spawn_process (cycle=cycle@entry=0x26594e0, proc=proc@entry=0x435f3d <ngx_worker_process_cycle>, data=data@entry=0x0,
  name=name@entry=0x49abb7 "worker process", respawn=respawn@entry=-3) at src/os/unix/ngx_process.c:198
#12 0x000000000043613d in ngx_start_worker_processes (cycle=cycle@entry=0x26594e0, n=1, type=type@entry=-3) at src/os/unix/ngx_process_cycle.c:358
#13 0x0000000000436bd6 in ngx_master_process_cycle (cycle=cycle@entry=0x26594e0) at src/os/unix/ngx_process_cycle.c:130
#14 0x000000000041168b in main (argc=<optimized out>, argv=<optimized out>) at src/core/nginx.c:367

eholzbach commented 7 years ago

@eramoto I hit this bug. Redeployed with your commit, worker process segfaults stopped. Thanks!

mfulz commented 7 years ago

Thanks your patch is working perfectly for me.

fpeterschmitt commented 7 years ago

The patch seems to be working great as well here: cluster of two nginx 1.10.1 reverse proxy, reproducable segfaults stopped. Thanks!

alexander-gridnev commented 7 years ago

Thanks, patch works for me! Have no segfaults since applying it.

kvspb / nginx-auth-ldap

Not insert the same request at ngx_http_auth_ldap_get_connection() #147