fpeterschmitt
commented
8 years ago Hi,
I can confirm this behavior too, with nginx 1.8.1 and 1.10.1.
I also have multiple ldap servers configured, and I got a segfault only on successful authentication.
Open quenode opened 8 years ago
fpeterschmitt
commented
8 years ago Hi,
I can confirm this behavior too, with nginx 1.8.1 and 1.10.1.
I also have multiple ldap servers configured, and I got a segfault only on successful authentication.
hanky
commented
8 years ago I have a similar problem with nginx crashing. I use nginx 1.8.0 on Centos 6.7. The nginx config contains only one ldap server.
My nginx config:
ldap_server fs {
url ldap://example.com/ou=People,dc=test_app,dc=ru?uid?sub?(objectClass=person);
require valid_user;
}
server {
listen 443;
server_name example.com;
auth_ldap "Forbidden";
auth_ldap_servers fs;
...
}A core dump:
Core was generated by `nginx: worker process '.
Program terminated with signal 11, Segmentation fault.
#0 0x00000000004b2e95 in ngx_http_auth_ldap_return_connection ()
Missing separate debuginfos, use: debuginfo-install nginx-1.8.0-13.el6.x86_64
(gdb) bt
#0 0x00000000004b2e95 in ngx_http_auth_ldap_return_connection ()
#1 0x00000000004b3f12 in ngx_http_auth_ldap_handler ()
#2 0x0000000000445305 in ngx_http_core_access_phase ()
#3 0x00000000004421fd in ngx_http_core_run_phases ()
#4 0x00000000004b53a8 in ngx_http_auth_ldap_read_handler ()
#5 0x0000000000437941 in ngx_epoll_process_events ()
#6 0x000000000042d705 in ngx_process_events_and_timers ()
#7 0x00000000004350b5 in ngx_worker_process_cycle ()
#8 0x0000000000433504 in ngx_spawn_process ()
#9 0x000000000043524c in ngx_start_worker_processes ()
#10 0x0000000000435a83 in ngx_master_process_cycle ()
#11 0x0000000000414126 in main ()
ballock
commented
7 years ago I encountered the same behaviour as @quenode and tried investigating. For now I found that this happens when I exceed the configurable amount of LDAP servers.
What looks like is happening is that the module assumes that the array of LDAP server entries has a constant size - this is suggested by the ngx_http_auth_ldap_ldap_server_block function - where ngx_array_create is called with a constant number of members. Just below that, a member is 'pushed' to the array with ngx_array_push, and the exit code of that function is compared to NULL. (See https://github.com/kvspb/nginx-auth-ldap/blob/master/ngx_http_auth_ldap_module.c#L337)
This would all be fine if ngx_array_push would return a NULL when exceeding the size of the array. However, in nginx's sources, in src/core/ngx_array.c you can see that it does its best to extend the array first, and only if it fails, would it return a NULL.
In the meantime, in ngx_http_auth_ldap_init_connections, the servers array is initialized with a new queue in servers[x]->free_connections. When the segfault happens, it's about free_connections which is not initialized for the server member. This gets traced to get executed from ngx_worker_process_init.
This leads to suggest that the array gets extended with a new member after the current members were initialized with free_connections queue.
I am yet looking in the code when the module configuration functions get executed, and when the worker gets init, but the fact this error happens suggests that it's first the worker that gets initialized, then the configuration gets parsed. If this is true, the queue initialization code would need to be moved.
ballock
commented
7 years ago I think I found the reason for this behaviour.
I have a number of nginx .conf files (9 actually), each per site:
ldap_server $name {
url ldaps://url;
require valid_user;
}
server {
server_name name
...
location / {
auth_ldap_servers $name
...
}
}The default auth_ldap_servers_size (servers_size) is 7. What this means is that when the configuration parser approaches the first section - ldap_server, it calls the ngx_http_auth_ldap_ldap_server_block function and allocates an array for the servers:
cnf->servers = ngx_array_create(cf->pool, cnf->servers_size, sizeof(ngx_http_auth_ldap_server_t));
Then, the parser moves on, and finds the server { stanza, and the location / { stanza underneath, and the auth_ldap_servers directive, for which it fires up the ngx_http_auth_ldap_servers function. This function sets up its own array of servers specifically for the loc_conf (location configuration), and since the location configuration refer to a particular ldap_server $name from the main configuration, it assumes it is safe to use a pointer to the particular mconf->servers->elts that corresponds to the matching $name (note I just copy-pasted some lines from the module to better illustrate the problem, this is not a whole excerpt from it):
s = &((ngx_http_auth_ldap_server_t *) mconf->servers->elts)[j];
server = s;
target = (ngx_http_auth_ldap_server_t **) ngx_array_push(cnf->servers);
*target = server;Here, mconf is the main configuration structure with the ldap_server stanza and cnf is the location configuration.
When the second (and up to 7th configuration file) is parsed, the ngx_http_auth_ldap_ldap_server_block function adds each new ldap_server to the array with:
server = ngx_array_push(cnf->servers);
and each location config uses a pointer to refer to the server.
Until now everything is fine. However, when we reach past the 7th element in the main configuration, and the ngx_http_auth_ldap_ldap_server_block function calls
server = ngx_array_push(cnf->servers);
what nginx does is it extends the array:
if (a->nelts == a->nalloc) {
new = ngx_palloc(p, 2 * size);
ngx_memcpy(new, a->elts, size);
a->elts = new;
a->nalloc *= 2;Now, the main configuration contains an array with 8 servers, each in a new memory location. The upcoming location will have a server pointer to the new array, but all the remaining 7 locations contain a pointer to a memory that is no longer occupied by the main config's servers array.
Since ngx_http_auth_ldap_init_connections is called later, and it is the function that initializes the free_connections and waiting_requests queues in the main configuration, all the 7 location point to memory locations that don't have these queues initialized. And when ngx_http_auth_ldap_return_connection is called, execution proceeds to
if (!ngx_queue_empty(&c->server->waiting_requests)) {and it's quite interesting to find that a non-initialized queue is returns from the call as a non-empty one! It's because ngx_queue_empty is actually a macro for (h == (h)->prev) and h->prev is 0x0, while h is a pointer. And that in turn leads to an attempt to remove a 0x0 element from the queue.
ibexmonj
commented
6 years ago Any fix around for this ?
ubuntu 14.04 nginx 1.10
Crashes when having to much ldap server in virtual hosts.
this what I use
Free IPA LDAP