ajp_cache_path causes segmentation fault on Nginx 1.8.0

[davidjb]

Using the latest version of this code, Nginx 1.8.0 seg faults on startup when the following is present within the configuration, noting this used to work with Nginx 1.6.3:

ajp_cache_path     /var/cache/nginx/ajp_cache  levels=1:2
                       keys_zone=ajp_cache:60m
                       inactive=100m;

Using gdb to get a backtrace shows:

Program received signal SIGSEGV, Segmentation fault.
ngx_array_push (a=0x0) at src/core/ngx_array.c:54
54          if (a->nelts == a->nalloc) {
(gdb) bt
#0  ngx_array_push (a=0x0) at src/core/ngx_array.c:54
#1  0x000000000045f907 in ngx_http_file_cache_set_slot (cf=0x7fffffffe0d0,
    cmd=0x6fe450, conf=0x0) at src/http/ngx_http_file_cache.c:2432
#2  0x000000000041a486 in ngx_conf_handler (cf=0x7fffffffe0d0, filename=0x0)
    at src/core/ngx_conf_file.c:391
#3  ngx_conf_parse (cf=0x7fffffffe0d0, filename=0x0)
    at src/core/ngx_conf_file.c:247
#4  0x0000000000435c9a in ngx_http_block (cf=0x7fffffffe0d0,
    cmd=<value optimized out>, conf=<value optimized out>)
    at src/http/ngx_http.c:240
#5  0x000000000041a486 in ngx_conf_handler (cf=0x7fffffffe0d0,
    filename=0x730f58) at src/core/ngx_conf_file.c:391
#6  ngx_conf_parse (cf=0x7fffffffe0d0, filename=0x730f58)
    at src/core/ngx_conf_file.c:247
#7  0x0000000000417cf6 in ngx_init_cycle (old_cycle=0x7fffffffe180)
    at src/core/ngx_cycle.c:264
#8  0x0000000000409196 in main (argc=<value optimized out>,
    argv=<value optimized out>) at src/core/nginx.c:345

[jbergstroem]

When you say 'latest version' do you mean latest git revision or released version? If you haven't been using git, could you try the latest revision (bf6cd93f2098b59260de8d494f0f4b1f11a84627)?

[davidjb]

The latest from git at the time of my post, which is that revision. I had to double-check because of the commit message, indicating compatibility fixes.

[itpp16]

Can you try this; Line 176, .. - key = ngx_array_push(&r->cache->keys); .. + key = ngx_array_push(&r->keys);

See also https://github.com/FRiCKLE/ngx_cache_purge/blob/master/ngx_cache_purge_module.c line 1381, a quick look, this looks similar to what ajp is doing. Or its a missing init via ngx_http_cache_t *c; (rc = ...)

[davidjb]

@itpp16 That diff breaks compilation:

cc -c -pipe  -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -O2 -g   -I src/core -I src/event -I src/event/modules -I src/os/unix -I /root/rpmbuild/BUILD/nginx-1.8.0/nginx_ajp_module -I /usr/include/libxml2 -I objs -I src/http -I src/http/modules -I src/mail \
                -o objs/addon/nginx_ajp_module/ngx_http_ajp_handler.o \
                /root/rpmbuild/BUILD/nginx-1.8.0/nginx_ajp_module/ngx_http_ajp_handler.c
/root/rpmbuild/BUILD/nginx-1.8.0/nginx_ajp_module/ngx_http_ajp_handler.c: In function ‘ngx_http_ajp_create_key’:
/root/rpmbuild/BUILD/nginx-1.8.0/nginx_ajp_module/ngx_http_ajp_handler.c:176: error: ‘ngx_http_request_t’ has no member named ‘keys’
make[1]: *** [objs/addon/nginx_ajp_module/ngx_http_ajp_handler.o] Error 1

I don't know enough about the internals of Nginx to judge on the latter.

[itpp16]

I know :( Still looking for a solution. Without the cache ajp does work fine.

[vozlt]

I fixed this issue. Give it a fly, please give me the result. If there is no problem, then I will try pull request.

Latest commit: https://github.com/vozlt/nginx_ajp_module/commit/3b41fcc14381ec167bc610d33c8fdeed24734701 Branch: https://github.com/vozlt/nginx_ajp_module/tree/vozlt-patch-0

[itpp16]

Good news is it works but during cache initialization (nginx startup) it crashes a worker, only once but still indicating some issue somewhere. I'll try to debug later, maybe https://github.com/davidjb can try and do a backtrace to see if its still happening at the ngx_array_push point.

[itpp16]

Its not array_push, its crashing a worker when the cache manager is called to cleanup or refill the old cache, caching by itself is not working either. Still looking deeper to see whats going on.

[jbergstroem]

@itpp16 any progress to report? sorry for ping, just very keen on hearing more about it.

[itpp16]

Me too :) haven't had more time yet for debugging.

[itpp16]

No progress in debugging but I did made a workaround, please try this to see if this works for you:

http {

proxy_cache_path  ajp_temp/ajp_cache levels=1:2 keys_zone=ajp_cache_zone:10m inactive=24h max_size=2m;

[...]

server {
    listen       880;

[...]

    location /ajptest1/ {
        proxy_pass            http://127.0.0.1:880/ajptest2/;
        proxy_cache           ajp_cache_zone;
        proxy_cache_key       "$host$request_uri$cookie_user";
        proxy_cache_valid     200 1d;
        proxy_cache_use_stale error timeout invalid_header updating http_500;
        add_header X-Cache $upstream_cache_status;
    }
    location /ajptest2/ {
        rewrite    /ajptest2/([^/]+) /$1 break;
        ajp_pass tomcats/;
    }

It's basically a local loop using default caching, tested here and works fine.

[aosaginohi]

We had the same problem today also with this on nginx 1.9.11.

[r4770]

Hi! still always open the issue.. I have the same segmentation fault on Nginx 1.19.2 when i try to configure cache path zone.