slact / nchan

Fast, horizontally scalable, multiprocess pub/sub queuing server and proxy for HTTP, long-polling, Websockets and EventSource (SSE), powered by Nginx.
https://nchan.io/
Other
3.01k stars 292 forks source link

worker process 107 exited on signal 11 (core dumped) set_real_ip_from with nchan_authorize_request #680

Open Kolobok12309 opened 9 months ago

Kolobok12309 commented 9 months ago

Nginx crashed if i use set_real_ip_from (and pass header X-Real-Ip) and use nchan_authorize_request

set_real_ip_from 0.0.0.0/0;

location = /auth {
    proxy_pass ${WS_AUTH_URL};
    proxy_pass_request_body off;
    proxy_set_header Content-Length "";
    proxy_set_header X-Subscriber-Type $nchan_subscriber_type;
    proxy_set_header X-Publisher-Type $nchan_publisher_type;
    proxy_set_header X-Prev-Message-Id $nchan_prev_message_id;
    proxy_set_header X-Channel-Id $nchan_channel_id;
    proxy_set_header X-Original-URI $request_uri;
    proxy_set_header X-Forwarded-For $remote_addr;
  }

location ~ ^/user/(?<user>(\d+))$ {
  nchan_authorize_request /auth;
  nchan_subscriber;
  nchan_channel_id "users" $user;
  nchan_channel_group "users";
}

If i comment nchan_authorize_request or set_real_ip_from or not pass header, all be fine

Nginx 1.25.2-alpine Nchan 1.3.6

Core dump

Core was generated by `nginx: worker process     '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000557b2732b464 in ngx_cidr_match ()
(gdb) backtrace
#0  0x0000557b2732b464 in ngx_cidr_match ()
#1  0x0000557b2736b27b in ?? ()
#2  0x0000557b2736e42f in ngx_http_get_forwarded_addr ()
#3  0x0000557b273b1717 in ?? ()
#4  0x0000557b2736c409 in ngx_http_core_generic_phase ()
#5  0x0000557b27368a5f in ngx_http_core_run_phases ()
#6  0x0000557b27370645 in ngx_http_run_posted_requests ()
#7  0x00007f8407c89e0f in nchan_requestmachine_run (rm=rm@entry=0x7f8407be35c0)
    at //nchan-1.3.6/src/util/nchan_fake_request.c:326
#8  0x00007f8407c8a772 in nchan_requestmachine_request (rm=0x7f8407be35c0, params=params@entry=0x7ffc1e2f3070)
    at //nchan-1.3.6/src/util/nchan_fake_request.c:489
#9  0x00007f8407c9338e in nchan_subscriber_subrequest (sub=sub@entry=0x7f8407be3800, 
    params=params@entry=0x7ffc1e2f3070) at //nchan-1.3.6/src/subscribers/common.c:315
#10 0x00007f8407c938bd in nchan_subscriber_authorize_subscribe_request (sub=0x7f8407be3800, 
    ch_id=0x7f8407c5f7b0) at //nchan-1.3.6/src/subscribers/common.c:128
#11 0x00007f8407c80484 in nchan_pubsub_handler (r=0x7f8407c5e880) at //nchan-1.3.6/src/nchan_module.c:793
#12 0x0000557b2736d388 in ngx_http_core_content_phase ()
#13 0x0000557b27368a5f in ngx_http_core_run_phases ()
#14 0x0000557b27372986 in ?? ()
#15 0x0000557b27372cd2 in ?? ()
#16 0x0000557b27349361 in ?? ()
#17 0x0000557b2734014b in ngx_process_events_and_timers ()
#18 0x0000557b273478fd in ?? ()
slact commented 8 months ago

Do you have a separate upstream block that proxy_pass refers to?

Kolobok12309 commented 8 months ago

Do you have a separate upstream block that proxy_pass refers to?

Yea, nchan placed after another nginx proxy Or, mb i'm wrong understand you, ${WS_AUTH_URL} is env with url(not upstream). Example http://tests-auth:4000/ws

slact commented 8 months ago

I suspect if you change the var to refer to an upstream block by name, this will work.

Upstreams routed by URL alone don't travel the same code path as ones that go though an upstream block. This has always been a pain in the ass, even without Nchan.

Kolobok12309 commented 8 months ago

I try it yesterday, same error

upstream auth {
  server tests-auth:4000; # Docker instance
}

server {
  ...
  location = /auth {
    proxy_pass http://auth/ws;
    ...
  }
  ...
}

Mb maybe I'm using upstream wrong?

Kolobok12309 commented 8 months ago

But all worked if i move set_real_ip_from 0.0.0.0/0; to location block (use it for allow/deny access to publication, pub and sub separated location).

From to From ``` server { set_real_ip_from 0.0.0.0/0; location = /auth { proxy_pass ${WS_AUTH_URL}; proxy_pass_request_body off; proxy_set_header Content-Length ""; proxy_set_header X-Subscriber-Type $nchan_subscriber_type; proxy_set_header X-Publisher-Type $nchan_publisher_type; proxy_set_header X-Prev-Message-Id $nchan_prev_message_id; proxy_set_header X-Channel-Id $nchan_channel_id; proxy_set_header X-Original-URI $request_uri; proxy_set_header X-Forwarded-For $remote_addr; } location ~ ^/user/(?(\d+))$ { nchan_authorize_request /auth; nchan_subscriber; nchan_channel_id "users" $user; nchan_channel_group "users"; } location ~ ^/pub/user/(?(\d+))$ { access 127.0.0.1; # In real case more ips deny all; nchan_publisher; nchan_channel_id $user; nchan_channel_group "users"; } } ``` To ``` server { location = /auth { proxy_pass ${WS_AUTH_URL}; proxy_pass_request_body off; proxy_set_header Content-Length ""; proxy_set_header X-Subscriber-Type $nchan_subscriber_type; proxy_set_header X-Publisher-Type $nchan_publisher_type; proxy_set_header X-Prev-Message-Id $nchan_prev_message_id; proxy_set_header X-Channel-Id $nchan_channel_id; proxy_set_header X-Original-URI $request_uri; proxy_set_header X-Forwarded-For $remote_addr; } location ~ ^/user/(?(\d+))$ { nchan_authorize_request /auth; nchan_subscriber; nchan_channel_id "users" $user; nchan_channel_group "users"; } location ~ ^/pub/user/(?(\d+))$ { set_real_ip_from 0.0.0.0/0; access 127.0.0.1; # In real case more ips deny all; nchan_publisher; nchan_channel_id $user; nchan_channel_group "users"; } } ```
slact commented 8 months ago

Huh, that's wild. I'll see if I can fix this.

Kolobok12309 commented 8 months ago

Huh, that's wild. I'll see if I can fix this.

I can create minimal repro in docker containers if you need

slact commented 8 months ago

Nah I got it. This is one of those bugs that's about struggling with Nginx guts.