403 when trying to use X-Accel-Redirect for websocket

[rs-orlov]

Hi.

I have following testing configuration:

server {
    listen      80;
    server_name polygon;
    index       index.php;

    location = /sub_upstream {
        proxy_pass http://polygon/index.php;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }

    location ~ /sub/internal-ws/(\w+)$ {
        internal;
        nchan_subscriber    websocket;
        nchan_channel_id    $1;
        nchan_channel_group test;
    }
}

and following php-script

<?php

header('X-Accel-Redirect: /sub/internal-ws/123');
header('X-Accel-Buffering: no');

exit;

When I'm trying to create websocket

 var ws = new WebSocket("ws://polygon/sub_upstream")

I'm always getting WebSocket connection to 'ws://polygon/sub_upstream' failed: Error during WebSocket handshake: Unexpected response code: 403

In access.log

127.0.0.1 - - [18/Oct/2016:00:31:56 +1000] "GET /index.php HTTP/1.0" 403 0 "-" ...
127.0.0.1 - - [18/Oct/2016:00:31:56 +1000] "GET /sub_upstream HTTP/1.1" 403 0 "-" ...

Am I doing smth wrong?

[slact]

It looks like your php script is returning a 403 Forbidden code. I double-checked X-Accel-Redirect functionality with websocket and it seems to be working fine. Please make sure your php script returns a 200 OK response.

[rs-orlov]

It turns out, that by default Nginx doesn't pass Connection and Upgrade headers required for moving to WS protocol through proxy_pass. So, after accel-redirect server doesn't know that client is waiting for moving to WS.

Following lines in config fixed the problem:

###
    location = /sub_upstream {
        proxy_pass http://polygon/index.php;
        proxy_set_header Connection $http_connection;    # <<< this one
        proxy_set_header Upgrade $http_upgrade;          # <<< and this one
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
###

You may want to update your example ;) https://nchan.slact.net/details#authenticate-and-hide-the-channel-id-with-x-accel-redirect

[rs-orlov]

Furthermore, you just can open connection directly(to http://polygon/index.php in my example). I think there really is no need in this one extra redirect through proxy_pass.

[slact]

Strange, I didn't need to pass those headers. I'll make a note of it in the documentation, thanks.

[trgtrg]

@rs-orlov Thanks for your solution, I've run into the same problem!

[slact]

@trgtrg : What version of Nginx are you using?

[trgtrg]

@slact I've installed it for ubuntu according to documentation with nginx-common.ubuntu.deb and nginx-extras.ubuntu.deb. Here is my nginx -V result:

nginx version: nginx/1.10.1
built with OpenSSL 1.0.1f 6 Jan 2014
TLS SNI support enabled
configure arguments: --with-cc-opt='-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --modules-path=/usr/lib/nginx/modules --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_v2_module --with-http_dav_module --with-threads --with-http_addition_module --with-http_flv_module --with-http_geoip_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_mp4_module --with-http_random_index_module --with-http_secure_link_module --with-http_sub_module --with-http_xslt_module --with-mail --with-mail_ssl_module --with-stream --with-stream_ssl_module --add-module=/build/nginx/debian/modules/headers-more-nginx-module --add-module=/build/nginx/debian/modules/nginx-auth-pam --add-module=/build/nginx/debian/modules/nginx-cache-purge --add-module=/build/nginx/debian/modules/nginx-dav-ext-module --add-module=/build/nginx/debian/modules/nginx-development-kit --add-module=/build/nginx/debian/modules/nginx-echo --add-module=/build/nginx/debian/modules/ngx-fancyindex --add-module=/build/nginx/debian/modules/nchan --add-module=/build/nginx/debian/modules/nginx-lua --add-module=/build/nginx/debian/modules/nginx-upload-progress --add-module=/build/nginx/debian/modules/nginx-upstream-fair --add-module=/build/nginx/debian/modules/ngx_http_substitutions_filter_module

[trgtrg]

The behaviour without those headers is really strange. The first time I reload nginx configuration client connects to websocket and stays in status pending for a long time. Then I post any message to that channel and it responds in normal way:

{"messages": 1, "requested": 27, "subscribers": 1, "last_message_id": "1488393231:0" }

But at that moment client disconnects with the message

Error during WebSocket handshake: Unexpected response code: 504

and all further connection attempts finish with

Error during WebSocket handshake: Unexpected response code: 200

until I do nginx reload

[slact]

Quite weird. Reopening this issue, will investigate.

[slact]

@trgtrg , could you post your nginx config? Edit out whatever you need to keep private.

[trgtrg]

Nginx config:

    server {
        server_name _;
        listen 80 default_server;

        root /local/www/www/;
        index index.php;

        location / {
            try_files $uri $uri/ /index.php?$args;
        }

        location ~ \.php$ {
            try_files $uri =404;
            include fastcgi_params;
            fastcgi_pass unix:/run/php/php7.0-fpm.sock;
            fastcgi_split_path_info ^(.+\.php)(.*)$;
            fastcgi_send_timeout 600s;
            fastcgi_param  SCRIPT_FILENAME $document_root$fastcgi_script_name;
        }

        location = /sub {
            proxy_pass http://127.0.0.1/sub/auth;

            # Without these 2 headers websocket won't work
            proxy_set_header Connection $http_connection;
            proxy_set_header Upgrade $http_upgrade;

            proxy_set_header Host $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Real-IP $remote_addr;
        }

        location ~ /sub/(\d+) {
            internal;
            nchan_subscriber;
            nchan_channel_id $1;
        }
    }

    server {
        listen 127.0.0.1:8080;

        location ~ /pub/(\d+) {
            nchan_publisher;
            nchan_channel_id $1;
        }
    }

And php script looks like this:

<?php
header('X-Accel-Redirect: /sub/1');
header('X-Accel-Buffering: no');
header('HTTP/1.1 200 OK');
die;

[haristku]

@trgtrg @rs-orlov i think the proxy_pass on your configuration is making a loop, so the cycle is broken because nginx will immediately respond to X-Accel-Redirect header.

you have 2 options:

one:

location ~ \.php$ {
            try_files $uri =404;
            include fastcgi_params;
            fastcgi_pass unix:/run/php/php7.0-fpm.sock;
            fastcgi_split_path_info ^(.+\.php)(.*)$;
            fastcgi_send_timeout 600s;
            fastcgi_param  SCRIPT_FILENAME $document_root$fastcgi_script_name;

            add_header 'X-Accel-Redirect' "$upstream_http_x1_accel_redirect";
            add_header 'X-Accel-Buffering' "$upstream_http_x1_accel_buffering";

            fastcgi_hide_header 'X1-Accel-Redirect';
            fastcgi_hide_header 'X1-Accel-Buffering';            
        }

and change the php script to this:

<?php
header('X1-Accel-Redirect: /sub/1'); #rename X-Accel-Buffering to X1-Accel-Buffering to prevent nginx from responding this header
header('X1-Accel-Buffering: no'); #rename X-Accel-Buffering to X1-Accel-Buffering to prevent nginx from responding this header
header('HTTP/1.1 200 OK');
die;

. . --
or two: use rewrite instead of proxy_pass to prevent loop cycle

location = /sub {
    rewrite ^/sub$ /index.php?$args last;

    #or use this
    #rewrite ^/sub$ /index.php?whatever=%2Fsub%2Fauth&$args last;

    #or use this
    #rewrite ^/sub$ /sub_auth.php?$args last;
}

with php script:

<?php
header('X-Accel-Redirect: /sub/1');
header('X-Accel-Buffering: no');
header('HTTP/1.1 200 OK');
die;

i hope i'm not misunderstood the problem.

ty, hrs.