Closed shi-yuan closed 7 years ago
Is it possible to use ProxyPass
directive with several nodes? I ended up using RewriteRule
for https://github.com/socketio/socket.io/pull/2819:
Header add Set-Cookie "SERVERID=sticky.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED
<Proxy "balancer://nodes_polling">
BalancerMember "http://server-john:3000" route=john
BalancerMember "http://server-paul:3000" route=paul
BalancerMember "http://server-george:3000" route=george
BalancerMember "http://server-ringo:3000" route=ringo
ProxySet stickysession=SERVERID
</Proxy>
<Proxy "balancer://nodes_ws">
BalancerMember "ws://server-john:3000" route=john
BalancerMember "ws://server-paul:3000" route=paul
BalancerMember "ws://server-george:3000" route=george
BalancerMember "ws://server-ringo:3000" route=ringo
ProxySet stickysession=SERVERID
</Proxy>
RewriteEngine On
RewriteCond %{HTTP:Upgrade} =websocket [NC]
RewriteRule /(.*) balancer://nodes_ws/$1 [P,L]
RewriteCond %{HTTP:Upgrade} !=websocket [NC]
RewriteRule /(.*) balancer://nodes_polling/$1 [P,L]
@darrachequesne It worked just fine! thanks
Thanks!
For anyone that is still having a problem with AWS Application Load Balancer try updating your Security Policy, I ran into the same problem even though it was working perfectly fine in the exact setup for another website but noticed the security policy was about 1 year behind. Updating it to a newer one seems to have solved the problem.
Having configured nginx and the client i was still struggling to find the exact solution. But @visibleajay made that comment and i realised that i didn't use the { 'transports': ['websocket'] } part into io.connect command.
So for anyone who might be into the same trouble,
var socket = io.connect('https://server.com/socket.io-path-here', { 'transports': ['websocket'] });
did the job, together with the correct nginx configurations.
@fott1 please be aware that using { 'transports': ['websocket'] }
means there's no fallback to long-polling when the websocket connection cannot be established.
Complete example with nginx: https://github.com/socketio/socket.io/tree/master/examples/cluster-nginx
@darrachequesne i believe you are right, what if i include in the array together with 'websocket' the 'polling', or 'xhr-polling'? Thank you for the provided example.
@fott1 the default is indeed ['polling', 'websocket']
(ref).
But you'll need to use the correct nginx configuration, since polling
(unlike websocket
) transport requires that every request is routed to the same socket.io server.
@darrachequesne in other words. if i have multiple server instances every request should be routed to the same instance? Correct me if i am wrong. Thank you for the tip.
@fott1 yep, that's right. The explanation is here: https://socket.io/docs/using-multiple-nodes/
FWIW - I was getting this on my local (no nginx, no proxies). Turns out with socket.io you have to explicitly specify the transports as first websocket, then polling - in both client and server. No idea why that wouldn't just be the default. Or maybe I'm doing it wrong.
Our original problem was we had polling before websocket, not realizing that order matters.
Original setup:
server:
io.set('transports', ['polling', 'websocket']);
client:
var socket = io.connect(server, { reconnect: true });
We realized our client was polling, so we removed that as an option. Then everything started failing and we got the 400 bad request.
So finally we figured out that you have to specify the order in both client and server, and if you don't the client will just go straight to polling, which makes little sense to me. You'd think it would default to websocket first.
Fix:
server:
io.set('transports', ['websocket', 'polling']);
client:
var socket = io.connect(server, { reconnect: true, transports: ['websocket', 'polling'] });
Again, please be aware that using { 'transports': ['websocket', 'polling'] }
means there's no fallback to long-polling when the websocket connection cannot be established.
Let's close that issue, please reopen if needed.
I took the approach of "extending" Elastic Beanstalk's default nginx configuration with the location settings similar to some earlier suggestions. Create a directory structure like so:
~/workspace/my-app/ |-- .ebextensions | `-- nginx | `-- conf.d | `-- myconf.conf `-- web.jar
where myconf.conf
, the name of which is arbitrary so long as it ends in .conf
contains the following:
server { location / { proxy_pass http://127.0.0.1:5000; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; proxy_set_header Host $host; } }
Be sure to adjust the port number to your needs.
I solved this problem by adding only { transports: ['polling'] } .
_
var socket = io.connect(server, {transports: ['polling']});
_
@hyewon330 By the looks if it you didn't really solve it š You simply configured socket.io to not even try using websockets in the first place. Sure, the error message is gone, but now you're forcing socket.io to use polling even if the connection between client and server would support websockets. Not sure whether that's critical for your application, just wanted to make sure you know š
For anyone using Nginx
, @tylercb solution works perfectly.
This solution fixed my issue with shiny apps. prefect.
@tylercb @rudolfschmidt @juanjoLenero @rwillett @cpres hello, I use your method but it does not work for me. I doubt if because I use other port rather than 8080. For example, my app is put in machine A with IP 170.8.8.8 monitoring port 5000, and I put nginx in machine B with IP 170.8.8.2 also monitoring port 5000. So I want to visit IP:5000 in B which skip to IP:5000 in A. The below is my nginx config in machine B:
upstream cuitccol.com{ #the name of server cluster
server 170.8.8.8:5000 max_fails=5 fail_timeout=50s; #for the first web server
}
server {
listen 5000;
server_name localhost;
#charset koi8-r;
#access_log logs/host.access.log main;
location / {
proxy_pass http://cuitccol.com;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
}
I do not know where goes wrong. Can you give some advises? thank you very much~ looking forward to your reply
I'm facing this issue for a while now in every env, including local. @darrachequesne Let me know if I need to provide more info:
I have a node js with express and the code below, which follows exactly the socket.io "How to use" section:
var app = require('express')();
var server = require('http').createServer(app);
var io = require('socket.io')(server);
io.on('connection', function(){ /* ā¦ */ });
server.listen(3000);
And I have set up a very simple client, with just the code below, which follows exactly the socket.io-client "How to use" section:
<script src="/socket.io/socket.io.js"></script>
<script>
var socket = io('http://localhost:3000');
socket.on('connect', function(){});
socket.on('event', function(data){});
socket.on('disconnect', function(){});
</script>
Despite successfully connecting, I keep getting this same error:
WebSocket connection to 'ws://localhost:3000/socket.io/?EIO=3&transport=websocket&sid=EWl2jAgb5dOGXwScAAAB' failed: Error during WebSocket handshake: Unexpected response code: 400
When looking at the browser console, the error is pointing to the line 112 from websocket.js, which states:
try {
this.ws = this.usingBrowserWebSocket ? (protocols ? new WebSocket(uri, protocols) : new WebSocket(uri)) : new WebSocket(uri, protocols, opts);
} catch (err) {
return this.emit('error', err);
}
Appreciate any ideas...
@rafapetter Check that you haven't required socket.io twice, I was moving setting up socket.io in www/bin to app.js and accidentally left a require socket.io in the bin which was causing this error.
Just adding {transports: ['websocket']}
option to the Socket.io client.
look like this.
import io from 'socket.io-client'
const socket = io('http://localhost:5000', {transports: ['websocket']})
@spookyUnknownUser on the server the socket.io is required only once.
@prapansak the client is a simple javascript file, inside a window.onload function, no ES6 import allowed. But I do follow the How to Use section:
<script src="/socket.io/socket.io.js"></script>
And when calling the socket as you've suggested:
const socket = io('http://localhost:5000', {transports: ['websocket']})
I get this error: WebSocket connection to 'ws://localhost:1337/socket.io/?EIO=3&transport=websocket' failed: Invalid frame header
Thanks guys, if you have any other idea let me know and I'll try it here.
Ok, finally got it solved!
@spookyUnknownUser your idea encourage me to look further for duplications. And as it turns out, on the server right after server.listen I was doing this:
io.attach(server, {
pingInterval: 40000,
pingTimeout: 25000,
});
Which in a way I guess is attaching the server a second time. So I removed it and right at the beginning, when requiring socket.io I've changed to:
var io = require('socket.io')(server, {
'pingInterval': 40000,
'pingTimeout': 25000
});
Now, no issues are shown, and everything works fine.
Thanks again for the insights guys
I have my socket server on EBS(AWS Elastic beanstalk). I faced similiar issue in the production environment, but could fix the issue without migrating to application load balancer and without changing ngnix or apache configurations.
Changes I made to solve this issue: Instead of https, i allowed ssl on 443 to my application port and opened port 443 on my security group
For those on AWS: Use the Application Load Balancer and make sure you have stickiness enabled on the Target Group.
Turns out this was happening intermittently on our server when it was overloaded and within peak usage time.
Allowing the default settings while io.connect
fires off creates XHR requests and a WS request shortly after (as mentioned above). All these XHR requests were overloading the server and thus returning the 400 errors. By using the suggestions above of adding transports: ['websocket']}
to the client code the issue has been resolved. Will post an update if the 400 errors persist but it seems solid for now.
have the same issue, neither suggested above server side changes resolved it. On the client side I have socket = io.connect();
in react module and it's able to figure out what server I'm connecting to.
Providing server
parameter to connect is kind of a hassle but without it how can I specify { reconnect: true, transports: ['websocket', 'polling'] }
? Sounds like an optional parameter is #1 but the crucial one is #2
i added this below, and it worked for me. location / { proxy_pass http://localhost:8080; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; proxy_set_header Host $host; } https://chrislea.com/2013/02/23/proxying-websockets-with-nginx/
I unfortunately have the same problem, even with the good headers set .. I created a stackoverflow with my config..
Make note of the double quotes here:
proxy_set_header Connection "upgrade";
I received this message using single quotes but using doubles quotes resolved it.
After about an hour of troubleshooting, it seems like there might be some additional configuration needed if you're running your endpoint as a cluster.
tylercb solution worked for me (NgInx). Thanks!
The key for us was the proxy_http_version 1.1;
in @tylercb's solution
I have edited NginX config on my server according those many comments.
But it works partially. At first, after server restart, it works bad. Mostly with hard-refresh. Later it works better, just sometimes need hard refresh.
Locally with sails lift
it works
But if i am using locally sails lift --prod
then i get the same thing, that socket cannot connect and 400 bad response. But after some tries it is again stable.
This seems still not solved - no final solution found why we have this..
I added the various configurations in the documentation: https://socket.io/docs/using-multiple-nodes/
Any suggestion for improvement is welcome! => https://github.com/socketio/socket.io-website
I just face this problem and still got an error even disable my nginx. Finally the problem because of express-status-monitor
middleware on express, this makes HTTP call on the first (request) handshake of WebSocket goes failed.
I try to disable that middleware and the WebSocket working well
There are several reasons why you would get 400 during the handshake. Following are few things that could be tried
location /{
proxy_pass http://127.0.0.1:5000/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
}
Maybe it will be help to some one. I got this error when created two instances of socket.io in my app. I called io = socketio(server)
twise and got error 400. That is some sort of stupid errors but... that was mine.
Make sure you enable Sticky sessions in your application load balancer (don't use classic load balancer). If you don't use sticky sessions, you'll probably get the 400 errors randomly.
This is because when the upgrade from polling to websocket is attempted, itās possible that the load balancer balances the upgrade request to a server that never encountered this socket id and throws the 400 error. More details here: https://socket.io/docs/using-multiple-nodes/
Also, if you're using elastic beanstalk, add the following websocket.config file in the .ebextensions folder to support Nginx upgrading to Websockets:
container_commands:
enable_websockets:
command: |
sed -i '/\s*proxy_set_header\s*Connection/c \
proxy_set_header Upgrade $http_upgrade;\
proxy_set_header Connection "upgrade";\
' /tmp/deployment/config/#etc#nginx#conf.d#00_elastic_beanstalk_proxy.conf
I googled because I got the same problem and I also use nginx. The solution is to add this part
proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; proxy_set_header Host $host;
into the nginx configuration file like tylercb mentioned.
So at least this works when you use nginx as reverse proxy, here is the explanation
WebSocket proxying
To turn a connection between a client and server from HTTP/1.1 into WebSocket, the protocol switch mechanism available in HTTP/1.1 is used.
There is one subtlety however: since the āUpgradeā is a hop-by-hop header, it is not passed from a client to proxied server. With forward proxying, clients may use the CONNECT method to circumvent this issue. This does not work with reverse proxying however, since clients are not aware of any proxy servers, and special processing on a proxy server is required.
Since version 1.3.13, nginx implements special mode of operation that allows setting up a tunnel between a client and proxied server if the proxied server returned a response with the code 101 (Switching Protocols), and the client asked for a protocol switch via the āUpgradeā header in a request.
As noted above, hop-by-hop headers including āUpgradeā and āConnectionā are not passed from a client to proxied server, therefore in order for the proxied server to know about the clientās intention to switch a protocol to WebSocket, these headers have to be passed explicitly:
location /chat/ {
proxy_pass http://backend;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
A more sophisticated example in which a value of the āConnectionā header field in a request to the proxied server depends on the presence of the āUpgradeā field in the client request header:
http {
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
server {
...
location /chat/ {
proxy_pass http://backend;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
}
By default, the connection will be closed if the proxied server does not transmit any data within 60 seconds. This timeout can be increased with the proxy_read_timeout directive. Alternatively, the proxied server can be configured to periodically send WebSocket ping frames to reset the timeout and check if the connection is still alive.
Any news on this issue? I referenced it here https://stackoverflow.com/questions/49575350/websocket-connection-to-wss-error-during-websocket-handshake-unexpected-re
Many users have this problem because of the nginx configuration and while on dedicated servers you can fix it using the ideas above (nginx configuration, etc) many users don't have access to these settings, so it would be great to have some kind of fix from sockets.io on this...
Nobody?
@deemeetree If you use multiple nodes which seems to me possible due the error message on your site "Session ID unknown", you should take a look to my answer on StackOverflow (https://stackoverflow.com/a/53163917/6271092).
To be short, socket.io as they say on their site (https://socket.io/docs/using-multiple-nodes/),
If you plan to distribute the load of connections among different processes or machines, you have to make sure that requests associated with a particular session id connect to the process that originated them.
This is due to certain transports like XHR Polling or JSONP Polling relying on firing several requests during the lifetime of the āsocketā. Failing to enable sticky balancing will result in the dreaded:
Error during WebSocket handshake: Unexpected response code: 400
So you cannot have multiple sockets that work on different nodes. Reconfigure your server as it says. And for your Express, configure your port like this,
parseInt(your_port) + parseInt(process.env.NODE_APP_INSTANCE);
Hope it helps.
Got same error directly connecting to my app hosted on Windows Server 2008R2 (no proxy, no IIS). Only dumb intermediate hardware in between.
I am facing same issue on Window server 2016 please mention your fix. Thanks
I have been facing this issue for quite some time. If anyone has any information please help. I am using apache server.
Any clue? Works great on dev but same issue on SSL.
Using PM2 Node Cluster mode: 4 x 1
Using apache proxy, still no clue
For those who are on httpd2/apache2, I have a solution that appears to work:
ProxyRequests Off
<Location />
ProxyPass http://127.0.0.1:1337/
ProxyPassReverse http://127.0.0.1:1337/
RewriteEngine On
RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC]
RewriteCond %{HTTP:CONNECTION} Upgrade$ [NC]
RewriteRule /socket.io/(.*) ws://127.0.0.1:1337/socket.io/$1 [P]
</Location>
Hey @ZeldaZach is that for the .htaccess file? thanks!
Can't find out a solution, I get this error on the browser console: WebSocket connection to 'ws://.../socket.io/?EIO=2&transport=websocket&sid=p3af7ZNfvogtq6tAAAG0' failed: Error during WebSocket handshake: Unexpected response code: 400.
Hava any advice ?