Proxying websockets to multiple backends doesn't work

[bduffany]

Is this a bug report?

Yes

Steps to reproduce

1. Set up a proxy server with

app.use(
  '/sockjs-node',
  createProxyMiddleware('/sockjs-node', {
    target: 'http://127.0.0.1:3000',
    ws: true,
  })
);
app.use(
  '/socket.io',
  createProxyMiddleware('/socket.io', {
    target: 'http://127.0.0.1:4000',
    ws: true,
  })
);

The /sockjs-node server can be quickly spun up with create-react-app since that is the socket for Webpack hot reload. /socket.io can be quickly spun up with a socket.io example server.

2. Load up the React app in your browser. The /sockjs-node requests are not proxied correctly. The HPM log output shows that it is trying to proxy the /sockjs-node request to port 4000, resulting in an ECONNREFUSED error.

Setup

• http-proxy-middleware: 1.0.5
• http-proxy-middleware configuration: see above
• server: express 4.17.1

client info

Chrome (latest) / Ubuntu 20.04 (Linux)

target server info

One is a webpack dev server, one is a simple express server with socket.io config

Reproducible Demo

https://github.com/bduffany/hpm-bug

[bduffany]

I found that if I change

app.use(
  '/sockjs-node',
  createProxyMiddleware('/sockjs-node', {
    target: 'http://127.0.0.1:3000',
    ws: true,
  })
);

to

app.use(
  createProxyMiddleware('/sockjs-node', {
    target: 'http://127.0.0.1:3000',
    ws: true,
  })
);

Then the issue is fixed. No ideas yet as to why. However, I think this fix results in worse performance, because express is now going to be running that proxy middleware on every request, even ones that don't match /sockjs-node. It might be OK though if you're only using this for local development.

[bduffany]

FYI added a reproducible demo repro here: https://github.com/bduffany/hpm-bug

One liner you can throw in your terminal to get it up and running, ready to debug in Chrome:

git clone https://github.com/bduffany/hpm-bug && cd hpm-bug && ((sleep 8 && google-chrome --auto-open-devtools-for-tabs http://localhost:8080) &) ; ./run.sh

[gregbarcza]

+1

[hbi99]

+1

[Cheyenne55]

You just saved my day with your fix. Spent 3 hours because I had this exact problem.

When proxying to several websocket servers Chrome was showing errors like "Invalid frame header" or "WebSocket connection failed: One or more reserved bits are on: reserved1 = 0, reserved2 = 1, reserved3 = 1" which googling them up led to absolute non-sense.

I think this issue should really be patched up.

[bduffany]

@Cheyenne55 I experienced the same problem with "One or more reserved bits are on," and Googling led to nothing helpful. I only discovered the root cause of the problem because I noticed in the log output that it was trying to proxy requests to the wrong port.

[stefanfuchs]

In my case, I was getting an error code like this: ERR_STREAM_WRITE_AFTER_END This was crashing the proxy server completely.

Separating the proxy server in 2 instances, each with only one websocket endpoint, solved the problem. it's a different kind of error than the previous posts, but I believe it might be related.

[gilsdav]

Same issue but it's because it listen a global "upgrade" event for all ws subscriptions. https://github.com/chimurai/http-proxy-middleware/blob/95233df91588f3f0bd5c21961ae6405acd3e647b/src/http-proxy-middleware.ts#L66 I had to handle it myself to choose when to call the upgrade function of the proxy that I need.

[randomevents]

I had the same issue in a way, but what was happening was that the proxy was constantly trying to upgrade the host server's websockets once a connection had been made with a proxy that had websockets.

What I did was follow the excellent bread crumbs that @gilsdav left and was able to put a solution together. First, the problem is that when the proxy tries to see if should proxy a websocket, it's testing the upgrade path against '/'. Which means it's going to try and catch everything. What I did in my proxy module was listen on the connection and only let the proxy upgrade paths that are assigned to it (which was expected behaviour). With the below I have multiple proxies with websockets and socket.io running on my host (with multiple name spaces), but the latter is handled by that package.

` moduleJS.add = function (objProxy) { if (objProxy && objProxy.length) { if (moduleJS.proxyLoaded === false) { npm.proxy = require('http-proxy-middleware'); moduleJS.proxyLoaded = true; }

    for (let x of objProxy)
        if (x.options.target !== global.serverUrl) {
            let instance = npm.proxy.createProxyMiddleware(x.options);
            if (x.secure && x.secure.enabled)
                npm.expressApp.use(x.path, app.Auth.cookieVerify(x.secure.permissions), instance);
            else
                npm.expressApp.use(x.path, instance);

            moduleJS.paths.push({ path: x.path, instance: instance, ws: x.wsManual ? true : false });
        }
}

}

global.serverConnection.listen.on('upgrade', function upgrade(request, socket, head) { let pathname = npm.url.parse(request.url).pathname; for (let x of moduleJS.paths) if (pathname.startsWith(${x.path}/) && x.instance && x.ws) { x.instance.upgrade(request, socket, head); console.log(Proxying WebSockets for: ${x.path}) } }); `

[cm226]

I was having the same problem, with a slightly different usage.

If you try to have 2 websocket backends, one which requires subscribing to the upgrade event and one that doesn't, when you send HTTP traffic to the first websocket, you get "Invalid frame header" on the second. My workaround for now is to use separate proxies for the web socket endpoints and the http traffic endpoints.

Looking at some Wireshark traffic it looks like http-proxy-middleware was forwarding the upgrade request to both backends.

Minimal repro code: https://github.com/cm226/http-proxy-middleware-multi-websocket

[dbertolotto]

I am having the same issue(s) reported here - webpack-dev-server and thus webpack/angular are using this library (see related bug). Is there a plan to fix this issue anytime in the future?

[wll8]

Multiple websoket proxies cause self or server errors

const express = require(`express`)
const { createProxyMiddleware } = require(`http-proxy-middleware`)
const p1 = createProxyMiddleware({
  target: `http://127.0.0.1:9000/`,
  changeOrigin: true,
  ws: true,
  logger: console,
  pathRewrite: {
    "^/a": `http://127.0.0.1:9000/asr`, // new WebSocket(`ws://127.0.0.1:3000/a/socket/x`)
  },
})
const p2 = createProxyMiddleware({
  target: `http://127.0.0.1:9000/`,
  changeOrigin: true,
  ws: true,
  logger: console,
  pathRewrite: {
    "^/b": `http://127.0.0.1:9000/im`, // new WebSocket(`ws://127.0.0.1:3000/b/frontend/x`)
  },
})

const app = express()
app.use(`/a`, p1)
app.use(`/b`, p2)

const server = app.listen(3000)
server.on(`upgrade`, p1.upgrade)
server.on(`upgrade`, p2.upgrade)

test

new WebSocket(`ws://127.0.0.1:9000/asr/socket/x`) // raw ok
new WebSocket(`ws://127.0.0.1:3000/a/socket/x`) // err
new WebSocket(`ws://127.0.0.1:9000/asr/socket/x`) // raw err

env

• node v14.15.5
• http-proxy-middleware 2.0.6

[innoavator]

Is there any solution to this issue?

[marcinmajkowski]

To understand the issue, it is worth to take look at the implementation.

Then one realizes that doing:

const p1 = createProxyMiddleware('http://127.0.0.1:9000/', { ws: true });
const p2 = createProxyMiddleware('http://127.0.0.1:9000/', { ws: true });

const app = express();
app.use(`/a`, p1);
app.use(`/b`, p2);

has same effect as doing:

const p1 = createProxyMiddleware('http://127.0.0.1:9000/', { ws: false });
const p2 = createProxyMiddleware('http://127.0.0.1:9000/', { ws: false });

const app = express();
app.use(`/a`, p1);
app.use(`/b`, p2);

server.on('upgrade', p1.upgrade);
server.on('upgrade', p2.upgrade);

Which makes it clear that in case of WebSockets, express routing is not respected.

Knowing that, in my case, I was able to workaround this by setting ws: false and by handling 'upgrade' routing to correct proxy middleware in my application.

I am not really sure what would be a clean solution to this issue as upgrade requests are not routed in express. WebSocket handling in this library seem a bit hacky already as (to achieve something which seems not supported in express) it has to access server through req:

https://github.com/chimurai/http-proxy-middleware/blob/master/src/http-proxy-middleware.ts#L61

Setting ws: false and doing this explicitly in application seem like a cleaner solution as it also makes it possible to unregister the listener in case proxy is used dynamically (e.g. there is an if selecting different proxy instance on some condition at runtime).

[hordesalik]

I have the same issue and the @marcinmajkowski 's comment above really helped me to solve the problem. The difference was only that I have 2 different servers to be proxied so I have to route upgrade requests to proper proxy servers:

const p1 = createProxyMiddleware('http://127.0.0.1:8000/', { ws: false });
const p2 = createProxyMiddleware('http://127.0.0.1:9000/', { ws: false });

const app = express();
app.use('/a', p1);
app.use('/b', p2);

server.on('upgrade', function(req, socket, head) {
    if (req.url.indexOf('/a') === 0) {
        p1.upgrade(req, socket, head);
    }
});
server.on('upgrade', function(req, socket, head) {
    if (req.url.indexOf('/b') === 0) {
        p2.upgrade(req, socket, head);
    }
});