Support for custom headers for handshake

[Misiu]

Hi,

please consider adding ability to add custom headers for handshake. In RFC6455 there one interesting point:

The request MAY include any other header fields, for example, cookies [RFC6265] and/or authentication-related header fields such as the |Authorization| header field [RFC2616], which are processed according to documents that define them.

I've found an example how to add custom header to handshake: https://blog.heckel.xyz/2014/10/30/http-basic-auth-for-websocket-connections-with-undertow/ but this is for Java and unfortunately isn't possible in HTML5.

When searching over the net I found many places question about this option, for example: https://github.com/sta/websocket-sharp/pull/22 https://stackoverflow.com/questions/4361173/http-headers-in-websockets-client-api/4361358#4361358 https://github.com/aspnet/SignalR/issues/888

For example in Python this is possible https://stackoverflow.com/questions/15381414/sending-custom-headers-in-websocket-handshake. Other languages also support this. Last place missing is the browser.

Please consider adding this into specification. Having this even as a draft would allow us to consider browser vendors to add support for it.

If this is incorrect place for adding request about specification please forgive me and please point me to right place.

[domenic]

We'd need implementer interest in this before moving forward, similar to https://github.com/whatwg/html/issues/2177.

[Misiu]

Mozilla - https://support.mozilla.org/en-US/questions/1176810 Chromium - https://bugs.chromium.org/p/chromium/issues/detail?id=768328 EDGE - https://wpdev.uservoice.com/forums/257854-microsoft-edge-developer/suggestions/31600090-websocket-support-for-custom-headers-for-handsha

@domenic I'll add feature request for other browsers soon. Hopefully some of them will show some interest.

[tyoshino]

Thank you Misiu for filing the feature request and the bugs for each vendor.

Regarding complexity, for the web platform, this would require non-trivial amount of work at both spec and impl side.

To introduce custom header feature, we need to make it issue a CORS preflight. Though this is not an HTTP API, considering the original reason why we introduced the CORS preflight, we need it for WebSocket + custom headers.

On Chromium, we have separate stacks for WebSocket and XHR/fetch() (for which we have CORS logic). Some non-trivial refactoring and gluing is needed.

[ricea]

I am one of the authors and maintainers of Chrome's WebSocket implementation. I oppose this proposal.

Reasons:

• The WebSocket handshake security model hinges on exposing no more capabilities for request forgery than are already possible using an img tag. It is not possible to add arbitrary headers to the request using an img tag.
• Apart from the limited sub-protocol negotiation, the WebSocket handshake is not intended as a means to exchange metadata about the connection. Non-browser implementations may use it this way, but I would not recommend it. It is better to think of it as a low-level operation like a TCP/IP SYN packet.
• It would be possible to add this to the WebSocket protocol by using a CORS preflight. However, the utility provided simply doesn't justify the additional implementation, standardisation and maintenance cost.

[Misiu]

@tyoshino @ricea thank You guys for Your reply. My main idea was to add ability to add Authentication header to WebSockets.

If You look at SignalR repo You will notice that many people are looking for a way to pass authentication via WebSocket - https://github.com/aspnet/SignalR/issues/888#issuecomment-346242065

Right now the only option to do this is to add token as part of query string. To simplify this only Authorization header is required.

[ricea]

@Misiu In the specific case of WebSockets I think passing an authentication token in the URL is okay. The reason is that, unlike HTTP URLs, wss: URLs are never exposed to the user. They can't bookmark them or copy-and-paste them. This minimises the risk of accidental sharing. In addition, their appearance in other web APIs is minimal [1]. For example, they won't appear in history. This reduces the risk of leakage via JS APIs.

The best thing from a security perspective would probably be to perform authentication after the WebSocket is connected, but I realise it is undesirable from a resource-usage perspective to permit unauthorised connections to be established in the first place.

Given the portability difficulties with using cookies or http auth, a short-lived authentication token in the URL is probably your best option at the moment.

A specific note with respect to the Authorization header is that normally you'd need to be able to handle a 401 response to use it. But in the WebSocket API error responses are never exposed to the page for security reasons. So that wouldn't work very well.

[1] I actually can't think of any other APIs that expose ws: or wss: URLs at all. Certainly they don't appear in resource timing.

[digitalpacman]

I cannot use WebSocketSharp because it's missing the ability to customize authorization headers. I am trying to integrate with a WebSocket api that takes in OAuth2 bearer tokens.

[digitalpacman]

@ricea That's not the main reason why you don't put authentication values in URLs. It's because of server logs. Auth tokens should be treated with the same security concern as username and password combos. You don't put username and password combinations in urls, so neither should auth tokens. If you have a vulnerability and a user downloads your http logs, they then have access to virtually all of your accounts.

[digitalpacman]

@ricea sadly implementers don't get to make the decision of what is required to use an api. I am currently trying to use an api that REQUIRES oauth2 bearer tokens in the authorization header for server-to-server communication. They offer cookies for browser based security, but I do not have that luxury because I have no sessionids for the user. So all you're doing by saying "we should not do this" is saying "you have to look elsewhere to implement with this api". Which would simply just drive people away from using this implementation. You aren't "saving" anyone but not allowing this, just alienating users.

[maleta]

I encountered this issue looking for solution on my problem - how to remove sid parameter from URL that is set and used by websockets. Now I see @ricea suggestion of adding short living token to URL with explanation that "wss: URLs are never exposed to the user."

OWASP Zap report is suggesting sid param removal from URL, also this question on stackoverflow is refering that CloudFlare is also suggesting sid param removal from URL: https://stackoverflow.com/questions/42759556/how-to-remove-socket-io-sid-parameter-from-url

So my question is, @ricea, do you think that this reported issue is completely irrelevant because of fact that ws/wss urls are never exposed to users?

[ricea]

@maleta

So my question is, @ricea, do you think that this reported issue is completely irrelevant because of fact that ws/wss urls are never exposed to users?

Security measures need to be considered in terms of what threats they are intended to protect against and to what extent they are effective in mitigating those threats.

In the case of the Cloudflare vulnerability, placing the sid parameter in a header rather than in the URL would have provided no protection whatsoever.

Quoting https://bugs.chromium.org/p/project-zero/issues/detail?id=1139:

... we observed encryption keys, cookies, passwords, chunks of POST data and even HTTPS requests for other major cloudflare-hosted sites from other users.

This is the trouble with checklist security. If you removed the sid parameter from the URL, you could check that item off the list and feel that you'd improved your security. But in practice there's no improvement at all.

@digitalpacman If someone has access to your server logs, they can do much worse things than just steal credentials. There's usually all kinds of personally-identifiable information in there. Access needs to be tightly restricted just as with any other kind of user data.

[digitalpacman]

@ricea there is only personally-identifiable information in there if you aren't doing good practices. At most there should be an IP address, but you can opt to not record that either. Not putting SECURITY CREDENTIALS in the url is an extremely popular best practice. Infact, governing bodies for security will instantly fail you if you are passing them in the URL.

Please look at the OAuth2 spec for example and explanation of these vulnerabilities. There is a reason it REQUIRES them in the POST body.

[annevk]

@ricea is this perhaps something we can reconsider if https://wicg.github.io/origin-policy/ becomes a thing? That should nullify most of the CORS preflight cost.

The reason I'm somewhat sympathetic to the requests is because I learned that middleboxes basically ruin full duplex HTTP so WebSocket will likely stick around.

[ricea]

@annevk I think origin policy will probably be implemented in the browser in the same place as CORS, so it would still mean re-wiring the handshake to go down the same code path.

The WebSocket handshake started out as purely a mechanism to negotiate a WebSocket connection. Over time we've added HTTP features like cookies and authentication, but we've paid a high price in implementation complexity.

The "right" way to do WebSocket authentication is to do it at the application layer after the handshake completes. No-one asks how to put an oauth token in a TCP/IP SYN packet. But the reality is that we've ended up in a fuzzy middle-ground that is hard to explain.

[annevk]

@ricea given that we do add browser-supplied cookies and authentication, it's quite a reasonable request that we also enable headers, since much WebSocket server-side infrastructure probably depends on such information being in the handshake.

And at least as far as the specification is concerned the handshake shares many aspects with Fetch, to enable mixed content blocking, HSTS, CSP, etc. So from that perspective this is not that much of a stretch. (I wonder if Chrome's approach is shared by other browsers.)

[davidfowl]

@ricea

The WebSocket handshake security model hinges on exposing no more capabilities for request forgery than are already possible using an img tag. It is not possible to add arbitrary headers to the request using an img tag.

Can you elaborate here. I don't understand why the security model hinges on not allowing headers to be set in the client API? I'm not sure I understand this.

[annevk]

@davidfowl part of the same-origin policy is that we don't send "attacker-controlled" headers to cross-origin URLs. That's why if you want to do that you need to use CORS, which uses a CORS preflight to make an explicit check that the cross-origin URL is okay with the headers about to be transmitted.

The objection here, at least from the Chrome team, is that supporting a CORS preflight for WebSocket is too costly.

[davidfowl]

@annevk Thanks for that clarity!

So attacker controlled headers are seen as more dangerous than attacker controlled query string. Would it help if we restricted the types of headers that could be sent?

[annevk]

Yes, you can basically reach any arbitrary URL (whatever the query string or path), but you can only control headers to a very limited extent (and methods too, only HEAD/GET/POST).

The request headers we allow "attackers" to control are listed at https://fetch.spec.whatwg.org/#cors-safelisted-request-header, but note that we plan to lock that down some more in https://github.com/whatwg/fetch/pull/736. For the kinds of use cases that people seem to have I don't think allowing these would be sufficient (or it would lead to hacks where you put authorization data in Accept-Language or some such).

[davidfowl]

@annevk The current design forces people to send what would typically go in the the Authorization header in the query string. Is that any less secure than allowing say the authorization header to bet set on the upgrade request? (even cross origin)

[annevk]

The concern is not with the security of the application making the request, it's with the security of the remote server. The remote server will have to be robust against arbitrary URLs, but it does not have to be robust against arbitrary headers, since it'll assume (due to the long history of the web and early establishment of the same-origin policy) that those cannot come from browsers.

[davidfowl]

I see, understand the push back now. So there needs to be a pre-flight request integrated into the flow and that's expensive in chrome because of the current implementation split between websockets and xhr/fetch. Is that a fair summary?

[annevk]

Excellent summary.

[davidfowl]

Should we get some of the other browser implementors to chime in? I can probably get somebody from the edge team to chime in here about the difficulty there. I'm not sure how to go about getting the other browser vendors interested enough to look at this issue (safari, firefox, anything else?)

[annevk]

Maybe @mcmanus for Firefox and @youennf for Safari? Feedback from Edge would be good.

[mcmanus]

I'm kinda surprised people aren't smuggling this through subprotocol - seems obvious enough.

architecturally speaking, a CORS preflight wouldn't be a big burden for firefox to implement.

otoh, designs that result in a lot of preflights suck so I'm not convinced we want to enable this rather than pushing it into the post websocket-handshake data.

[benaadams]

rather than pushing it into the post websocket-handshake data.

Then its a per web app custom protocol implementation; open to a slow loris type, slow auth issue from an unknown user; depending on how the application developer implements it; vs a well understood header handling implementation by the webserver?

[digitalpacman]

@davidfowl Actually yes it is inherently less secure to send it via querystring. This is why the OAuth spec requires the credentials to be passed in headers or body, always. The reasoning is because log files of webservers and routers often log the URL including the querystring parameters for identifying requests. Therefore, without monitoring every system in the entire world yourself you rely on them properly stripping credentials from their logs to protect people. That entire problem disappears when you send sensitive credentials in the headers or body because those are rarely, if ever, logged. Mainly because of their size and the understanding that it is not directly secure. Some headers are logged by webservers, but those would be exclusively called out.

[davidfowl]

@digitalpacman I'm fully aware. I'm mostly trying to get the implementors to agree we should try to solve this problem.

otoh, designs that result in a lot of preflights suck so I'm not convinced we want to enable this rather than pushing it into the post websocket-handshake data.

The problem is that you've left the realm of http auth here and you now have to invent a mini protocol over the websocket itself. You also need to let the "attacker" successfully establish a connection to your server, and you now have to wait for data to be sent over it (which needs to timeout appropriately) to authenticate. So you end up with completely different authentication methods for websocket and non-websockets clients.

[digitalpacman]

@davidfowl I've given up the fight. I think we should fork, fix, and ignore this repository for life. I physically switched from c# to nodejs because of this. I don't get a choice in how a websocket server I'm integrating with does it's authing protocol. It could be custom, it could be headers, it could be url, it could be in a postflight check, or preflight check. A libraries job is not to dictate HOW integrations must occur, unless it's implementing a strict standard like OAuth2. The Websocket spec specifies header authing is available, so it should be available.

I physically cannot use this library because of the resistance of the owner. The library I am integrating with requires that a preflight OAuth2 http request occurs to receive a token that is passed within a Bearer authorization header. I get no choice in the matter.

[davidfowl]

@digitalpacman I'm not sure what you mean by switching from c# to nodejs. The issue here is about the browser. Non-browser clients support setting headers just fine.

[digitalpacman]

@davidfowl Whoops. I thought this was a different discussion but it still applies. There is a C# websocket library that refuses to allow custom headers. They only support Basic, because that's what "browsers do". I thought this was that forum.

If you scroll up, I posted about WebSocketSharp, the offending library. Because @ricea apposes this standard change. He's refusing to accept any pull requests that add in any kind of security methods other than the singular method he added.

[webmutation]

I live for the simple things in life. Like when a spec allows for something and the implementation does not comply even though it already has a bunch of added things like cookies and basic auth :) All browsers should remove web sockets from their list of supported specs if they are not spec compliant.

There is one HEADER that should be allowed that is the only use case that is being talked about, that almost all other websocket implementation support Authorization so that people do not have to reinvent in particular if you are using an API gateway, you can't do OAuth because of this issue. So instead of browser vendors aligning with the spec, everyone has to invent their own mini auth service for JavaScript the special needs kid of websockets

[lamba92]

Has there been any news on the topic?

[fabrizziocht]

Is the problem solved?

[Misiu]

@davidfowl Edge is using Chromium, so maybe now this could be implemented? I fully support what You wrote here: whatwg/websockets#16. The whole idea is to disallow establishing connections for users without auth header and that didn't successfully authenticate.

As I wrote in my initial message, RFC 6455 clearly says that request MAY include any other headers, but hey, that's just a standard, we don't have to follow it....or should we 🤔🤔🤔

[benaadams]

So a websocket request is actually two requests; an http(s) upgrade request and then the ws(s) request.

CORS aside could the headers be passed for the upgrade part if it matches? e.g.

https GET example.com/index.html
-> wss://example.com/websocket
https UPGRADE example.com/websocket (include https example.com headers)
(wss) example.com/websocket

i.e. wss might be a different scheme so different origin; but it makes a https upgrade request first which is the same scheme, and that's when we want the headers

[mjkahlke]

I put together a POC using javascript in a Chrome browser to pass in an array of two strings in the Sec-WebSocket-Protocol header; on the server side I have a javax.websocket.server.ServerEndpoint with a custom configurator which overrides javax.websocket.ServerEndpointConfig.Configurator.getNegotiatedSubprotocol() to treat the first string as the protocol and the second string as the authorization token.

If the protocol and token don't match I return "" which results in an HTTP upgrade response minus the Sec-WebSocket-Protocol header indicating the server does not agree to any of the protocols the client provided in its request. The websocket connection fails in that case.

I realize this is a hack and requires custom authorization code in the server, but it occurs during the websocket handshake, not afterwards.

I don't actually plan to make use of this as I'm working with a non-browser client application that is able to set either a cookie or an Authorization: Bearer xxx header to suit my purposes, but I would be interested in your thoughts and opinions.

[yeryomenkom]

Any progress on this?

[krisleland]

Hey, team. Any chance of looking at this again?

[davidfowl]

@kalidasya what is that?

[kalidasya]

oops sorry I was subscribed on a similar issue on python websockets :D did not double check repo before comment

[jonenst]

I would have liked to be able to use the same auth for my regular requests and for the websocket handshakes (http header "Authorization: bearer XX"). I'm using the same origin in all requests.

Does the cors preflight check need to happen if websocket handshake is done at the same origin ? If not, maybe as a first step allow the authorization header only in this case ?

[jasonliu0704]

This would be really helpful for us.

[JonathanHuot]

@ricea So to summarize, and if we look into the W3 doc https://www.w3.org/TR/websockets it explains

(..) The headers to send appropriate cookies must be a Cookie header whose value is the cookie-string computed from the user's cookie store and the URL url; for these purposes this is not a "non-HTTP" API.

So here, it conceals adding Cookies, and treating this similarly to HTTP. Then later, doc explains stopping all HTTP behaviors of the response for security reasons:

When the user agent validates the server's response during the "establish a WebSocket connection" algorithm, if the status code received from the server is not 101 (e.g. it is a redirect), the user agent must fail the WebSocket connection.

Which is against the RFC, but makes sense in term of security reasons. Note also HTTP Response 101 are also updating Cookies by accepting Set-Cookie with the following:

When the WebSocket connection is established, the user agent must queue a task to run these steps: (..)

4. Act as if the user agent had received a set-cookie-string consisting of the cookies set during the server's opening handshake, for the URL url given to the WebSocket() constructor. [COOKIES] [RFC3629] [WSP]

Basically, Cookies are good but not Authorization header. Most modern applications are moving to a Token Based Authorization (i.e. OAuth2.0), if the WebSocket API could accept the Authorization header, it will allow that to coexist.

In addition, we're not talking only about WebServer hosting WebSocket applications, but also a way for Reverse Proxy applications to add authentication check before being forwarded to the servers (unauthenticated servers trust incoming requests because behind Reverse Proxy).

Also, it seems nobody care about handling 401 and such responses because anyway the browser will treat it as a failure of connection. So no additional code maintenance is required.

[ricea]

@ricea So to summarize, and if we look into the W3 doc https://www.w3.org/TR/websockets it explains

That doc is obsolete. The current controlling standards are https://html.spec.whatwg.org/multipage/web-sockets.html and https://fetch.spec.whatwg.org/#websocket-protocol.

Basically, Cookies are good but not Authorization header. Most modern applications are moving to a Token Based Authorization (i.e. OAuth2.0), if the WebSocket API could accept the Authorization header, it will allow that to coexist.

In the living standard, 401 responses are correctly handled by sending stored credentials.

In addition, we're not talking only about WebServer hosting WebSocket applications, but also a way for Reverse Proxy applications to add authentication check before being forwarded to the servers (unauthenticated servers trust incoming requests because behind Reverse Proxy).

This requires the Reverse Proxy to understand the WebSocket protocol, in which case it can equally well perform an authentication handshake after the upgrade is complete.

Also, it seems nobody care about handling 401 and such responses because anyway the browser will treat it as a failure of connection. So no additional code maintenance is required.

Digest authentication (https://tools.ietf.org/html/rfc7616) and other schemes that use challenges require correct handling of 401 responses. Without visibility of 401 responses, the facility to send Authorization headers would be extremely limited in practice.

I maintain that the correct way to do custom authorization for WebSockets is after the handshake upgrade is complete.

[benaadams]

I maintain that the correct way to do custom authorization for WebSockets is after the handshake upgrade is complete.

You might want to reject the upgrade if not authorised (returning 401 Unauthorized, 403 Forbidden, 404 Not Found); rather than accepting the upgrade and then waiting for an indeterminate amount of time for a custom authorisation to be sent to then disconnect.

[davidfowl]

Right, doing it after the connection also requires a custom non-HTTP based authentication.

[ricea]

You might want to reject the upgrade if not authorised (returning 401 Unauthorized, 403 Forbidden, 404 Not Found); rather than accepting the upgrade and then waiting for an indeterminate amount of time for a custom authorisation to be sent to then disconnect.

There has to be a timeout, just as HTTP servers have a timeout when waiting for the headers to be sent.

[ricea]

Right, doing it after the connection also requires a custom non-HTTP based authentication.

Standard HTTP authentication is already supported by WebSockets. It's only non-standard uses of the Authorization header that have this problem.

I would also like to reiterate that the WebSocket protocol is not HTTP. It just uses an HTTP-based handshake.