Add configuration for WebSocket support

[whitlockjc]

Per the nginx documentation:

As noted above, hop-by-hop headers including “Upgrade” and “Connection” are not passed from a client to proxied server

I propose we have some configuration to enable WebSocket support (and subsequent nginx configuration). Please see the following for more details on nginx and WebSockets: http://nginx.org/en/docs/http/websocket.html

[whitlockjc]

For the short term, let's add a new annotation called websocketPaths and use that.

[arghya]

For our edge router we did this:

proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $connection_header;

Something similar could be done, so that these headers are added if the previous hop sends these.

[whitlockjc]

I'm not asking for implementation details, that's already documented in the link I provided in the issue. I'm asking how should we mark paths as being a WebSocket path? Do we extend the existing path syntax? Do we add a new annotation, like websocketPaths?

[theganyo]

Is it possible to just add those options to all public paths? If so, what's the downside?

[theganyo]

And actually, do private paths also need to be able to enable those options?

[whitlockjc]

I thought about adding it to all paths but mucking with those automatically seems like it could cause issues, especially if we do this the nginx way and downgrade/upgrade the connection to HTTP/1.1 for all connections.

[theganyo]

Hmm. "Seems like it could?" I think I wouldn't add the complexity until there's some evidence that it would cause a problem.

[whitlockjc]

What I ended up doing was enabling this without needing to annotate your Pods...for now.