gravitee-io / issues

Gravitee.io - API Platform - Issues
65 stars 26 forks source link

[Gateway] WebSocket, ERROR i.vertx.core.net.impl.ConnectionBase - RSV != 0 and no extension negotiated, RSV:4 #8115

Closed grignix closed 1 year ago

grignix commented 2 years ago

Describe the bug

WebSocket connection not working if using direct connection or nginx instead of gravitee, then everything works

gravitee 3.15.12 Chrome 103.0.5060.114 Edge 103.0.1264.44

in logs

20:04:59.313 [vert.x-eventloop-thread-1] [] DEBUG i.n.h.c.h.w.WebSocketClientHandshaker13 - WebSocket version 13 client handshake key: D+tdRpdVxIJv2Gu7Mcg5Yw==, expected response: OQZkMudxqMok2smXX18VeFDklIo= 20:04:59.322 [vert.x-eventloop-thread-1] [] DEBUG i.v.c.h.impl.Http1xClientConnection - WebSocket handshake complete 20:04:59.323 [vert.x-eventloop-thread-1] [] TRACE i.n.h.c.h.w.WebSocket08FrameDecoder - Decoding WebSocket Frame opCode=1 20:04:59.323 [vert.x-eventloop-thread-1] [] ERROR i.vertx.core.net.impl.ConnectionBase - RSV != 0 and no extension negotiated, RSV:4 io.netty.handler.codec.http.websocketx.CorruptedWebSocketFrameException: RSV != 0 and no extension negotiated, RSV:4 at io.netty.handler.codec.http.websocketx.WebSocket08FrameDecoder.protocolViolation(WebSocket08FrameDecoder.java:424) at io.netty.handler.codec.http.websocketx.WebSocket08FrameDecoder.protocolViolation(WebSocket08FrameDecoder.java:420) at io.netty.handler.codec.http.websocketx.WebSocket08FrameDecoder.decode(WebSocket08FrameDecoder.java:197) at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:510) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:449) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:279) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:327) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:299) at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1372) at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1235) at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1284) at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:510) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:449) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:279) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:829) 20:04:59.325 [vert.x-eventloop-thread-1] [] DEBUG i.n.h.c.h.w.WebSocketServerHandshaker - [id: 0xf0fe3b1b, L:/192.168.86.228:443 - R:/172.19.88.36:56061] WebSocket version V13 server handshake 20:04:59.325 [vert.x-eventloop-thread-1] [] DEBUG i.n.h.c.h.w.WebSocketServerHandshaker - WebSocket version 13 server handshake key: n46pHCe4LiERcgyFr2rdnw==, response: CZYUeCRzchRsm+K3kjeqNOpH5qc= 20:04:59.327 [vert.x-eventloop-thread-1] [] TRACE i.n.h.c.h.w.WebSocket08FrameEncoder - Encoding WebSocket Frame opCode=8 length=2 20:04:59.336 [vert.x-eventloop-thread-1] [] TRACE i.n.h.c.h.w.WebSocket08FrameDecoder - Decoding WebSocket Frame opCode=8 20:04:59.336 [vert.x-eventloop-thread-1] [] TRACE i.n.h.c.h.w.WebSocket08FrameDecoder - Decoding WebSocket Frame length=2 20:04:59.337 [vert.x-eventloop-thread-1] [] TRACE i.v.core.http.impl.HttpServerImpl - Connection failure io.netty.handler.codec.http.websocketx.CorruptedWebSocketFrameException: Invalid close frame getStatus code: 400 at io.netty.handler.codec.http.websocketx.WebSocket08FrameDecoder.protocolViolation(WebSocket08FrameDecoder.java:424) at io.netty.handler.codec.http.websocketx.WebSocket08FrameDecoder.protocolViolation(WebSocket08FrameDecoder.java:420) at io.netty.handler.codec.http.websocketx.WebSocket08FrameDecoder.checkCloseFrameBody(WebSocket08FrameDecoder.java:473) at io.netty.handler.codec.http.websocketx.WebSocket08FrameDecoder.decode(WebSocket08FrameDecoder.java:339) at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:510) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:449) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:279) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1372) at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1235) at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1284) at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:510) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:449) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:279) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:829)
grignix commented 2 years ago

chrome: if use direct connection or nginx instead of gravity, then everything works image003 chrome: when connected via gravitee, in gravitee logs the error described above image002

firefox: does not work, the problem is described here https://github.com/gravitee-io/issues/issues/8224

Majlon commented 2 years ago

Hello, I'm also experiencing problems with connection to WebSockets.

GETws://192.168.0.136:8073/websockets [HTTP/1.1 502 Bad Gateway 5ms] Firefox can’t establish a connection to the server at ws://192.168.0.136:8073/websockets.

Port 8073 is gateway and 7071 is WebSocket Server part of application.

Here is APIM entity

{
  "id" : "d607dc5d-66f7-4d39-87dc-5d66f72d39db",
  "name" : "wsockets",
  "version" : "1.0",
  "description" : "wsockets-testing",
  "groups" : [ ],
  "visibility" : "PUBLIC",
  "state" : "STARTED",
  "tags" : [ ],
  "labels" : [ ],
  "entrypoints" : [ {
    "target" : "https://api.company.com/websockets"
  } ],
  "execution_mode" : "v3",
  "context_path" : "/websockets",
  "proxy" : {
    "virtual_hosts" : [ {
      "path" : "/websockets"
    } ],
    "strip_context_path" : false,
    "preserve_host" : false,
    "groups" : [ {
      "name" : "default-group",
      "endpoints" : [ {
        "backup" : false,
        "inherit" : true,
        "name" : "default",
        "weight" : 1,
        "type" : "http",
        "target" : "ws://192.168.0.136:7071/"
      } ],
      "load_balancing" : {
        "type" : "ROUND_ROBIN"
      },
      "http" : {
        "connectTimeout" : 5000,
        "idleTimeout" : 60000,
        "keepAlive" : true,
        "readTimeout" : 10000,
        "pipelining" : false,
        "maxConcurrentConnections" : 100,
        "useCompression" : true,
        "followRedirects" : false
      }
    } ]
  },
  "flow_mode" : "DEFAULT",
  "flows" : [ ],
  "plans" : [ {
    "id" : "8835d340-abcf-47ac-b5d3-40abcf27aca1",
    "name" : "Basic",
    "description" : "asd",
    "validation" : "AUTO",
    "security" : "KEY_LESS",
    "type" : "API",
    "status" : "PUBLISHED",
    "api" : "d607dc5d-66f7-4d39-87dc-5d66f72d39db",
    "order" : 0,
    "characteristics" : [ ],
    "created_at" : 1658436280890,
    "updated_at" : 1658436839962,
    "published_at" : 1658436280890,
    "paths" : { },
    "flows" : [ {
      "name" : "",
      "path-operator" : {
        "path" : "/",
        "operator" : "STARTS_WITH"
      },
      "condition" : "",
      "consumers" : [ ],
      "methods" : [ ],
      "pre" : [ ],
      "post" : [ ],
      "enabled" : true
    } ],
    "comment_required" : false
  } ],
  "gravitee" : "2.0.0",
  "deployed_at" : 1658436843048,
  "created_at" : 1658436280763,
  "updated_at" : 1658436843048,
  "owner" : {
    "id" : "2c3e61f2-84ba-4469-be61-f284ba1469f9",
    "displayName" : "admin",
    "type" : "USER"
  },
  "properties" : [ ],
  "services" : { },
  "picture_url" : "http://192.168.0.136:8083/management/organizations/DEFAULT/environments/DEFAULT/apis/d607dc5d-66f7-4d39-87dc-5d66f72d39db/picture?hash=1658436843048",
  "resources" : [ ],
  "path_mappings" : [ ],
  "response_templates" : { },
  "lifecycle_state" : "PUBLISHED",
  "disable_membership_notifications" : false,
  "background_url" : "http://192.168.0.136:8083/management/organizations/DEFAULT/environments/DEFAULT/apis/d607dc5d-66f7-4d39-87dc-5d66f72d39db/background?hash=1658436843048"
}

Am I missing something? Any help or advice is greatly appreciated! I can provide

Best Regards Milan.

grignix commented 2 years ago

Hi @mouligno Please tell me if there are any solutions to my problem?

grignix commented 2 years ago

it seems that gravity cannot work with compression in websockets, the Sec-WebSocket-Extensions: permessage-deflate header causes an error (RSV != 0 and no extension negotiated, RSV:4), if you put nginx before gravity and cut the Sec-WebSocket-Extensions: permessage-deflate header, then error (RSV != 0 and no extension negotiated, RSV:4) disappears

Sachkov-Aleksandr commented 2 years ago

Hi, we are facing the same problem on the latest version 3.19.0, please tell me when it is planned to solve this problem?

grignix commented 1 year ago

in version 3.20.1 this error was fixed, is it possible to backport changes to version 3.15.x?

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

gaetanmaisse commented 1 year ago

Just a quick update! I'm going to close this issue since it's fixed in APIM 3.20.1+ (no backport on older version is planned). If you have more insights or want to revisit this issue, feel free to reopen or create a new one!

HK-hub commented 1 year ago

Please tell me if there are any solutions , because I also have this problem

grignix commented 1 year ago

Please tell me if there are any solutions , because I also have this problem

if you use Nginx as a backend, configure it like this: proxy_set_header Sec-WebSocket-Extensions ""; proxy_hide_header Sec-WebSocket-Extensions;