search

SeleniumHQ / docker-selenium

Provides a simple way to run Selenium Grid with Chrome, Firefox, and Edge using Docker, making it easier to perform browser automation
http://www.selenium.dev/docker-selenium/
Other
7.76k stars 2.5k forks source link

[🐛 Bug]: wss error when using hub-node in k8s cluster #2288

Closed alramostpt closed 1 week ago

alramostpt commented 3 weeks ago

What happened?

When I try to connect to the novnc in a hub-node setup in Kubernetes, I get a wss error where the hub complains about not receiving any bytes in the headers to parse. When I do a port-forward to the hub I'm able to use the novnc.

Command used to start Selenium Grid with Docker (or Kubernetes)

Whatever this container uses: selenium/hub:latest

Relevant log output

hub-log
22:16:14.668 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "d80e4ab48e2ab4583c1babb28d5edb33","eventTime": 1719267374666516390,"eventName": "exception","attributes": {"exception.message": "Unable to execute request for an existing session: java.io.IOException: HTTP\u002f1.1 header parser received no bytes","exception.stacktrace": "java.io.UncheckedIOException: java.io.IOException: HTTP\u002f1.1 header parser received no bytes\n\tat org.openqa.selenium.remote.http.jdk.JdkHttpClient.execute0(JdkHttpClient.java:439)\n\tat org.openqa.selenium.remote.http.RetryRequest.lambda$apply$5(RetryRequest.java:81)\n\tat dev.failsafe.Functions.lambda$toCtxSupplier$11(Functions.java:243)\n\tat dev.failsafe.Functions.lambda$get$0(Functions.java:46)\n\tat dev.failsafe.internal.RetryPolicyExecutor.lambda$apply$0(RetryPolicyExecutor.java:74)\n\tat dev.failsafe.internal.RetryPolicyExecutor.lambda$apply$0(RetryPolicyExecutor.java:74)\n\tat dev.failsafe.internal.FallbackExecutor.lambda$apply$0(FallbackExecutor.java:51)\n\tat dev.failsafe.SyncExecutionImpl.executeSync(SyncExecutionImpl.java:187)\n\tat dev.failsafe.FailsafeExecutor.call(FailsafeExecutor.java:376)\n\tat dev.failsafe.FailsafeExecutor.get(FailsafeExecutor.java:112)\n\tat org.openqa.selenium.remote.http.RetryRequest.lambda$apply$6(RetryRequest.java:81)\n\tat org.openqa.selenium.remote.http.AddSeleniumUserAgent.lambda$apply$0(AddSeleniumUserAgent.java:42)\n\tat org.openqa.selenium.remote.http.Filter.lambda$andFinally$1(Filter.java:55)\n\tat org.openqa.selenium.remote.http.jdk.JdkHttpClient.execute(JdkHttpClient.java:355)\n\tat org.openqa.selenium.remote.tracing.TracedHttpClient.execute(TracedHttpClient.java:54)\n\tat org.openqa.selenium.grid.web.ReverseProxyHandler.execute(ReverseProxyHandler.java:90)\n\tat org.openqa.selenium.grid.router.HandleSession.execute(HandleSession.java:181)\n\tat org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:397)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:69)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:360)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:69)\n\tat org.openqa.selenium.grid.router.Router.execute(Router.java:87)\n\tat org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:63)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:360)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:69)\n\tat org.openqa.selenium.grid.security.BasicAuthenticationFilter.lambda$apply$0(BasicAuthenticationFilter.java:54)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:63)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:360)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:69)\n\tat org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)\n\tat org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:63)\n\tat org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:63)\n\tat org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)\n\tat java.base\u002fjava.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)\n\tat java.base\u002fjava.util.concurrent.FutureTask.run(Unknown Source)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)\n\tat java.base\u002fjava.lang.Thread.run(Unknown Source)\nCaused by: java.io.IOException: HTTP\u002f1.1 header parser received no bytes\n\tat java.net.http\u002fjdk.internal.net.http.common.Utils.wrapWithExtraDetail(Unknown Source)\n\tat java.net.http\u002fjdk.internal.net.http.Http1Response$HeadersReader.onReadError(Unknown Source)\n\tat java.net.http\u002fjdk.internal.net.http.Http1AsyncReceiver.checkForErrors(Unknown Source)\n\tat java.net.http\u002fjdk.internal.net.http.Http1AsyncReceiver.flush(Unknown Source)\n\tat java.net.http\u002fjdk.internal.net.http.common.SequentialScheduler$LockingRestartableTask.run(Unknown Source)\n\tat java.net.http\u002fjdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask.run(Unknown Source)\n\tat java.net.http\u002fjdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(Unknown Source)\n\t... 3 more\nCaused by: java.io.EOFException: EOF reached while reading\n\tat java.net.http\u002fjdk.internal.net.http.Http1AsyncReceiver$Http1TubeSubscriber.onComplete(Unknown Source)\n\tat java.net.http\u002fjdk.internal.net.http.SocketTube$InternalReadPublisher$ReadSubscription.signalCompletion(Unknown Source)\n\tat java.net.http\u002fjdk.internal.net.http.SocketTube$InternalReadPublisher$InternalReadSubscription.read(Unknown Source)\n\tat java.net.http\u002fjdk.internal.net.http.SocketTube$SocketFlowTask.run(Unknown Source)\n\tat java.net.http\u002fjdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(Unknown Source)\n\tat java.net.http\u002fjdk.internal.net.http.common.SequentialScheduler.runOrSchedule(Unknown Source)\n\tat java.net.http\u002fjdk.internal.net.http.common.SequentialScheduler.runOrSchedule(Unknown Source)\n\tat java.net.http\u002fjdk.internal.net.http.SocketTube$InternalReadPublisher$InternalReadSubscription.signalReadable(Unknown Source)\n\tat java.net.http\u002fjdk.internal.net.http.SocketTube$InternalReadPublisher$ReadEvent.signalEvent(Unknown Source)\n\tat java.net.http\u002fjdk.internal.net.http.SocketTube$SocketFlowEvent.handle(Unknown Source)\n\tat java.net.http\u002fjdk.internal.net.http.HttpClientImpl$SelectorManager.handleEvent(Unknown Source)\n\tat java.net.http\u002fjdk.internal.net.http.HttpClientImpl$SelectorManager.lambda$run$3(Unknown Source)\n\tat java.base\u002fjava.util.ArrayList.forEach(Unknown Source)\n\tat java.net.http\u002fjdk.internal.net.http.HttpClientImpl$SelectorManager.run(Unknown Source)\n","exception.type": "java.io.UncheckedIOException","http.client_ip": "192.168.0.161","http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.router.HandleSession","http.host": "{REDACTED}","http.method": "GET","http.scheme": "HTTP","http.target": "\u002fsession\u002f21e51f2f1bf022ac211b3a6361de7884\u002fse\u002fvnc","http.user_agent": "Mozilla\u002f5.0 (Windows NT 10.0; Win64; x64; rv:122.0) Gecko\u002f20100101 Firefox\u002f122.0","session.id": "21e51f2f1bf022ac211b3a6361de7884"}}

browser log:
Firefox can’t establish a connection to the server at wss://{domain}/session/3b6cf41e6b946dc16a9feacc7b0f788f/se/vnc.

Operating System

AWS Linux

Docker Selenium version (image tag)

n/a

Selenium Grid chart version (chart version)

Selenium Grid 4.22.0 (revision c5f3146703)

github-actions[bot] commented 3 weeks ago

@alramostpt, thank you for creating this issue. We will troubleshoot it as soon as we can.

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

VietND96 commented 3 weeks ago

Are you using helm chart to deploy or your created YAML? Can you share YAML to debug further

alramostpt commented 3 weeks ago

I'm not using Helm, Just a straight forward K8s deployment: 

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: sel-hub
  name: sel-hub
  namespace: selenium
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sel-hub
  template:
    metadata:
      labels:
        app: sel-hub
    spec:
      containers:
      - env:
        - name: SE_OPTS
          value: --username {REDACTED} --password {REDACTED}
        image: selenium/hub:latest
        imagePullPolicy: Always
        name: sel-hub
        ports:
        - containerPort: 4442
          name: p4442
          protocol: TCP
        - containerPort: 4443
          name: p4443
          protocol: TCP
        - containerPort: 4444
          name: p4444
          protocol: TCP
      restartPolicy: Always
apiVersion: v1
kind: Service
metadata:
  name: sel-hub-svc
  namespace: selenium
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 4444
  - name: http2
    port: 4444
    protocol: TCP
    targetPort: 4444
  - name: publish
    port: 4442
    protocol: TCP
    targetPort: 4442
  - name: subcribe
    port: 4443
    protocol: TCP
    targetPort: 4443
  selector:
    app: sel-hub
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: {REDACTED}
    kubernetes.io/tls-acme: "true"
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
  name: sel-hub-ingress
  namespace: selenium
spec:
  ingressClassName: nginx-{REDACTED}
  rules:
  - host: {REDACTED}
    http:
      paths:
      - backend:
          service:
            name: sel-hub-svc
            port:
              number: 80
        path: /
        pathType: Prefix
  tls:
  - hosts:
    - {REDACTED}
    secretName: {REDACTED}
alramostpt commented 3 weeks ago

Ok, so I tried a few more things. So adding this annotation to the ingress seems to have made the trick for me: nginx.org/websocket-services: {service-name}.

Leaving this here is case it helps others.

This issue can be considered resolved. Thanks for your time.

VietND96 commented 3 weeks ago

What is the value set to env var SE_NODE_GRID_URL in Node config? Since this is used to construct the URL for other WebSocket

VietND96 commented 3 weeks ago

By the way, which kind of ingress service is used in your case? NGINX controller or something else?

alramostpt commented 3 weeks ago

What is the value set to env var SE_NODE_GRID_URL in Node config? Since this is used to construct the URL for other WebSocket

Whatever is the default value of it since I'm not setting that on my side.

By the way, which kind of ingress service is used in your case? NGINX controller or something else?

Yes, I'm using NGINX as ingress controller. There is a hint of that in my ingress definition. Sorry that I did not stated that directly.

VietND96 commented 1 week ago

Resolved by following comment https://github.com/SeleniumHQ/docker-selenium/issues/2288#issuecomment-2189377116