[🐛 Bug]: Memory leaks on Selenium Hub and enabled CDP

IlyaUk commented 1 month ago

What happened?

An attempt to run tests that use CDP for files download on remote machine via Selenium Hub (hub and nodes are on the same machine) shows a problem with memory leak on Hub side. As a result test run (around 200 tests and about 10 tests that use CDP) fails with the following exception: Response code 500. Message: session not created: DevToolsActivePort file doesn't exist After that Docker stops the Hub container with OutOfMemory exception. If we dive deep into Selenium server thread dump then we see the following: About 16132 threads open with stacktrace: java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPoll.wait(java.base@17.0.11/Native Method) at sun.nio.ch.EPollSelectorImpl.doSelect(java.base@17.0.11/Unknown Source) at sun.nio.ch.SelectorImpl.lockAndDoSelect(java.base@17.0.11/Unknown Source)

locked <0x0000000618403338> (a sun.nio.ch.Util$2)
locked <0x00000006184032e8> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(java.base@17.0.11/Unknown Source) at zmq.poll.Poller.run(Poller.java:236) at java.lang.Thread.run(java.base@17.0.11/Unknown Source)
The only one workaround to avoid the issue - set Selenium flag --enable-cdp false for Chrome nodes and disable all tests that use CDP features.

Command used to start Selenium Grid with Docker (or Kubernetes)

version: "3"
services:
  hub:
    image: selenium/hub:4.22.0-20240621             
    container_name: hub
    ports:
      - "4444:4444"
      - "4443:4443"
      - "4442:4442"
    healthcheck:
      test: "/opt/bin/check-grid.sh --host 0.0.0.0 --port 4444"
      interval: "15s"
      timeout: "30s"
      retries: 5
    restart: always

  chrome:
    image: selenium/node-chrome:4.22.0-20240621
    shm_size: 2gb
    depends_on:
      - hub 
    environment:
      - SE_EVENT_BUS_HOST=hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_VNC_NO_PASSWORD=1
      - SE_SCREEN_WIDTH=1600
      - SE_SCREEN_HEIGHT=900
      - SE_NODE_MAX_SESSIONS=1
      - SE_NODE_MAX_INSTANCES=1
      - SE_NODE_GRID_URL=http://localhost:4444
      - SE_NODE_SESSION_TIMEOUT=120
      - SE_OPTS=--enable-managed-downloads true
    restart: always
    deploy:
      mode: replicated
      replicas: 16

Relevant log output

16:19:43.216 INFO [LocalDistributor.newSession] - Session request received by the Distributor: 
 [Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=<-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: normal, unhandledPromptBehavior: accept}]
16:19:46.802 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "61711431acc8f028fc6d505a3737dbc4","eventTime": 1721319514178276326,"eventName": "Session request received by the Distributor","attributes": {"logger": "org.openqa.selenium.grid.distributor.local.LocalDistributor","request.payload": "[Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=\u003c-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: normal, se:downloadsEnabled: false, unhandledPromptBehavior: accept}]"}}

16:19:46.802 WARN [SeleniumSpanExporter$1.lambda$export$1] - org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist 
Host info: host: '048617283a4f', ip: '172.18.0.16'
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
    at org.openqa.selenium.grid.node.remote.RemoteNode.newSession(RemoteNode.java:157)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession(LocalDistributor.java:652)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession(LocalDistributor.java:571)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest(LocalDistributor.java:834)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1(LocalDistributor.java:791)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)

16:19:46.802 WARN [SeleniumSpanExporter$1.lambda$export$1] - Unable to create session: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist 
Host info: host: '048617283a4f', ip: '172.18.0.16'
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
16:19:46.803 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "61711431acc8f028fc6d505a3737dbc4","eventTime": 1721319586802543052,"eventName": "exception","attributes": {"exception.message": "Unable to create session: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist \nHost info: host: '048617283a4f', ip: '172.18.0.16'\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown","exception.stacktrace": "org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist \nHost info: host: '048617283a4f', ip: '172.18.0.16'\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\n\tat org.openqa.selenium.grid.node.remote.RemoteNode.newSession(RemoteNode.java:157)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession(LocalDistributor.java:652)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession(LocalDistributor.java:571)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest(LocalDistributor.java:834)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1(LocalDistributor.java:791)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)\n\tat java.base\u002fjava.lang.Thread.run(Unknown Source)\n","exception.type": "org.openqa.selenium.SessionNotCreatedException","logger": "org.openqa.selenium.grid.distributor.local.LocalDistributor","request.payload": "[Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=\u003c-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: normal, se:downloadsEnabled: false, unhandledPromptBehavior: accept}]"}}

16:19:46.804 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "61711431acc8f028fc6d505a3737dbc4","eventTime": 1721319586803890987,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "1974","http.scheme": "HTTP","http.status_code": 500,"http.target": "\u002fsession","http.user_agent": "selenium\u002f4.22.0 (java unix)"}}

16:19:49.137 INFO [LocalDistributor.newSession] - Session request received by the Distributor: 
 [Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=<-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: normal, se:downloadsEnabled: false, unhandledPromptBehavior: accept}]
16:19:56.568 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "0ae456ef54eb12ee4469c3c47734b88a","eventTime": 1721319462675940687,"eventName": "Session request received by the Distributor","attributes": {"logger": "org.openqa.selenium.grid.distributor.local.LocalDistributor","request.payload": "[Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=\u003c-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: normal, unhandledPromptBehavior: accept}]"}}

16:19:56.587 WARN [SeleniumSpanExporter$1.lambda$export$1] - org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist 
Host info: host: '21782a01b177', ip: '172.18.0.13'
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
    at org.openqa.selenium.grid.node.remote.RemoteNode.newSession(RemoteNode.java:157)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession(LocalDistributor.java:652)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession(LocalDistributor.java:571)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest(LocalDistributor.java:834)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1(LocalDistributor.java:791)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)

16:19:56.587 WARN [SeleniumSpanExporter$1.lambda$export$1] - Unable to create session: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist 
Host info: host: '21782a01b177', ip: '172.18.0.13'
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
16:19:56.587 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "0ae456ef54eb12ee4469c3c47734b88a","eventTime": 1721319596567720207,"eventName": "exception","attributes": {"exception.message": "Unable to create session: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist \nHost info: host: '21782a01b177', ip: '172.18.0.13'\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown","exception.stacktrace": "org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist \nHost info: host: '21782a01b177', ip: '172.18.0.13'\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\n\tat org.openqa.selenium.grid.node.remote.RemoteNode.newSession(RemoteNode.java:157)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession(LocalDistributor.java:652)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession(LocalDistributor.java:571)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest(LocalDistributor.java:834)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1(LocalDistributor.java:791)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)\n\tat java.base\u002fjava.lang.Thread.run(Unknown Source)\n","exception.type": "org.openqa.selenium.SessionNotCreatedException","logger": "org.openqa.selenium.grid.distributor.local.LocalDistributor","request.payload": "[Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=\u003c-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: normal, unhandledPromptBehavior: accept}]"}}

16:19:56.605 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "0ae456ef54eb12ee4469c3c47734b88a","eventTime": 1721319596605322631,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "1936","http.scheme": "HTTP","http.status_code": 500,"http.target": "\u002fsession","http.user_agent": "selenium\u002f4.22.0 (java unix)"}}

16:20:57.570 INFO [LocalDistributor.newSession] - Session request received by the Distributor: 
 [Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=<-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: normal, unhandledPromptBehavior: accept}]
16:21:01.568 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "231f2030dedc13b799d35172f2e174d1","eventTime": 1721319589137535412,"eventName": "Session request received by the Distributor","attributes": {"logger": "org.openqa.selenium.grid.distributor.local.LocalDistributor","request.payload": "[Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=\u003c-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: normal, se:downloadsEnabled: false, unhandledPromptBehavior: accept}]"}}

16:21:01.568 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "1776403ab402e4d69344db4b1e3eac00","eventTime": 1721319583216633573,"eventName": "Session request received by the Distributor","attributes": {"logger": "org.openqa.selenium.grid.distributor.local.LocalDistributor","request.payload": "[Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=\u003c-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: normal, unhandledPromptBehavior: accept}]"}}

16:21:01.722 WARN [SeleniumSpanExporter$1.lambda$export$1] - org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist 
Host info: host: '76720f9d5b8d', ip: '172.18.0.12'
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
    at org.openqa.selenium.grid.node.remote.RemoteNode.newSession(RemoteNode.java:157)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession(LocalDistributor.java:652)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession(LocalDistributor.java:571)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest(LocalDistributor.java:834)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1(LocalDistributor.java:791)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)

16:21:01.722 WARN [SeleniumSpanExporter$1.lambda$export$1] - org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist 
Host info: host: 'b3238a3e4104', ip: '172.18.0.8'
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
    at org.openqa.selenium.grid.node.remote.RemoteNode.newSession(RemoteNode.java:157)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession(LocalDistributor.java:652)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession(LocalDistributor.java:571)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest(LocalDistributor.java:834)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1(LocalDistributor.java:791)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)

16:21:01.722 WARN [SeleniumSpanExporter$1.lambda$export$1] - Unable to create session: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist 
Host info: host: '76720f9d5b8d', ip: '172.18.0.12'
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
16:21:01.722 WARN [SeleniumSpanExporter$1.lambda$export$1] - Unable to create session: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist 
Host info: host: 'b3238a3e4104', ip: '172.18.0.8'
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
16:21:01.723 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "1776403ab402e4d69344db4b1e3eac00","eventTime": 1721319661517911831,"eventName": "exception","attributes": {"exception.message": "Unable to create session: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist \nHost info: host: 'b3238a3e4104', ip: '172.18.0.8'\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown","exception.stacktrace": "org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist \nHost info: host: 'b3238a3e4104', ip: '172.18.0.8'\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\n\tat org.openqa.selenium.grid.node.remote.RemoteNode.newSession(RemoteNode.java:157)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession(LocalDistributor.java:652)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession(LocalDistributor.java:571)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest(LocalDistributor.java:834)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1(LocalDistributor.java:791)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)\n\tat java.base\u002fjava.lang.Thread.run(Unknown Source)\n","exception.type": "org.openqa.selenium.SessionNotCreatedException","logger": "org.openqa.selenium.grid.distributor.local.LocalDistributor","request.payload": "[Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=\u003c-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: normal, unhandledPromptBehavior: accept}]"}}

16:21:01.723 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "231f2030dedc13b799d35172f2e174d1","eventTime": 1721319661518352446,"eventName": "exception","attributes": {"exception.message": "Unable to create session: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist \nHost info: host: '76720f9d5b8d', ip: '172.18.0.12'\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown","exception.stacktrace": "org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist \nHost info: host: '76720f9d5b8d', ip: '172.18.0.12'\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\n\tat org.openqa.selenium.grid.node.remote.RemoteNode.newSession(RemoteNode.java:157)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession(LocalDistributor.java:652)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession(LocalDistributor.java:571)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest(LocalDistributor.java:834)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1(LocalDistributor.java:791)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)\n\tat java.base\u002fjava.lang.Thread.run(Unknown Source)\n","exception.type": "org.openqa.selenium.SessionNotCreatedException","logger": "org.openqa.selenium.grid.distributor.local.LocalDistributor","request.payload": "[Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=\u003c-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: normal, se:downloadsEnabled: false, unhandledPromptBehavior: accept}]"}}

16:21:01.724 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "231f2030dedc13b799d35172f2e174d1","eventTime": 1721319661723953633,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "1974","http.scheme": "HTTP","http.status_code": 500,"http.target": "\u002fsession","http.user_agent": "selenium\u002f4.22.0 (java unix)"}}

16:21:01.724 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "1776403ab402e4d69344db4b1e3eac00","eventTime": 1721319661724146123,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "1936","http.scheme": "HTTP","http.status_code": 500,"http.target": "\u002fsession","http.user_agent": "selenium\u002f4.22.0 (java unix)"}}

16:21:11.175 INFO [LocalDistributor.newSession] - Session request received by the Distributor: 
 [Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=<-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: normal, unhandledPromptBehavior: accept}]
16:21:31.319 INFO [LocalDistributor.newSession] - Session request received by the Distributor: 
 [Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=<-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: normal, unhandledPromptBehavior: accept}]
16:21:40.155 INFO [LocalDistributor.newSession] - Session request received by the Distributor: 
 [Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=<-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: normal, se:downloadsEnabled: false, unhandledPromptBehavior: accept}]
16:21:40.455 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "29f2705f5770f23899b26d2d5954035a","eventTime": 1721319548049803595,"eventName": "Session request received by the Distributor","attributes": {"logger": "org.openqa.selenium.grid.distributor.local.LocalDistributor","request.payload": "[Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=\u003c-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: normal, unhandledPromptBehavior: accept}]"}}

16:21:40.456 WARN [SeleniumSpanExporter$1.lambda$export$1] - org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist 
Host info: host: '272b3789ad29', ip: '172.18.0.10'
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
    at org.openqa.selenium.grid.node.remote.RemoteNode.newSession(RemoteNode.java:157)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession(LocalDistributor.java:652)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession(LocalDistributor.java:571)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest(LocalDistributor.java:834)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1(LocalDistributor.java:791)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)

16:21:40.456 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "ee82cc0d662d52c549550b0d5982a2b7","eventTime": 1721319552030846334,"eventName": "Session request received by the Distributor","attributes": {"logger": "org.openqa.selenium.grid.distributor.local.LocalDistributor","request.payload": "[Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=\u003c-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: normal, unhandledPromptBehavior: accept}]"}}

16:21:40.456 WARN [SeleniumSpanExporter$1.lambda$export$1] - Unable to create session: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist 
Host info: host: '272b3789ad29', ip: '172.18.0.10'
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
16:21:40.456 WARN [SeleniumSpanExporter$1.lambda$export$1] - org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist 
Host info: host: '1b66a0eea552', ip: '172.18.0.14'
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
    at org.openqa.selenium.grid.node.remote.RemoteNode.newSession(RemoteNode.java:157)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession(LocalDistributor.java:652)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession(LocalDistributor.java:571)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest(LocalDistributor.java:834)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1(LocalDistributor.java:791)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)

16:21:40.456 WARN [SeleniumSpanExporter$1.lambda$export$1] - Unable to create session: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist 
Host info: host: '1b66a0eea552', ip: '172.18.0.14'
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'
Driver info: driver.version: unknown
16:21:40.456 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "29f2705f5770f23899b26d2d5954035a","eventTime": 1721319700455617990,"eventName": "exception","attributes": {"exception.message": "Unable to create session: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist \nHost info: host: '272b3789ad29', ip: '172.18.0.10'\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown","exception.stacktrace": "org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist \nHost info: host: '272b3789ad29', ip: '172.18.0.10'\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\n\tat org.openqa.selenium.grid.node.remote.RemoteNode.newSession(RemoteNode.java:157)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession(LocalDistributor.java:652)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession(LocalDistributor.java:571)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest(LocalDistributor.java:834)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1(LocalDistributor.java:791)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)\n\tat java.base\u002fjava.lang.Thread.run(Unknown Source)\n","exception.type": "org.openqa.selenium.SessionNotCreatedException","logger": "org.openqa.selenium.grid.distributor.local.LocalDistributor","request.payload": "[Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=\u003c-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: normal, unhandledPromptBehavior: accept}]"}}

16:21:40.456 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "ee82cc0d662d52c549550b0d5982a2b7","eventTime": 1721319700456016923,"eventName": "exception","attributes": {"exception.message": "Unable to create session: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist \nHost info: host: '1b66a0eea552', ip: '172.18.0.14'\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown","exception.stacktrace": "org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: DevToolsActivePort file doesn't exist \nHost info: host: '1b66a0eea552', ip: '172.18.0.14'\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\nBuild info: version: '4.22.0', revision: 'c5f3146703'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.11'\nDriver info: driver.version: unknown\n\tat org.openqa.selenium.grid.node.remote.RemoteNode.newSession(RemoteNode.java:157)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession(LocalDistributor.java:652)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession(LocalDistributor.java:571)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest(LocalDistributor.java:834)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1(LocalDistributor.java:791)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)\n\tat java.base\u002fjava.lang.Thread.run(Unknown Source)\n","exception.type": "org.openqa.selenium.SessionNotCreatedException","logger": "org.openqa.selenium.grid.distributor.local.LocalDistributor","request.payload": "[Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=\u003c-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: normal, unhandledPromptBehavior: accept}]"}}

16:21:40.495 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "ee82cc0d662d52c549550b0d5982a2b7","eventTime": 1721319700495380127,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "1936","http.scheme": "HTTP","http.status_code": 500,"http.target": "\u002fsession","http.user_agent": "selenium\u002f4.22.0 (java unix)"}}

16:21:40.495 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "29f2705f5770f23899b26d2d5954035a","eventTime": 1721319700495380160,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "1936","http.scheme": "HTTP","http.status_code": 500,"http.target": "\u002fsession","http.user_agent": "selenium\u002f4.22.0 (java unix)"}}

/opt/bin/start-selenium-grid-hub.sh: line 124:    10 Killed                  java ${JAVA_OPTS:-$SE_JAVA_OPTS} -jar /opt/selenium/selenium-server.jar ${EXTRA_LIBS} hub --session-request-timeout ${SE_SESSION_REQUEST_TIMEOUT} --session-retry-interval ${SE_SESSION_RETRY_INTERVAL} --healthcheck-interval ${SE_HEALTHCHECK_INTERVAL} --relax-checks ${SE_RELAX_CHECKS} --bind-host ${SE_BIND_HOST} --config /opt/selenium/config.toml ${HOST_CONFIG} ${PORT_CONFIG} ${SUB_PATH_CONFIG} ${SE_OPTS}
2024-07-18 16:26:39,131 INFO exited: selenium-grid-hub (exit status 137; not expected)

Operating System

Ubuntu

Docker Selenium version (image tag)

4.22.0-20240621

Selenium Grid chart version (chart version)

-

github-actions[bot] commented 1 month ago

@IlyaUk, thank you for creating this issue. We will troubleshoot it as soon as we can.

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

diemol commented 1 month ago

This is the recommended method for downloads: https://www.selenium.dev/documentation/grid/configuration/cli_options/#enabling-managed-downloads-by-the-node.

However, this is not a tool to download massive files. It is a use case in automation, but it does not cover all situations.

You might try to give more memory to the Hub and adjust it to your use case. Memory will be released when the GC passes, but it is also possible that the size of the file is big enough to exhaust the memory available.

IlyaUk commented 1 month ago

Hi @diemol, Thanks for the advice with --enable-managed-downloads true option. This option is enabled in my Selenium configuration. The above described problem can not be resolved by memory increase. I gave more than 4Gb for Hub and still no effect. Also I don't try to download large files. Each of my 10 tests that use CDP downloads a file with size around 10-50 Kb. And in general the problem with CDP is not with files download only. An attempt to use CDP for request interceptions also shows problems with memory consumption on Linux machine. But I have only one such case so the negative trend with memory is not so noticeable.

diemol commented 1 month ago

We need a way to reproduce the issue.

github-actions[bot] commented 1 month ago

Hi, @IlyaUk. Please follow the issue template, we need more information to reproduce the issue.

Either a complete code snippet and URL/HTML (if more than one file is needed, provide a GitHub repo and instructions to run the code), the specific versions used, or a more detailed description to help us understand the issue.

Note: If you cannot share your code and URL/HTML, any complete code snippet and URL/HTML that reproduces the issue is good enough.

Reply to this issue when all information is provided, thank you.

IlyaUk commented 1 month ago

We need a way to reproduce the issue.

Hi @diemol, I sent an invitation to my project (JavaProject) with small test that reproduces the problem. Docker compose file was previously attached to the issue.

zerikv commented 1 month ago

Hello, Same problem here (same version of Selenium, same stacktrace in Hub logs). Tests run during ~20 min on a grid of 8 Firefox nodes, errors start to occur near the end of job.

For me, disabling the VNC client help :

SE_START_VNC: "false"

joerg1985 commented 1 month ago

The DriverServiceSessionFactory is not closing the HttpClient when the session fails to start, will fix this the next days.

joerg1985 commented 1 month ago

https://github.com/SeleniumHQ/selenium/commit/97d56d04e1b4ab4f8e527f8849b777c1e91d13f7 is the promised commit @IlyaUk could you update to the next nightly and check?

IlyaUk commented 1 month ago

Hi @joerg1985, @diemol The fix https://github.com/SeleniumHQ/selenium/commit/97d56d04e1b4ab4f8e527f8849b777c1e91d13f7 didn't help. Selenium Hub revision: 7c8727b Docker-compose file:

version: "3"
services:
  tmp-hub:
    image: selenium/hub:nightly
    container_name: tmp-hub
    ports:
      - "4444:4444"
      - "4443:4443"
      - "4442:4442"
    healthcheck:
      test: "/opt/bin/check-grid.sh --host 0.0.0.0 --port 4444"
      interval: "15s"
      timeout: "30s"
      retries: 5
    restart: always

  chrome:
    image: selenium/node-chrome:nightly
    shm_size: 2gb
    depends_on:
      - tmp-hub
    environment:
      - SE_EVENT_BUS_HOST=tmp-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_VNC_NO_PASSWORD=1
      - SE_SCREEN_WIDTH=1600           
      - SE_SCREEN_HEIGHT=900             
      - SE_NODE_MAX_SESSIONS=1                         
      - SE_NODE_MAX_INSTANCES=1                                 
      - SE_NODE_GRID_URL=http://localhost:4444
      - SE_OPTS=--enable-managed-downloads true
    restart: always
    deploy:
      mode: replicated
      replicas: 16

Selenium hub logs output:

13:12:05.172 INFO [LocalDistributor.newSession] - Session request received by the Distributor: 
 [Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=<-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: eager, se:downloadsEnabled: false, unhandledPromptBehavior: accept}]
13:12:44.808 INFO [LocalDistributor.newSession] - Session created by the Distributor. Id: c6bb0fbdfc014f5f695c0ffdb8532013 
 Caps: Capabilities {acceptInsecureCerts: true, browserName: chrome, browserVersion: 127.0.6533.72, chrome: {chromedriverVersion: 127.0.6533.72 (9755e24ca85a..., userDataDir: /tmp/.org.chromium.Chromium...}, fedcm:accounts: true, goog:chromeOptions: {debuggerAddress: localhost:41275}, networkConnectionEnabled: false, pageLoadStrategy: eager, platformName: linux, proxy: {}, se:bidiEnabled: false, se:cdp: ws://172.30.4.67:4444/sessi..., se:cdpVersion: 127.0.6533.72, se:downloadsEnabled: false, se:vnc: ws://172.30.4.67:4444/sessi..., se:vncEnabled: true, se:vncLocalAddress: ws://172.18.0.6:7900, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: accept, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true}
13:12:45.467 INFO [GridModel.release] - Releasing slot for session id 50a6c73f8fe0bf337f8334af9fe6bfed
13:12:45.467 INFO [LocalSessionMap.lambda$new$0] - Deleted session from local Session Map, Id: 50a6c73f8fe0bf337f8334af9fe6bfed
13:12:55.143 INFO [LocalDistributor.newSession] - Session request received by the Distributor: 
 [Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=<-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: eager, unhandledPromptBehavior: accept}]
13:13:52.086 INFO [LocalDistributor.newSession] - Session created by the Distributor. Id: 641ec35ff3f68944efa27fef028c7c63 
 Caps: Capabilities {acceptInsecureCerts: true, browserName: chrome, browserVersion: 127.0.6533.72, chrome: {chromedriverVersion: 127.0.6533.72 (9755e24ca85a..., userDataDir: /tmp/.org.chromium.Chromium...}, fedcm:accounts: true, goog:chromeOptions: {debuggerAddress: localhost:39097}, networkConnectionEnabled: false, pageLoadStrategy: eager, platformName: linux, proxy: {}, se:bidiEnabled: false, se:cdp: ws://172.30.4.67:4444/sessi..., se:cdpVersion: 127.0.6533.72, se:vnc: ws://172.30.4.67:4444/sessi..., se:vncEnabled: true, se:vncLocalAddress: ws://172.18.0.8:7900, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: accept, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true}
13:15:19.227 INFO [GridModel.release] - Releasing slot for session id 9fe2f02f0f83e27320023f8037a61163
13:15:19.227 INFO [LocalSessionMap.lambda$new$0] - Deleted session from local Session Map, Id: 9fe2f02f0f83e27320023f8037a61163
13:15:20.133 INFO [GridModel.release] - Releasing slot for session id bde56e1d476bd0088fa9fe1aa26d374e
13:15:20.133 INFO [LocalSessionMap.lambda$new$0] - Deleted session from local Session Map, Id: bde56e1d476bd0088fa9fe1aa26d374e
13:15:23.337 INFO [LocalDistributor.newSession] - Session request received by the Distributor: 
 [Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=<-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: eager, unhandledPromptBehavior: accept}]
13:15:24.336 INFO [LocalDistributor.newSession] - Session created by the Distributor. Id: 030bb33f2b4520f77b9a09c085b8e46f 
 Caps: Capabilities {acceptInsecureCerts: true, browserName: chrome, browserVersion: 127.0.6533.72, chrome: {chromedriverVersion: 127.0.6533.72 (9755e24ca85a..., userDataDir: /tmp/.org.chromium.Chromium...}, fedcm:accounts: true, goog:chromeOptions: {debuggerAddress: localhost:37113}, networkConnectionEnabled: false, pageLoadStrategy: eager, platformName: linux, proxy: {}, se:bidiEnabled: false, se:cdp: ws://172.30.4.67:4444/sessi..., se:cdpVersion: 127.0.6533.72, se:vnc: ws://172.30.4.67:4444/sessi..., se:vncEnabled: true, se:vncLocalAddress: ws://172.18.0.16:7900, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: accept, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true}
13:15:32.698 INFO [GridModel.release] - Releasing slot for session id 030bb33f2b4520f77b9a09c085b8e46f
13:15:32.698 INFO [LocalSessionMap.lambda$new$0] - Deleted session from local Session Map, Id: 030bb33f2b4520f77b9a09c085b8e46f
13:15:32.759 INFO [LocalDistributor.newSession] - Session request received by the Distributor: 
 [Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=<-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: eager, unhandledPromptBehavior: accept}]
13:15:34.300 INFO [LocalDistributor.newSession] - Session created by the Distributor. Id: cfc8d3ee00b9149909c19243817d69d5 
 Caps: Capabilities {acceptInsecureCerts: true, browserName: chrome, browserVersion: 127.0.6533.72, chrome: {chromedriverVersion: 127.0.6533.72 (9755e24ca85a..., userDataDir: /tmp/.org.chromium.Chromium...}, fedcm:accounts: true, goog:chromeOptions: {debuggerAddress: localhost:43473}, networkConnectionEnabled: false, pageLoadStrategy: eager, platformName: linux, proxy: {}, se:bidiEnabled: false, se:cdp: ws://172.30.4.67:4444/sessi..., se:cdpVersion: 127.0.6533.72, se:vnc: ws://172.30.4.67:4444/sessi..., se:vncEnabled: true, se:vncLocalAddress: ws://172.18.0.13:7900, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: accept, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true}
13:15:59.677 INFO [LocalSessionMap.lambda$new$0] - Deleted session from local Session Map, Id: cfc8d3ee00b9149909c19243817d69d5
13:15:59.677 INFO [GridModel.release] - Releasing slot for session id cfc8d3ee00b9149909c19243817d69d5
13:15:59.922 INFO [LocalDistributor.newSession] - Session request received by the Distributor: 
 [Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=<-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: eager, unhandledPromptBehavior: accept}]
13:16:23.700 INFO [LocalDistributor.newSession] - Session request received by the Distributor: 
 [Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=<-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: eager, se:downloadsEnabled: false, unhandledPromptBehavior: accept}]
13:16:57.771 INFO [LocalDistributor.newSession] - Session created by the Distributor. Id: a4277b3e4668ec24a5f32e77d1b073b2 
 Caps: Capabilities {acceptInsecureCerts: true, browserName: chrome, browserVersion: 127.0.6533.72, chrome: {chromedriverVersion: 127.0.6533.72 (9755e24ca85a..., userDataDir: /tmp/.org.chromium.Chromium...}, fedcm:accounts: true, goog:chromeOptions: {debuggerAddress: localhost:36559}, networkConnectionEnabled: false, pageLoadStrategy: eager, platformName: linux, proxy: {}, se:bidiEnabled: false, se:cdp: ws://172.30.4.67:4444/sessi..., se:cdpVersion: 127.0.6533.72, se:vnc: ws://172.30.4.67:4444/sessi..., se:vncEnabled: true, se:vncLocalAddress: ws://172.18.0.10:7900, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: accept, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true}
13:18:15.592 INFO [LocalDistributor.newSession] - Session request received by the Distributor: 
 [Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=<-loopb..., --disable-dev-shm-usage, --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: eager, se:downloadsEnabled: false, unhandledPromptBehavior: accept}]
/opt/bin/start-selenium-grid-hub.sh: line 124:    10 Killed                  java ${JAVA_OPTS:-$SE_JAVA_OPTS} -jar /opt/selenium/selenium-server.jar ${EXTRA_LIBS} hub --session-request-timeout ${SE_SESSION_REQUEST_TIMEOUT} --session-retry-interval ${SE_SESSION_RETRY_INTERVAL} --healthcheck-interval ${SE_HEALTHCHECK_INTERVAL} --relax-checks ${SE_RELAX_CHECKS} --bind-host ${SE_BIND_HOST} --config /opt/selenium/config.toml ${HOST_CONFIG} ${PORT_CONFIG} ${SUB_PATH_CONFIG} ${SE_OPTS}
2024-08-06 13:31:38,980 WARN exited: selenium-grid-hub (exit status 137; not expected)`

joerg1985 commented 2 weeks ago

@IlyaUk @EvgenyMarchuk i have done some fixed related releasing the websocket in https://github.com/SeleniumHQ/selenium/commit/ed3edee0ac4f09a5555aec7c7ab20609d8b394f2, https://github.com/SeleniumHQ/selenium/commit/5bac4795ff6acc9f1ca1a6436aea0970ff98fb07 and https://github.com/SeleniumHQ/selenium/commit/7612405e34d282992b26a22d7bee921753020026.

I hope the leak is gone now, could you check this?

IlyaUk commented 1 week ago

Hi @joerg1985. Unfortunately the above fixes didn't help. Relevant logs output is below:

19:21:11.496 INFO [LocalDistributor.newSession] - Session request received by the Distributor: 
 [Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=<-loopb..., --disable-dev-shm-usage, --disable-search-engine-cho..., --unsafely-disable-devtools..., --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {autofill.credit_card_enabled: false, autofill.profile_enabled: false, credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: eager, se:downloadsEnabled: false, unhandledPromptBehavior: accept}]
19:22:43.911 INFO [LocalDistributor.newSession] - Session request received by the Distributor: 
 [Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=<-loopb..., --disable-dev-shm-usage, --disable-search-engine-cho..., --unsafely-disable-devtools..., --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {autofill.credit_card_enabled: false, autofill.profile_enabled: false, credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: eager, se:downloadsEnabled: false, unhandledPromptBehavior: accept}]
19:23:18.028 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "04d7720543da77be538cb871ceb71dd9","eventTime": 1724786471496188585,"eventName": "Session request received by the Distributor","attributes": {"logger": "org.openqa.selenium.grid.distributor.local.LocalDistributor","request.payload": "[Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=\u003c-loopb..., --disable-dev-shm-usage, --disable-search-engine-cho..., --unsafely-disable-devtools..., --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {autofill.credit_card_enabled: false, autofill.profile_enabled: false, credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: eager, se:downloadsEnabled: false, unhandledPromptBehavior: accept}]"}}

19:23:18.028 WARN [SeleniumSpanExporter$1.lambda$export$1] - org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created
from chrome not reachable 
Host info: host: '8c61b151e7d6', ip: '172.19.0.16'
Build info: version: '4.24.0-SNAPSHOT', revision: '7c8727b'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.12'
Driver info: driver.version: unknown
Build info: version: '4.24.0-SNAPSHOT', revision: '7c8727b'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.12'
Driver info: driver.version: unknown
Build info: version: '4.24.0-SNAPSHOT', revision: '7c8727b'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.12'
Driver info: driver.version: unknown
    at org.openqa.selenium.grid.node.remote.RemoteNode.newSession(RemoteNode.java:157)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession(LocalDistributor.java:654)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession(LocalDistributor.java:573)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest(LocalDistributor.java:836)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1(LocalDistributor.java:793)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:840)

19:23:18.028 WARN [SeleniumSpanExporter$1.lambda$export$1] - Unable to create session: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created
from chrome not reachable 
Host info: host: '8c61b151e7d6', ip: '172.19.0.16'
Build info: version: '4.24.0-SNAPSHOT', revision: '7c8727b'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.12'
Driver info: driver.version: unknown
Build info: version: '4.24.0-SNAPSHOT', revision: '7c8727b'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.12'
Driver info: driver.version: unknown
Build info: version: '4.24.0-SNAPSHOT', revision: '7c8727b'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.12'
Driver info: driver.version: unknown
19:23:18.028 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "04d7720543da77be538cb871ceb71dd9","eventTime": 1724786597988818214,"eventName": "exception","attributes": {"exception.message": "Unable to create session: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created\nfrom chrome not reachable \nHost info: host: '8c61b151e7d6', ip: '172.19.0.16'\nBuild info: version: '4.24.0-SNAPSHOT', revision: '7c8727b'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.12'\nDriver info: driver.version: unknown\nBuild info: version: '4.24.0-SNAPSHOT', revision: '7c8727b'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.12'\nDriver info: driver.version: unknown\nBuild info: version: '4.24.0-SNAPSHOT', revision: '7c8727b'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.12'\nDriver info: driver.version: unknown","exception.stacktrace": "org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created\nfrom chrome not reachable \nHost info: host: '8c61b151e7d6', ip: '172.19.0.16'\nBuild info: version: '4.24.0-SNAPSHOT', revision: '7c8727b'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.12'\nDriver info: driver.version: unknown\nBuild info: version: '4.24.0-SNAPSHOT', revision: '7c8727b'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.12'\nDriver info: driver.version: unknown\nBuild info: version: '4.24.0-SNAPSHOT', revision: '7c8727b'\nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1062-aws', java.version: '17.0.12'\nDriver info: driver.version: unknown\n\tat org.openqa.selenium.grid.node.remote.RemoteNode.newSession(RemoteNode.java:157)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession(LocalDistributor.java:654)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession(LocalDistributor.java:573)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest(LocalDistributor.java:836)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1(LocalDistributor.java:793)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)\n\tat java.base\u002fjava.lang.Thread.run(Thread.java:840)\n","exception.type": "org.openqa.selenium.SessionNotCreatedException","logger": "org.openqa.selenium.grid.distributor.local.LocalDistributor","request.payload": "[Capabilities {acceptInsecureCerts: true, browserName: chrome, goog:chromeOptions: {args: [--headless=new, --proxy-bypass-list=\u003c-loopb..., --disable-dev-shm-usage, --disable-search-engine-cho..., --unsafely-disable-devtools..., --disable-background-networ..., --enable-features=NetworkSe..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --window-size=1600,900], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {autofill.credit_card_enabled: false, autofill.profile_enabled: false, credentials_enable_service: false, plugins.always_open_pdf_externally: true, profile.default_content_setting_values.automatic_downloads: 1, profile.password_manager_enabled: false, profile.password_manager_leak_detection: false, safebrowsing.enabled: true}}, pageLoadStrategy: eager, se:downloadsEnabled: false, unhandledPromptBehavior: accept}]"}}

19:23:18.244 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "04d7720543da77be538cb871ceb71dd9","eventTime": 1724786598226963867,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "2185","http.scheme": "HTTP","http.status_code": 500,"http.target": "\u002fsession","http.user_agent": "selenium\u002f4.23.1 (java unix)"}}

/opt/bin/start-selenium-grid-hub.sh: line 124:    10 Killed                  java ${JAVA_OPTS:-$SE_JAVA_OPTS} -jar /opt/selenium/selenium-server.jar ${EXTRA_LIBS} hub --session-request-timeout ${SE_SESSION_REQUEST_TIMEOUT} --session-retry-interval ${SE_SESSION_RETRY_INTERVAL} --healthcheck-interval ${SE_HEALTHCHECK_INTERVAL} --relax-checks ${SE_RELAX_CHECKS} --bind-host ${SE_BIND_HOST} --config /opt/selenium/config.toml ${HOST_CONFIG} ${PORT_CONFIG} ${SUB_PATH_CONFIG} ${SE_OPTS}
2024-08-27 19:33:50,677 WARN exited: selenium-grid-hub (exit status 137; not expected)

diemol commented 1 week ago

But those fixes have not been released, are only available in the Nightly version.

IlyaUk commented 1 week ago

But those fixes have not been released, are only available in the Nightly version.

Sure, I tried to reproduce the issue using the nightly builds. Here is my docker compose file:

version: "3"
services:
  hub:
    image: selenium/hub:nightly
    container_name: hub
    ports:
      - "4444:4444"
      - "4443:4443"
      - "4442:4442"
    healthcheck:
      test: "/opt/bin/check-grid.sh --host 0.0.0.0 --port 4444"
      interval: "15s"
      timeout: "30s"
      retries: 5
    restart: always

  chrome:
    image: selenium/node-chrome:nightly
    shm_size: 2gb
    depends_on:
      - hub
    environment:
      - SE_EVENT_BUS_HOST=hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_VNC_NO_PASSWORD=1
      - SE_SCREEN_WIDTH=1600           
      - SE_SCREEN_HEIGHT=900             
      - SE_NODE_MAX_SESSIONS=1                         
      - SE_NODE_MAX_INSTANCES=1                                 
      - SE_NODE_GRID_URL=http://localhost:4444
      - SE_OPTS=--enable-managed-downloads true
    restart: always
    deploy:
      mode: replicated
      replicas: 16

joerg1985 commented 1 week ago

@IlyaUk from todays log the revision is 7c8727b, this revision is the same you tested 3 weeks ago. You might have used a cached version of the selenium/hub:nightly image?

IlyaUk commented 1 week ago

@IlyaUk from todays log the revision is 7c8727b, this revision is the same you tested 3 weeks ago. You might have used a cached version of the selenium/hub:nightly image?

May I ask you please to provide the expected revision number so I can double check the issue thorougly?

joerg1985 commented 1 week ago

The current nightly is https://github.com/SeleniumHQ/selenium/commit/58ca1331ab8671090f2cdb05c17a9af54841523e

Or you could try 4.24.0, this release is brand new and contains the fixes.

IlyaUk commented 4 days ago

Thanks a lot for the help @joerg1985. The original issue has been fixed and not reproducible anymore.

joerg1985 commented 4 days ago

@IlyaUk thanks for reporting and testing the fix.

SeleniumHQ / docker-selenium