Closed lszasz closed 1 year ago
https://github.com/SeleniumHQ/docker-selenium/issues/1608
session not created,
` System.InvalidOperationException: 'Could not start a new session. Could not start a new session. Timed out waiting for driver server to shutdown. Build info: version: '4.2.2', revision: '683ccb65d6' System info: host: '5d999618bcb0', ip: 'Unknown', os.name: 'Linux', os.arch: 'amd64', os.version: '5.4.0-120-generic', java.version: '11.0.15' Driver info: driver.version: unknown
`
Docker Selenium version:
selenium/node-chrome:4.2.1-20220531 and selenium/node-chrome:4.2.2-20220609
Thank you for reporting. I am able to reproduce, however I do not get the exact same stacktrace but the Grid is not usable after the NoSuchSessionException
is triggered.
Not sure how to solve this for now, as the initial server implementation has a few assumptions that make it hard to fix. I need to re-think it and see how we can fix this.
Hitting same issue in our test environment with max 60 sessions in our selenium hub. Any suggestion on recent stable version of the hub I could upgrade. So far we failed two attempt to upgrade to Selenium4 and sticking to Selenium3 due to session issues we had in 4.1.3 and 4.3.0. Thanks
Got the same error, seem the session is released by grid after 5 minutes inactive. happens with 4.3 latest version also
selenium_v4_docker_grid-node-docker-1 | 14:55:07.923 INFO [Container.stop] - Stopping container 506d4645a7016e38b31cb588235732750cd8d0dbfeedce8298e8c7216217f704 selenium_v4_docker_grid-node-docker-1 | 14:55:12.621 INFO [SessionSlot.stop] - Stopping session 081b1d9e13a96a2708a20608ffe63f06 selenium-hub | 14:55:12.629 INFO [GridModel.release] - Releasing slot for session id 081b1d9e13a96a2708a20608ffe63f06 selenium-hub | 14:55:12.628 INFO [LocalSessionMap.lambda$new$0] - Deleted session from local Session Map, Id: 081b1d9e13a96a2708a20608ffe63f06 selenium-hub | 14:55:38.204 WARN [SeleniumSpanExporter$1.lambda$export$1] - Unable to execute request for an existing session: Unable to find session with ID: 081b1d9e13a96a2708a20608ffe63f06 selenium-hub | Build info: version: '4.3.0', revision: 'a4995e2c09*' selenium-hub | System info: host: 'f9feb90f4553', ip: '172.22.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-41-generic', java.version: '11.0.15' selenium-hub | Driver info: driver.version: unknown selenium-hub | 14:55:38.206 WARN [SeleniumSpanExporter$1.lambda$export$1] - org.openqa.selenium.NoSuchSessionException: Unable to find session with ID: 081b1d9e13a96a2708a20608ffe63f06
This issue is also there on selenium 4.3.0 version, We did selenium gird setup without docker and still issue is there with below error,
Driver info: driver.version: unknown
org.openqa.selenium.json.JsonException: Expected to read a START_MAP but instead have: END. Last 0 characters read:
Build info: version: '4.3.0', revision: 'a49935e2c109*'
System info: host: 'abcdeftestrf', ip: '100.189.167.991', os.name: 'Windows', os.arch: 'amd64', os.version: '10.0', java.version: '1.8.0_271'
Driver info: driver.version: unknown
at org.openqa.selenium.json.JsonInput.expect(JsonInput.java:307)
at org.openqa.selenium.json.JsonInput.beginObject(JsonInput.java:229)
at org.openqa.selenium.remote.NewSessionPayload.getOss(NewSessionPayload.java:306)
at org.openqa.selenium.remote.NewSessionPayload.
@diemol We already set SE_NODE_SESSION_TIMEOUT=1800 but don't know why grid release/delete sessions which inactive 300s at this log
Releasing slot for session id b12420f9f4126c5a3ab5e26170a0d7a0 seleniumhub | 13:03:37.692 INFO [LocalSessionMap.lambda$new$0] - Deleted session from local Session Map
I'm having the same issue with only one session running interactions with the session ended. Does anyone know a solution? The grid version used to run the scripts is 4.5.0
. Below is the grid log and script to reproduce the bug.
20:34:12.907 INFO [Hub.execute] - Started Selenium Hub 4.5.0 (revision fe167b119a): http://172.18.0.2:4444
20:34:14.805 INFO [Node.<init>] - Binding additional locator mechanisms: name, id, relative
20:34:15.058 INFO [GridModel.setAvailability] - Switching Node 13ba1304-3bfb-4210-9c5c-17e01a3b31ff (uri: http://172.18.0.3:5555) from DOWN to UP
20:34:15.058 INFO [LocalDistributor.add] - Added node 13ba1304-3bfb-4210-9c5c-17e01a3b31ff at http://172.18.0.3:5555. Health check every 120s
20:34:27.585 INFO [LocalDistributor.newSession] - Session request received by the Distributor:
[Capabilities {browserName: chrome, goog:chromeOptions: {args: [--kiosk-printing, --no-sandbox, --disable-dev-shm-usage], extensions: [], prefs: {download.default_directory: /home/seluser/Downloads}}, pageLoadStrategy: normal}]
20:34:28.185 INFO [LocalDistributor.newSession] - Session created by the Distributor. Id: 1b14713841bb35cdcb0863772c314a76
Caps: Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 105.0.5195.52, chrome: {chromedriverVersion: 105.0.5195.52 (412c95e51883..., userDataDir: /tmp/.org.chromium.Chromium...}, goog:chromeOptions: {debuggerAddress: localhost:41641}, networkConnectionEnabled: false, pageLoadStrategy: normal, platformName: LINUX, proxy: {}, se:cdp: ws://172.18.0.3:4444/sessio..., se:cdpVersion: 105.0.5195.52, se:vnc: ws://172.18.0.3:4444/sessio..., se:vncEnabled: true, se:vncLocalAddress: ws://172.18.0.3:7900, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:virtualAuthenticators: true}
20:39:44.906 INFO [LocalSessionMap.lambda$new$0] - Deleted session from local Session Map, Id: 1b14713841bb35cdcb0863772c314a76
20:39:44.907 INFO [GridModel.release] - Releasing slot for session id 1b14713841bb35cdcb0863772c314a76
20:40:20.211 WARN [SeleniumSpanExporter$1.lambda$export$1] - Unable to execute request for an existing session: Unable to find session with ID: 1b14713841bb35cdcb0863772c314a76
Build info: version: '4.5.0', revision: 'fe167b119a'
System info: os.name: 'Linux', os.arch: 'aarch64', os.version: '5.10.124-linuxkit', java.version: '11.0.16'
Driver info: driver.version: unknown
20:40:20.212 WARN [SeleniumSpanExporter$1.lambda$export$1] - org.openqa.selenium.NoSuchSessionException: Unable to find session with ID: 1b14713841bb35cdcb0863772c314a76
Build info: version: '4.5.0', revision: 'fe167b119a'
System info: os.name: 'Linux', os.arch: 'aarch64', os.version: '5.10.124-linuxkit', java.version: '11.0.16'
Driver info: driver.version: unknown
at org.openqa.selenium.grid.sessionmap.local.LocalSessionMap.get(LocalSessionMap.java:129)
at org.openqa.selenium.grid.router.HandleSession.lambda$loadSessionId$4(HandleSession.java:159)
at io.opentelemetry.context.Context.lambda$wrap$2(Context.java:224)
at org.openqa.selenium.grid.router.HandleSession.execute(HandleSession.java:122)
at org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:373)
at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
at org.openqa.selenium.grid.router.Router.execute(Router.java:91)
at org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)
at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
at org.openqa.selenium.remote.http.Route$NestedRoute.handle(Route.java:270)
at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
at org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)
at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
at org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
20:40:20.214 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "bb1b1689ef4f58008cb4b9a5b0566e46","eventTime": 1666384820208521208,"eventName": "exception","attributes": {"exception.message": "Unable to execute request for an existing session: Unable to find session with ID: 1b14713841bb35cdcb0863772c314a76\nBuild info: version: '4.5.0', revision: 'fe167b119a'\nSystem info: os.name: 'Linux', os.arch: 'aarch64', os.version: '5.10.124-linuxkit', java.version: '11.0.16'\nDriver info: driver.version: unknown","exception.stacktrace": "org.openqa.selenium.NoSuchSessionException: Unable to find session with ID: 1b14713841bb35cdcb0863772c314a76\nBuild info: version: '4.5.0', revision: 'fe167b119a'\nSystem info: os.name: 'Linux', os.arch: 'aarch64', os.version: '5.10.124-linuxkit', java.version: '11.0.16'\nDriver info: driver.version: unknown\n\tat org.openqa.selenium.grid.sessionmap.local.LocalSessionMap.get(LocalSessionMap.java:129)\n\tat org.openqa.selenium.grid.router.HandleSession.lambda$loadSessionId$4(HandleSession.java:159)\n\tat io.opentelemetry.context.Context.lambda$wrap$2(Context.java:224)\n\tat org.openqa.selenium.grid.router.HandleSession.execute(HandleSession.java:122)\n\tat org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:373)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.grid.router.Router.execute(Router.java:91)\n\tat org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$NestedRoute.handle(Route.java:270)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)\n\tat org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)\n\tat java.base\u002fjava.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat java.base\u002fjava.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base\u002fjava.lang.Thread.run(Thread.java:829)\n","exception.type": "org.openqa.selenium.NoSuchSessionException","http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.router.HandleSession","http.host": "127.0.0.1:4444","http.method": "POST","http.request_content_length": "1191","http.scheme": "HTTP","http.target": "\u002fsession\u002f1b14713841bb35cdcb0863772c314a76\u002felement","http.user_agent": "selenium\u002f4.5.0 (python mac)","session.id": "1b14713841bb35cdcb0863772c314a76"}}
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.common.exceptions import InvalidSessionIdException
import time
configuration = {"browserName": "chrome"}
options = webdriver.ChromeOptions()
options.add_argument("--kiosk-printing")
options.add_argument("--no-sandbox")
options.add_argument("--disable-dev-shm-usage")
driver = webdriver.Remote(
command_executor="http://127.0.0.1:4444/wd/hub",
desired_capabilities=configuration,
options=options,
)
try:
driver.maximize_window() # Note: driver.maximize_window does not work on Linux selenium version v2, instead set window size and window position like driver.set_window_position(0,0) and driver.set_window_size(1920,1080)
driver.get("https://github.com/lszasz/selenium-test")
time.sleep(320)
for i in range(0, 5):
try:
driver.find_element(By.XPATH, "//*[contains(concat(' ', @class, ' '), ' x-component ') and contains(concat(' ', @class, ' '), ' x-fit-item ') and not(contains(@class, 'x-hidden-offsets'))]//*[contains(concat(' ', @class, ' '), ' x-panel ') and not(contains(@class, 'x-hidden-offsets')) and count(.//*[contains(concat(' ', @class, ' '), ' x-toolbar ') and count(.//h3[contains(concat(' ', @class, ' '), ' x-toolbar-item ') and contains(text(),'Bwb_simple_trapr25164550427.txt')]) > 0]) > 0]//*[contains(concat(' ', @class, ' '), ' x-tab-bar ') and count(.//a[contains(concat(' ', @class, ' '), ' x-tab-active ') and count(*//text()[.='Details']) > 0]) > 0]//following-sibling::*[contains(concat(' ', @class, ' '), ' x-panel-body ')]/*[contains(concat(' ', @class, ' '), ' x-tabpanel-child ') and not(contains(@class, 'x-hidden-offsets'))]//*[contains(concat(' ', @class, ' '), ' x-toolbar ')]//a[contains(concat(' ', @class, ' '), ' x-btn ') and (count(*//text()[contains(.,'Download Package')]) > 0 or count(*//text()[contains(.,'Download Source File')]) > 0 or count(*//text()[contains(.,'Download SDLXLIFF')]) > 0) and count(.//*[contains(concat(' ', @class, ' '), ' fa-download ')]) > 0]").click()
except Exception as e:
if type(e) == InvalidSessionIdException:
break
print(e)
finally:
driver.quit()
@diemol - There seems to be two issues that are mentioned in this defect.
4.5.3
Here's the docker compose file I used
version: "3"
services:
chrome:
image: selenium/node-chrome:4.5.3-20221024
shm_size: 2gb
depends_on:
- selenium-hub
environment:
- SE_NODE_SESSION_TIMEOUT=2
- SE_EVENT_BUS_HOST=selenium-hub
- SE_EVENT_BUS_PUBLISH_PORT=4442
- SE_EVENT_BUS_SUBSCRIBE_PORT=4443
selenium-hub:
image: selenium/hub:4.5.3-20221024
container_name: selenium-hub
ports:
- "4442:4442"
- "4443:4443"
- "4444:4444"
Here's a sample that I ran
import java.net.URL;
import java.util.concurrent.TimeUnit;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.remote.RemoteWebDriver;
public class SessionsExample {
public static void main(String[] args) throws Exception {
try {
System.err.println("Running First test with an induced timeout");
runTest(true);
} catch (Exception ignored) {
}
System.err.println("Running Second test without any timeouts");
runTest(false);
}
private static void runTest(boolean addTimeOut) throws Exception {
RemoteWebDriver driver = null;
try {
driver = new RemoteWebDriver(new URL("http://localhost:4444"), new ChromeOptions());
System.err.println("Session ID " + driver.getSessionId().toString());
driver.get("https://www.selenium.dev/");
if (addTimeOut) {
TimeUnit.SECONDS.sleep(45);
System.err.println("Slept for 45 seconds");
}
System.err.println("Title " + driver.getTitle());
} finally {
if (driver != null) {
driver.quit();
}
}
}
}
Expected to read a START_MAP but instead have: END
. This is also being tracked by https://github.com/SeleniumHQ/docker-selenium/issues/1507So if the first issue is no longer happening (atleast not for me on 4.5.3) then maybe we can close this off.
@lszasz @roderiano @SACTAN - Thoughts ?
This was an actual issue caused by the retries in the findElement
method. When the session timed out, the client tried to do a findElement
and the Grid replied with a 404 right away, but this did not allow the Grid to read the whole content from the request. This leaves an input pipe waiting for content and eventually causing a lock. That is what causes the Grid to hang and no process new requests (like serving GraphQL requests from the Grid UI).
The fix closes the input pipeline when a 404 is responded. It will be part of 4.6.0, which should come out in the next few days.
This was an actual issue caused by the retries in the
findElement
method. When the session timed out, the client tried to do afindElement
and the Grid replied with a 404 right away, but this did not allow the Grid to read the whole content from the request. This leaves an input pipe waiting for content and eventually causing a lock. That is what causes the Grid to hang and no process new requests (like serving GraphQL requests from the Grid UI).The fix closes the input pipeline when a 404 is responded. It will be part of 4.6.0, which should come out in the next few days.
Same happening to our Team. After a some hours, doing some selenium work, closing windows and quitting sessions, we face exactly this. But session seems to be created, but operating on it (setting a page_load_timeout) raises a 404, and then all is the same. Also happens with other operations, it's not necessarily linked to set_page_load_timeout. It seems the session is created but in a buggy way, and server cannot later on make use of it.
Updated finally from 3.x a few days ago to 4.6.0 and everything is running smooth. Thanks a lot @diemol!
This was an actual issue caused by the retries in the
findElement
method. When the session timed out, the client tried to do afindElement
and the Grid replied with a 404 right away, but this did not allow the Grid to read the whole content from the request. This leaves an input pipe waiting for content and eventually causing a lock. That is what causes the Grid to hang and no process new requests (like serving GraphQL requests from the Grid UI).The fix closes the input pipeline when a 404 is responded. It will be part of 4.6.0, which should come out in the next few days.
i am on 4.7 and it steel happening. element is, Exception: Timed out after 30 seconds ----> OpenQA.Selenium.WebDriverException : invalid session id TearDown : OpenQA.Selenium.WebDriverException : invalid session id and the strange thing is that this exception always happening on the same method
@lior277 - Can you please add some additional context?
@lior277 - Can you please add some additional context?
- Are you running against the Selenium Grid ? If yes what version ?
- Can you please help share a sample test case, that can be executed to reproduce the problem ?
If you are facing this issue, please create a new issue with a complete way to reproduce it, otherwise we cannot really have a look at it.
@diemol @krmahadevan I'm also facing this issue with latest version of selenium Grid 4.8.1 (revision 8ebccac989) It was coming in Selenium Grid 4.1.1 version but even after upgrading it to latest, I'm still finding the same issue. Is there any open ticket for this issue or any upcoming fixes in new release? Thanks
Im using a container (standalone-chrome latest ver.) and facing the same issue. Downgraded to 110.0-chromedriver-110.0-grid-4.8.0-20230210
and everything works fine.
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
What happened?
Steps to reproduce:
Actual: The hub crashes and from this point on will be unable to create new session
You can reproduce the issue using the test from https://github.com/lszasz/selenium-test
Moreover /status is not reflecting that there are problems with the grid.
Command used to start Selenium Grid with Docker
Relevant log output
Operating System
Centos 7
Docker Selenium version (tag)
selenium/hub:4.2.2