search

SeleniumHQ / selenium

A browser automation framework and ecosystem.
https://selenium.dev
Apache License 2.0
30.8k stars 8.2k forks source link

[🐛 Bug]: DefaultSlotMatcher is considering extension capabilities to determine whether the slot matches the requested capabilities #14461

Open sbabcoc opened 2 months ago

sbabcoc commented 2 months ago

What happened?

My Selenium framework includes the ability to manage a local Grid instance, including Appium nodes. Because the Appium server needs an execution context to function, I have the ability to run the server via PM2. When this configuration is employed, the shutdown procedure is different. Consequently, I need to include a property in the capabilities object that indicates this configuration. This is done via an extension capability names nord:options. The expectation is that this capability will be available to query, but won't affect slot matching. Unfortunately, this isn't currently the case. Adding this extension capability causes the corresponding node to no longer match Appium session requests.

The scenario that caused my immediate issue is that both the node capabilities and the new session request define values for nord:options, each specifying a different informational property. The slot matcher compares the values of these capabilities and rejects the slot because the objects differ.

How can we reproduce the issue?

Create a grid with a node whose capabilities include a `nord:options` extension capability. Try to acquire a session from this node with capabilities that specify a different value for `nord:options`.

EXPECT: The slot matcher should provide a session
ACTUAL: No matching session is provided

Relevant log output

FAILED: com.nordstrom.automation.selenium.android.AndroidTest.testSearchActivity
        {
  "constName": "ANDROID",
  "description": "",
  "className": "com.nordstrom.automation.selenium.platform.TargetType",
  "class": "com.nordstrom.automation.selenium.listeners.PlatformInterceptor$PlatformIdentity"
}
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
    at org.openqa.selenium.support.ui.FluentWait.propagateIfNotIgnored(FluentWait.java:252)
    at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:213)
    at com.nordstrom.automation.selenium.core.DriverManager.injectDriver(DriverManager.java:169)
    at com.nordstrom.automation.selenium.core.DriverManager.beforeInvocation(DriverManager.java:105)
    at com.nordstrom.automation.selenium.listeners.DriverListener.beforeInvocation(DriverListener.java:55)
    at com.nordstrom.automation.testng.AbstractListenerChain.beforeInvocation(AbstractListenerChain.java:231)
    at org.testng.internal.invokers.InvokedMethodListenerInvoker.invokeListener(InvokedMethodListenerInvoker.java:56)
    at org.testng.internal.invokers.BaseInvoker.runInvokedMethodListeners(BaseInvoker.java:67)
    at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:642)
    at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:221)
    at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:50)
    at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:969)
    at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:194)
    at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:148)
    at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
    at org.testng.TestRunner.privateRun(TestRunner.java:829)
    at org.testng.TestRunner.run(TestRunner.java:602)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:437)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:431)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:391)
    at org.testng.SuiteRunner.run(SuiteRunner.java:330)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1256)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1176)
    at org.testng.TestNG.runSuites(TestNG.java:1099)
    at org.testng.TestNG.run(TestNG.java:1067)
    at org.testng.remote.AbstractRemoteTestNG.run(AbstractRemoteTestNG.java:115)
    at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:293)
    at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:91)
Caused by: java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:74)
    at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486)
    at com.nordstrom.automation.selenium.core.GridUtility.getDriver(GridUtility.java:166)
    at com.nordstrom.automation.selenium.core.GridUtility.getDriver(GridUtility.java:128)
    at com.nordstrom.automation.selenium.core.DriverManager$1.apply(DriverManager.java:284)
    at com.nordstrom.automation.selenium.core.DriverManager$1.apply(DriverManager.java:1)
    at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:203)
    ... 29 more
Caused by: org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Response code 500. Message: Could not start a new session. New session request timed out 
Host info: host: 'DESKTOP-21IB3RG', ip: '192.168.254.20'
Build info: version: '4.23.0', revision: '4df0a231af'
System info: os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '21.0.4'
Driver info: driver.version: unknown
Build info: version: '4.23.0', revision: '4df0a231af'
System info: os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '21.0.4'
Driver info: io.appium.java_client.android.AndroidDriver
Command: [null, newSession {capabilities=[Capabilities {appium:app: https://github.com/appium/a..., appium:automationName: Espresso, appium:forceEspressoRebuild: true, appium:host: 0.0.0.0, appium:showGradleLog: true, nord:options: {personality: Espresso}, platformName: ANDROID}]}]
Capabilities {appium:app: https://github.com/appium/a..., appium:automationName: Espresso, appium:forceEspressoRebuild: true, appium:host: 0.0.0.0, appium:showGradleLog: true, nord:options: {personality: Espresso}, platformName: ANDROID}
    at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:114)
    at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:75)
    at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:61)
    at io.appium.java_client.remote.AppiumCommandExecutor.createSession(AppiumCommandExecutor.java:176)
    at io.appium.java_client.remote.AppiumCommandExecutor.execute(AppiumCommandExecutor.java:237)
    at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:527)
    at io.appium.java_client.AppiumDriver.startSession(AppiumDriver.java:270)
    at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:163)
    at io.appium.java_client.AppiumDriver.<init>(AppiumDriver.java:91)
    at io.appium.java_client.AppiumDriver.<init>(AppiumDriver.java:103)
    at io.appium.java_client.android.AndroidDriver.<init>(AndroidDriver.java:109)
    at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
    ... 36 more

Operating System

Windows 10

Selenium version

4.23.0

What are the browser(s) and version(s) where you see this issue?

All

What are the browser driver(s) and version(s) where you see this issue?

AndroidDriver (RemoteWebDriver)

Are you using Selenium Grid?

4.23.0

github-actions[bot] commented 2 months ago

@sbabcoc, thank you for creating this issue. We will troubleshoot it as soon as we can.

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

sbabcoc commented 2 months ago

The wrinkle here is that we now have Appium extension capabilities which are being matched through this improper behavior. Fixing this is going to require either adding explicit matching for Appium capabilities in DefaultNodeMatcher or requiring a specific Appium node matcher.

sbabcoc commented 2 months ago

There was an issue filed about one year ago that seems related to what I'm seeing: #12640

sbabcoc commented 2 months ago

Also #14216 The Appium-specific logic added to RelaySessionFactory is a direct result of incorporating too much processing on new session requests. Decisions about how to handle unspecified values should be made by the consumer, not applied by generic transport-layer implementation.

sbabcoc commented 2 months ago

The W3C specification confirms my instincts. https://www.w3.org/TR/webdriver1/#new-session

... an intermediary node cannot forward extension capabilities specific to that implementation to an endpoint node...

The spec strongly recommends that vendor-specific features be passed via top-level parameters, not via requested capabilities. Regardless, these sorts of vendor-specific features should be processed by vendor-specific slot matchers. The default slot matcher should not include handling of vendor-specific features.