[🐛 Bug]: Selenium4 unable to connect to RemoteWebDriver with proxy

What happened?

Selenium4 unable to connect to RemoteWebDriver with proxy

How can we reproduce the issue?

URL browserStackHubURL = new URL("<BS_URL>");
            DesiredCapabilities capabilities = new DesiredCapabilities();
            capabilities.setCapability("browserName", "chrome");
            HashMap<String, Object> browserstackOptions = new HashMap<String, Object>();
            browserstackOptions.put("os", "Windows");
            browserstackOptions.put("osVersion", "10");
            browserstackOptions.put("buildName", "BStack-[Java] Selenium 4 Sample Test");
            browserstackOptions.put("sessionName", "Selenium 4 test");
            browserstackOptions.put("seleniumVersion", "4.0.0");
            capabilities.setCapability("bstack:options", browserstackOptions);
            InetSocketAddress inetSocketAddress = new InetSocketAddress(Constants.BS_PROXY_HOST, Constants.BS_PROXY_PORT);
            System.out.println("Debug Proxy :: ClientConfig");
            ClientConfig config = ClientConfig.defaultConfig()
                    .baseUrl(browserStackHubURL)
                    .readTimeout(Duration.ofSeconds(60))
                    .proxy(new Proxy(HTTP, inetSocketAddress));
            HttpClient.Factory httpClientFactory = HttpClient.Factory.createDefault();
            httpClientFactory.createClient(config);
            HttpCommandExecutor executor = new HttpCommandExecutor(new HashMap<>(), config, httpClientFactory);
             Webdriver webdDriver = new RemoteWebDriver(executor, capabilities);

Relevant log output

-Dio.netty.leakDetection.level: simple
-Dio.netty.leakDetection.targetRecords: 4
Loaded default ResourceLeakDetector: io.netty.util.ResourceLeakDetector@47a86fbb
-Dio.netty.noUnsafe: false
Java version: 11
sun.misc.Unsafe.theUnsafe: available
sun.misc.Unsafe.copyMemory: available
java.nio.Buffer.address: available
direct buffer constructor: unavailable: Reflective setAccessible(true) disabled
java.nio.Bits.unaligned: available, true
jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable: class io.netty.util.internal.PlatformDependent0$6 cannot access class jdk.internal.misc.Unsafe (in module java.base) because module java.base does not export jdk.internal.misc to unnamed module @442d9b6e
java.nio.DirectByteBuffer.<init>(long, int): unavailable
sun.misc.Unsafe: available
maxDirectMemory: 16844324864 bytes (maybe)
-Dio.netty.tmpdir: /tmp (java.io.tmpdir)
-Dio.netty.bitMode: 64 (sun.arch.data.model)
-Dio.netty.maxDirectMemory: -1 bytes
-Dio.netty.uninitializedArrayAllocationThreshold: -1
java.nio.ByteBuffer.cleaner(): available
-Dio.netty.noPreferDirect: false
org.jctools-core.MpscChunkedArrayQueue: available
-Dio.netty.threadLocalMap.stringBuilder.initialSize: 1024
-Dio.netty.threadLocalMap.stringBuilder.maxSize: 4096
Default protocols (JDK): [TLSv1.3, TLSv1.2, TLSv1.1, TLSv1]
Default cipher suites (JDK): [TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384]
-Dio.netty.eventLoopThreads: 16
-Dio.netty.noKeySetOptimization: false
-Dio.netty.selectorAutoRebuildThreshold: 512
-Dio.netty.allocator.numHeapArenas: 16
-Dio.netty.allocator.numDirectArenas: 16
-Dio.netty.allocator.pageSize: 8192
-Dio.netty.allocator.maxOrder: 11
-Dio.netty.allocator.chunkSize: 16777216
-Dio.netty.allocator.smallCacheSize: 256
-Dio.netty.allocator.normalCacheSize: 64
-Dio.netty.allocator.maxCachedBufferCapacity: 32768
-Dio.netty.allocator.cacheTrimInterval: 8192
-Dio.netty.allocator.cacheTrimIntervalMillis: 0
-Dio.netty.allocator.useCacheForAllThreads: true
-Dio.netty.allocator.maxCachedByteBuffersPerChunk: 1023
-Dio.netty.allocator.type: pooled
-Dio.netty.threadLocalDirectBufferSize: 0
-Dio.netty.maxThreadLocalCharBufferSize: 16384
Build HttpCommandExecutor with proxy
Stsrting remote webdriver
-Dio.netty.processId: 1612 (auto-detected)
-Djava.net.preferIPv4Stack: false
-Djava.net.preferIPv6Addresses: false
Loopback interface: lo (lo, 0:0:0:0:0:0:0:1%lo)
/proc/sys/net/core/somaxconn: 128
-Dio.netty.machineId: 00:50:56:ff:fe:8f:65:96 (auto-detected)
-Dio.netty.buffer.checkAccessible: true
-Dio.netty.buffer.checkBounds: true
Loaded default ResourceLeakDetector: io.netty.util.ResourceLeakDetector@44e93b08
Trying to recover from failing to connect channel [id: 0xc9468f46] with a retry value of true
Failed to recover from connect exception: io.netty.channel.ConnectTimeoutException: connection timed out: hub.browserstack.com/207.254.8.9:443 with channel [id: 0xc9468f46]
connection timed out: hub.browserstack.com/207.254.8.9:443
Trying to recover from failing to connect channel [id: 0x2c222e25] with a retry value of true
Failed to recover from connect exception: io.netty.channel.ConnectTimeoutException: connection timed out: hub.browserstack.com/207.254.8.9:443 with channel [id: 0x2c222e25]
connection timed out: hub.browserstack.com/207.254.8.9:443
Trying to recover from failing to connect channel [id: 0x76b6edb6] with a retry value of true

Keep retrying for 20 more time.. Then
Caused by: java.io.UncheckedIOException: java.net.ConnectException: connection timed out: hub.browserstack.com/207.254.8.9:443
        at org.openqa.selenium.remote.http.netty.NettyHttpHandler.makeCall(NettyHttpHandler.java:80)
        at org.openqa.selenium.remote.http.RetryRequest.lambda$apply$6(RetryRequest.java:80)
        at net.jodah.failsafe.Functions.lambda$get$0(Functions.java:48)
Caused by: java.net.ConnectException: connection timed out: hub.browserstack.com/207.254.8.9:443
        at org.asynchttpclient.netty.channel.NettyConnectListener.onFailure(NettyConnectListener.java:179)
        at org.asynchttpclient.netty.channel.NettyChannelConnector$1.onFailure(NettyChannelConnector.java:108)
Caused by: io.netty.channel.ConnectTimeoutException: connection timed out: hub.browserstack.com/207.254.8.9:443
        at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:261)
        ... 9 more

Operating System

macos

Selenium version

4.1.1

What are the browser(s) and version(s) where you see this issue?

chrome

What are the browser driver(s) and version(s) where you see this issue?

any version

Are you using Selenium Grid?

No response

@sumanguduru, thank you for creating this issue. We will troubleshoot it as soon as we can.

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

And i tried with RemoteWebDriver.builder() and getting different Driver info: driver.version: unknown****

URL browserStackHubURL = new URL("https://:<>key@hub.browserstack.com/wd/hub"); DesiredCapabilities capabilities = new DesiredCapabilities(); capabilities.setCapability("browserName", "chrome"); HashMap<String, Object> browserstackOptions = new HashMap<>(); browserstackOptions.put("os", "Windows"); browserstackOptions.put("osVersion", "10"); browserstackOptions.put("buildName", "BStack-[Java] Selenium 4 Sample Test"); browserstackOptions.put("sessionName", "Selenium 4 test"); browserstackOptions.put("seleniumVersion", "4.0.0"); capabilities.setCapability("bstack:options", browserstackOptions); capabilities.setCapability(CapabilityType.BROWSER_NAME, "Chrome"); capabilities.setCapability(CapabilityType.BROWSER_VERSION, "97");

    InetSocketAddress inetSocketAddress = new InetSocketAddress(Constants.BS_PROXY_HOST, Constants.BS_PROXY_PORT);

    ClientConfig config = ClientConfig.defaultConfig()
            .readTimeout(Duration.ofSeconds(60))
            .baseUrl(browserStackHubURL)
            .proxy(new Proxy(HTTP, inetSocketAddress));

    WebDriver driver = RemoteWebDriver.builder()
            .oneOf(capabilities)
            .config(config)
            .build();

org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Response code 500. Message: java.net.ConnectException: connection timed out: hub.browserstack.com/207.254.8.8:443 Build info: version: '4.1.1', revision: 'e8fcc2cecf' System info: host: 'myhost', ip: 'myip', os.name: 'Linux', os.arch: 'amd64', os.version: '3.10.0-1160.49.1.el7.x86_64', java.version: '11.0.1' Driver info: driver.version: unknown at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:126) at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:84) at org.openqa.selenium.remote.RemoteWebDriverBuilder.getRemoteDriver(RemoteWebDriverBuilder.java:387) at org.openqa.selenium.remote.RemoteWebDriverBuilder.build(RemoteWebDriverBuilder.java:360) at com.mastercard.commercial.track.testrunners.RunCucumber.beforeClass(RunCucumber.java:86) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:410) at org.apache.maven.surefire.junitcore.pc.InvokerStrategy.schedule(InvokerStrategy.java:54) at org.apache.maven.surefire.junitcore.pc.Scheduler.schedule(Scheduler.java:367) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:410) at org.apache.maven.surefire.junitcore.pc.InvokerStrategy.schedule(InvokerStrategy.java:54) at org.apache.maven.surefire.junitcore.pc.Scheduler.schedule(Scheduler.java:367) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.apache.maven.surefire.junitcore.pc.ParallelComputerBuilder$PC$1.run(ParallelComputerBuilder.java:590) at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeLazy(JUnitCoreWrapper.java:119) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:87) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75) at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:158) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)

However if i set system proxies before the above code snippets its connecting to remoteWebdriver without any issues, But i cant use useSystemProxies because it's blocking some of the API calls i'm making inside the test suite.

   System.setProperty("java.net.useSystemProxies", "true");
    System.setProperty("http.proxyHost", Constants.BS_PROXY_HOST);
    System.setProperty("http.proxyPort", String.valueOf(Constants.BS_PROXY_PORT));

I think you want to set the proxy in the options class not in the service class?

The service class affects how wire calls are made, the options class affects how assets are loaded into the browser. Which one are you interested in?

@titusfortner , I am not sure about exact difference between options and service class, currently im not able to connect to remote webdriver through outbound proxy, i was using selenium3.1.4 and was able to connect to remotewebdriver through HttpCommandExecutor by passing HTTPClient.Factory with proxy host and port, and now im trying to update selenium to 4.12 and above code snippet i am using to connect to remote webdriver. its not connecting

The below code snippet is from NettyClient.java (selenium)


.setNettyTimer(TIMER)
.setRequestTimeout(toClampedInt(config.readTimeout().toMillis()))
.setConnectTimeout(toClampedInt(config.connectionTimeout().toMillis()))
.setReadTimeout(toClampedInt(config.readTimeout().toMillis()));
.setReadTimeout(toClampedInt(config.readTimeout().toMillis()))
.setUseProxyProperties(true)
.setUseProxySelector(true);

**I think its missing  proxy server details which user is passing through Client config. but im not sure though**

` .setProxyServer(new ProxyServer.Builder("config.proxy().address()",11).build())`

This is a duplicate of #10231

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

SeleniumHQ / selenium