search

HtmlUnit / htmlunit

HtmlUnit is a "GUI-Less browser for Java programs".
https://www.htmlunit.org
Apache License 2.0
878 stars 172 forks source link

Timeout when I try to use PROXY #897

Open alexvrv opened 2 weeks ago

alexvrv commented 2 weeks ago

Hi, I'm trying to use a proxy server but I only get "org.apache.http.conn.ConnectTimeoutException". This is the code I'm using. I've tried 3 methods for setting the PROXY server xD

String url = "https://forexe.mfinante.gov.ro/CABWeb/home";
            final WebClient webClient = new WebClient(BrowserVersion.CHROME, "x.xx.xxx.xxx", 8080);

//            ProxyConfig proxyConfig = new ProxyConfig("x.xx.xxx.xxx", 8080, "https");
//            webClient.getOptions().setProxyConfig(proxyConfig);

//            webClient.getOptions().getProxyConfig().setProxyHost("x.xx.xxx.xxx");
//            webClient.getOptions().getProxyConfig().setProxyPort(8080);
//            webClient.getOptions().getProxyConfig().setSocksProxy(false);

            webClient.getOptions().setSSLClientCertificateKeyStore(ks.getKeyStore(), null);
            webClient.getOptions().setSSLInsecureProtocol("TLS");
//            webClient.getOptions().setUseInsecureSSL(true);
            webClient.getCookieManager().setCookiesEnabled(true);
            webClient.setCssErrorHandler(new SilentCssErrorHandler());
            webClient.getOptions().setThrowExceptionOnScriptError(false);
            webClient.setIncorrectnessListener((message, origin) -> {});
            webClient.getOptions().setSSLContext(ks.getSSLContext(alias));

           HtmlPage htmlpage = webClient.getPage(url);

and this is the error

org.apache.http.conn.ConnectTimeoutException: Connect to x.xx.xxx.xxx:8080 [/x.xx.xxx.xxx] failed: Connect timed out
        at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151)
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)
        at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:401)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
        at org.htmlunit.HttpWebConnection.getResponse(HttpWebConnection.java:191)
        at org.htmlunit.WebClient.getWebResponseOrUseCached(WebClient.java:1726)

        at org.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1631)
        at org.htmlunit.WebClient.loadWebResponse(WebClient.java:1556)
        at org.htmlunit.WebClient.getPage(WebClient.java:485)
        at org.htmlunit.WebClient.getPage(WebClient.java:402)
        at org.htmlunit.WebClient.getPage(WebClient.java:538)
        at org.htmlunit.WebClient.getPage(WebClient.java:520)
        at CABApp2.onOK(CABApp2.java:106)
        at CABApp2.lambda$new$0(CABApp2.java:36)
        at java.desktop/javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:1972)
        at java.desktop/javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2313)
        at java.desktop/javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:405)
        at java.desktop/javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:262)
        at java.desktop/javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:279)
        at java.desktop/java.awt.Component.processMouseEvent(Component.java:6626)
        at java.desktop/javax.swing.JComponent.processMouseEvent(JComponent.java:3389)
        at java.desktop/java.awt.Component.processEvent(Component.java:6391)
        at java.desktop/java.awt.Container.processEvent(Container.java:2266)
        at java.desktop/java.awt.Component.dispatchEventImpl(Component.java:5001)
        at java.desktop/java.awt.Container.dispatchEventImpl(Container.java:2324)
        at java.desktop/java.awt.Component.dispatchEvent(Component.java:4833)
        at java.desktop/java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4948)
        at java.desktop/java.awt.LightweightDispatcher.processMouseEvent(Container.java:4575)
        at java.desktop/java.awt.LightweightDispatcher.dispatchEvent(Container.java:4516)
        at java.desktop/java.awt.Container.dispatchEventImpl(Container.java:2310)
        at java.desktop/java.awt.Window.dispatchEventImpl(Window.java:2780)
        at java.desktop/java.awt.Component.dispatchEvent(Component.java:4833)
        at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:773)
        at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:722)
        at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:716)
        at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
        at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:86)
        at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:97)
        at java.desktop/java.awt.EventQueue$5.run(EventQueue.java:746)
        at java.desktop/java.awt.EventQueue$5.run(EventQueue.java:744)
        at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
        at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:86)
        at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:743)
        at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
        at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
        at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:117)
        at java.desktop/java.awt.WaitDispatchSupport$2.run(WaitDispatchSupport.java:191)
        at java.desktop/java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:318)
        at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:771)
        at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:722)
        at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:716)
        at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
        at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:86)
        at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:741)
        at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
        at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
        at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113)
        at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109)
        at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
        at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90)
Caused by: java.net.SocketTimeoutException: Connect timed out
        at java.base/java.net.SocksSocketImpl.readSocksReply(SocksSocketImpl.java:125)
        at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:390)
        at java.base/java.net.Socket.connect(Socket.java:633)
        at org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:75)
        at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
        ... 64 more

Am I missing something? ...

rbri commented 2 weeks ago

@alexvrv i use the final WebClient webClient = new WebClient(BrowserVersion.CHROME, "x.xx.xxx.xxx", 8080); approach from time to time to do some wire inspection using the Charles Web Proxy as local running proxy. Had no problems so far.

Does your proxy requires authentication?

alexvrv commented 2 weeks ago

No, it's without authentification. Just an IP and a PORT. JSOUP and HttpURLConnection works fine with just "System.setProperty("java.net.useSystemProxies", "true");". They take the Proxy settings from Windows.

rbri commented 1 week ago

Wild guess - does the proxy support http or only http/2?

alexvrv commented 1 week ago

Uff I really don't know xD but it works with every website (even old ones, government ones) so I guess it supports both.

rbri commented 5 days ago

maybe you can try to capture a wire log (see https://hc.apache.org/httpcomponents-client-5.4.x/logging.html)

Hopefully this might give a glue what is going on