search

MachinePublishers / jBrowserDriver

A programmable, embeddable web browser driver compatible with the Selenium WebDriver spec -- headless, WebKit-based, pure Java
Other
809 stars 143 forks source link

jBrowserDriver hangs for very long time after "Unexpected error" #283

Open static-max opened 7 years ago

static-max commented 7 years ago

jbrowserdriver: 0.17.8

Sometimes, the whole JVM hangs and it takes 30 minutes to multiple hours for the JVM to exit. The only error I get:

[2017-07-02T00:24:41.950][Instance 1][Port 48566] Jul 02, 2017 2:24:41 AM com.sun.webkit.network.URLLoader doRun
[2017-07-02T00:24:41.950][Instance 1][Port 48566] WARNUNG: Unexpected error
[2017-07-02T00:24:41.950][Instance 1][Port 48566] java.io.IOException: The target server failed to respond: https://www.XXXXX.com
[2017-07-02T00:24:41.950][Instance 1][Port 48566]       at com.machinepublishers.jbrowserdriver.StreamConnection.exec(StreamConnection.java:367)
[2017-07-02T00:24:41.950][Instance 1][Port 48566]       at com.machinepublishers.jbrowserdriver.StreamConnection.getResponseCode(StreamConnection.java:447)
[2017-07-02T00:24:41.950][Instance 1][Port 48566]       at com.sun.webkit.network.URLLoader.receiveResponse(URLLoader.java:420)
[2017-07-02T00:24:41.950][Instance 1][Port 48566]       at com.sun.webkit.network.URLLoader.doRun(URLLoader.java:164)
[2017-07-02T00:24:41.950][Instance 1][Port 48566]       at com.sun.webkit.network.URLLoader.lambda$run$96(URLLoader.java:129)
[2017-07-02T00:24:41.950][Instance 1][Port 48566]       at java.security.AccessController.doPrivileged(Native Method)
[2017-07-02T00:24:41.950][Instance 1][Port 48566]       at com.sun.webkit.network.URLLoader.run(URLLoader.java:128)
[2017-07-02T00:24:41.950][Instance 1][Port 48566]       at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
[2017-07-02T00:24:41.950][Instance 1][Port 48566]       at java.util.concurrent.FutureTask.run(Unknown Source)
[2017-07-02T00:24:41.950][Instance 1][Port 48566]       at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[2017-07-02T00:24:41.950][Instance 1][Port 48566]       at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[2017-07-02T00:24:41.950][Instance 1][Port 48566]       at java.lang.Thread.run(Unknown Source)
[2017-07-02T00:24:41.950][Instance 1][Port 48566] Caused by: org.apache.http.NoHttpResponseException: The target server failed to respond
[2017-07-02T00:24:41.950][Instance 1][Port 48566]       at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
[2017-07-02T00:24:41.950][Instance 1][Port 48566]       at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
[2017-07-02T00:24:41.950][Instance 1][Port 48566]       at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
[2017-07-02T00:24:41.965][Instance 1][Port 48566]       at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165)
[2017-07-02T00:24:41.965][Instance 1][Port 48566]       at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:167)
[2017-07-02T00:24:41.965][Instance 1][Port 48566]       at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
[2017-07-02T00:24:41.965][Instance 1][Port 48566]       at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
[2017-07-02T00:24:41.965][Instance 1][Port 48566]       at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
[2017-07-02T00:24:41.965][Instance 1][Port 48566]       at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
[2017-07-02T00:24:41.965][Instance 1][Port 48566]       at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
[2017-07-02T00:24:41.965][Instance 1][Port 48566]       at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
[2017-07-02T00:24:41.965][Instance 1][Port 48566]       at com.machinepublishers.jbrowserdriver.StreamConnectionClient.execute(StreamConnectionClient.java:175)
[2017-07-02T00:24:41.965][Instance 1][Port 48566]       at com.machinepublishers.jbrowserdriver.StreamConnection.exec(StreamConnection.java:333)
[2017-07-02T00:24:41.965][Instance 1][Port 48566]       ... 11 more
[2017-07-02T00:24:41.965][Instance 1][Port 48566]

The error is logged, but the JVM process seems not to recognize is and waits a very long time and then shuts down without any additinal message.

hollingsworthd commented 7 years ago

It might be useful to explicitly set timeouts for the following because otherwise they rely on "system defaults" according to the underlying http library (Apache HttpComponents) --

http://machinepublishers.github.io/jBrowserDriver/com/machinepublishers/jbrowserdriver/Settings.Builder.html#socketTimeout-int- http://machinepublishers.github.io/jBrowserDriver/com/machinepublishers/jbrowserdriver/Settings.Builder.html#connectionReqTimeout-int- http://machinepublishers.github.io/jBrowserDriver/com/machinepublishers/jbrowserdriver/Settings.Builder.html#connectTimeout-int-

Perhaps though it's something related to this, https://stackoverflow.com/a/44703601

Or maybe garbage collection or something else is contributing to these really excessive delays.

hollingsworthd commented 7 years ago

If there are too many connections tying up too many resources, perhaps a lower limit for http://machinepublishers.github.io/jBrowserDriver/com/machinepublishers/jbrowserdriver/Settings.Builder.html#maxConnections-int- would help also.

What does CPU and memory usage look like when this hang occurs?