search

MachinePublishers / jBrowserDriver

A programmable, embeddable web browser driver compatible with the Selenium WebDriver spec -- headless, WebKit-based, pure Java
Other
809 stars 143 forks source link

Retry failed requests (e.g., java.lang.NullPointerException: Inflater has been closed) #242

Closed coyun closed 6 years ago

coyun commented 7 years ago

java.lang.NullPointerException: Inflater has been closed [2017-01-19T05:26:07.841][Instance 1][Port 55267] at java.util.zip.Inflater.ensureOpen(Inflater.java:389) [2017-01-19T05:26:07.841][Instance 1][Port 55267] at java.util.zip.Inflater.inflate(Inflater.java:257) [2017-01-19T05:26:07.841][Instance 1][Port 55267] at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152) [2017-01-19T05:26:07.841][Instance 1][Port 55267] at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:117) [2017-01-19T05:26:07.841][Instance 1][Port 55267] at org.apache.http.client.entity.LazyDecompressingInputStream.read(LazyDecompressingInputStream.java:73) [2017-01-19T05:26:07.841][Instance 1][Port 55267] at com.machinepublishers.jbrowserdriver.Util.toBytes(Util.java:98) [2017-01-19T05:26:07.841][Instance 1][Port 55267] at com.machinepublishers.jbrowserdriver.ResponseHandler.handleResponse(ResponseHandler.java:52) [2017-01-19T05:26:07.842][Instance 1][Port 55267] at com.machinepublishers.jbrowserdriver.StreamConnection.getInputStream(StreamConnection.java:402) [2017-01-19T05:26:07.842][Instance 1][Port 55267] at com.sun.webkit.network.URLLoader.receiveResponse(URLLoader.java:493) [2017-01-19T05:26:07.842][Instance 1][Port 55267] at com.sun.webkit.network.URLLoader.doRun(URLLoader.java:162) [2017-01-19T05:26:07.842][Instance 1][Port 55267] at com.sun.webkit.network.URLLoader.lambda$run$90(URLLoader.java:127) [2017-01-19T05:26:07.842][Instance 1][Port 55267] at com.sun.webkit.network.URLLoader$$Lambda$149/22435197.run(Unknown Source) [2017-01-19T05:26:07.842][Instance 1][Port 55267] at java.security.AccessController.doPrivileged(Native Method) [2017-01-19T05:26:07.842][Instance 1][Port 55267] at com.sun.webkit.network.URLLoader.run(URLLoader.java:126) [2017-01-19T05:26:07.842][Instance 1][Port 55267] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [2017-01-19T05:26:07.842][Instance 1][Port 55267] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [2017-01-19T05:26:07.842][Instance 1][Port 55267] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [2017-01-19T05:26:07.842][Instance 1][Port 55267] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [2017-01-19T05:26:07.842][Instance 1][Port 55267] at java.lang.Thread.run(Thread.java:745) [2017-01-19T05:26:07.842][Instance 1][Port 55267]

hollingsworthd commented 7 years ago

I've seen this bug before and after some googling it appears to afflict other projects too but don't know of a root cause. It occurs to me, perhaps within the StreamConnection and StreamConnectionClient classes in this project, the Apache HttpComponents client which actually executes http requests can be set to retry failed requests like this. If everything else appears to be working normally this probably doesn't need to be a huge concern in your use--most web pages these days make tons of requests to ad servers and such and some of them fail sometimes and the driver can move on gracefully.

hollingsworthd commented 7 years ago

Based on info reported in #267 problem occurs in 0.16 but not 0.15

arya6000 commented 7 years ago

I've also noticed this, and I have noticed when this problem occurs "sometimes" the program hangs, but sometimes it will continue working.

brilliantcontract commented 7 years ago

I have similar problem, there is a log:

http://careers.mountsinai.org/find-your-place/jobs/sleep-technician-per-diem-new-york-ny-4 00:01:34.036 [qtp282828951-14] INFO bc.lib.filetransfer.downloader.webdriver.DownloaderWebDriver - Try download page (1/5) [without proxy]: http://careers.mountsinai.org/find-your-place/jobs/sleep-technician-per-diem-new-york-ny-4 [2017-04-21T23:01:44.766][Instance 1][Port 44721] java.lang.NullPointerException: Inflater has been closed [2017-04-21T23:01:44.766][Instance 1][Port 44721] at java.util.zip.Inflater.ensureOpen(Inflater.java:389) [2017-04-21T23:01:44.767][Instance 1][Port 44721] at java.util.zip.Inflater.inflate(Inflater.java:257) [2017-04-21T23:01:44.767][Instance 1][Port 44721] at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152) [2017-04-21T23:01:44.767][Instance 1][Port 44721] at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:117) [2017-04-21T23:01:44.767][Instance 1][Port 44721] at org.apache.http.client.entity.LazyDecompressingInputStream.read(LazyDecompressingInputStream.java:73) [2017-04-21T23:01:44.767][Instance 1][Port 44721] at com.machinepublishers.jbrowserdriver.Util.toBytes(Util.java:98) [2017-04-21T23:01:44.768][Instance 1][Port 44721] at com.machinepublishers.jbrowserdriver.ResponseHandler.handleResponse(ResponseHandler.java:52) [2017-04-21T23:01:44.768][Instance 1][Port 44721] at com.machinepublishers.jbrowserdriver.StreamConnection.getInputStream(StreamConnection.java:408) [2017-04-21T23:01:44.768][Instance 1][Port 44721] at com.sun.webkit.network.URLLoader.receiveResponse(URLLoader.java:493) [2017-04-21T23:01:44.768][Instance 1][Port 44721] at com.sun.webkit.network.URLLoader.doRun(URLLoader.java:162) [2017-04-21T23:01:44.768][Instance 1][Port 44721] at com.sun.webkit.network.URLLoader.lambda$run$90(URLLoader.java:127) [2017-04-21T23:01:44.769][Instance 1][Port 44721] at com.sun.webkit.network.URLLoader$$Lambda$151/797004414.run(Unknown Source) [2017-04-21T23:01:44.769][Instance 1][Port 44721] at java.security.AccessController.doPrivileged(Native Method) [2017-04-21T23:01:44.769][Instance 1][Port 44721] at com.sun.webkit.network.URLLoader.run(URLLoader.java:126) [2017-04-21T23:01:44.769][Instance 1][Port 44721] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [2017-04-21T23:01:44.769][Instance 1][Port 44721] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [2017-04-21T23:01:44.771][Instance 1][Port 44721] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [2017-04-21T23:01:44.771][Instance 1][Port 44721] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [2017-04-21T23:01:44.771][Instance 1][Port 44721] at java.lang.Thread.run(Thread.java:745)

hollingsworthd commented 6 years ago

See instead #286