MachinePublishers / jBrowserDriver

A programmable, embeddable web browser driver compatible with the Selenium WebDriver spec -- headless, WebKit-based, pure Java
Other
809 stars 143 forks source link

EOFException on linux #301

Open qabi opened 6 years ago

qabi commented 6 years ago

In a small script running on Linux (Ubuntu 16.04, OpenJDK 8) I keep getting this exception at seemingly random times, after a long passive wait/timeout:

org.openqa.selenium.WebDriverException: Build info: version: '2.53.0', revision: '35ae25b1534ae328c771e0856c93e187490ca824', time: '2016-03-15 10:43:46'
System info: host: 'web1', ip: '10.181.162.115', os.name: 'Linux', os.arch: 'amd64', os.version: '4.4.0-96-generic', java.version: '1.8.0_131'
Driver info: driver.version: JBrowserDriver
    at com.machinepublishers.jbrowserdriver.Util.handleException(Util.java:139)
    at com.machinepublishers.jbrowserdriver.JBrowserDriver.findElementsByCssSelector(JBrowserDriver.java:929)
    at [removed]
Caused by: java.io.EOFException
    at java.io.DataInputStream.readByte(DataInputStream.java:267)
    at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:222)
    at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
    at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:227)
    at java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:179)
    at com.machinepublishers.jbrowserdriver.$Proxy1.findElementsByCssSelector(Unknown Source)
    at com.machinepublishers.jbrowserdriver.JBrowserDriver.findElementsByCssSelector(JBrowserDriver.java:925)

It seems that it is the communication with a sub-process that is going wrong? Is a sub-process crashing or something like that?

The same script seems to work find on a mac (Oracle Java 8). And It also seemed to work better previously. Changes might have been made in the script, but nothing that should be able to cause this?

I would very much appreciate a solution for this problem. Anything i can do to help debug/troubleshoot this issue?

efernandesng commented 6 years ago

@qabi I run with OpenJDK 8 (1.8.0_144) on Arch Linux without any problem. What is the version (java -version) of your openjdk?

You tested with Oracle Java 8 on Linux?

qabi commented 6 years ago

openjdk version "1.8.0_131" OpenJDK Runtime Environment (build 1.8.0_131-8u131-b11-2ubuntu1.16.04.3-b11) OpenJDK 64-Bit Server VM (build 25.131-b11, mixed mode).

Let me know if I can help, how I can help, etc.

It is the standard one in Ubuntu 16.04.

efernandesng commented 6 years ago

You have installed OpenJFX (apt-get install openjfx)? JavaFx is required by jBrowserDriver and is not included in OpenJDK package.

qabi commented 6 years ago

Yes, and it does work to some extent. It just hangs and dies sometimes. I have also seen the script complete without problems a few times.

There seems to be no recognizable pattern in which call hangs and dies. It is always the same method, but then when the script is altered, it changes to always be a different method.

qabi commented 6 years ago

Any news on this one? Will it be in version 1 (and when is this expected?)

Thanks.

qabi commented 6 years ago

Any idea on how this could be fixed, or worked around? I'd be happy to help, to the best of my abilities :)

qabi commented 6 years ago

This exception seems to be related to excessive CPU usage. I have a JBrowserDriver script running on a (somewhat underpowered) virtual linux server. This process uses so much CPU that the load rises to over 10, any SSH connections are cut off and the HTTP server stops responding.

Any ideas on how to debug this further? To find out why it is using so much CPU?

smatei commented 5 years ago

Hello,

It happens for me when running in an Ubuntu Docker container. You have the setup here:

https://github.com/smatei/scraper-ubuntu-docker

When running on Windows machine or Centos machine, without container, it works. This is how the Docker setup looks like:

FROM ubuntu:18.04

RUN apt-get update \

##################################################################

install java

&& apt-get install -y openjdk-8-jdk \

&& apt-get install -y maven \

&& apt-get install -y git-core \

&& apt-get install -y openjfx \

##########################

cleanup install packages

&& rm -rf /var/lib/apt/lists/*

##########################

ENTRYPOINT bash

I have used an array of domains in the test, but it fails with just one too.