groupon / Selenium-Grid-Extras

Simplify the management of the Selenium Grid Nodes and stabilize said nodes by cleaning up the test environment after the build has been completed
BSD 3-Clause "New" or "Revised" License
536 stars 209 forks source link

Running out of heap space #306

Open VivaLosDoyers opened 7 years ago

VivaLosDoyers commented 7 years ago

I've got a test that gets run via Selenium Grid Extras to a Windows node. In this test, I use a Selenium file_detector to send a large file (about 1.2GB) from the running machine to the node:

    $browser.file_detector = lambda do |args|
      str = args.first.to_s
      str if File.exist?(str)
    end

I can perform this test with small files no problem. But when I send this larger file, my heap runs out.

[remote server] org.openqa.grid.web.servlet.handler.RequestHandler(RequestHandler.java):138:in `process': cannot forward the request Java heap space (org.openqa.grid.common.exception.GridException) (Selenium::WebDriver::Error::UnknownError)
    from [remote server] org.openqa.grid.web.servlet.DriverServlet(DriverServlet.java):83:in `process'
    from [remote server] org.openqa.grid.web.servlet.DriverServlet(DriverServlet.java):67:in `doPost'

On both the node and hub machines, I'm running GridExtras with an increased heap size:

/usr/bin/java -d64 -Xms512m -Xmx16g -jar SeleniumGridExtras-1.12.10-SNAPSHOT-jar-with-dependencies.jar

I noticed when probing the hub, it looks like the child web driver processes are the ones bailing out due to heap space. Since SeleniumGridExtras calls these processes, I presume they're using some sort of default heap space that isn't enough?

mezzanine@qagrid01:~$ ps -eo pmem,pcpu,vsize,pid,cmd | sort -k 1 -nr | head -5
28.3 14.4 21438400 3661 /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -cp /home/mezzanine/grid/SeleniumGridExtras-1.12.10-SNAPSHOT-jar-with-dependencies.jar:/tmp/webdriver/3.0.1.jar org.openqa.grid.selenium.GridLauncherV3 -role hub -log log/hub_4444.log -hubConfig hub_4444.json
 0.6 53.7 10327332 2319 /usr/lib/virtualbox/VirtualBox --comment QAWINNODE01 --startvm 228d932d-cb3f-4ba8-95d9-1f1ea93b66e6 --no-startvm-errormsgbox
 0.4  0.4 20674976 3630 java -d64 -Xms512m -Xmx16g -jar SeleniumGridExtras-1.12.10-SNAPSHOT-jar-with-dependencies.jar

Is there a way I can run the grid in such a way that it will just use however much memory it needs? I hope this makes sense. I'm not really that familiar with the JVM.

smccarthy commented 7 years ago

@VivaLosDoyers You can set the JVM arguments in the selenium grid config file:

"grid_jvm_x_options": "-Xmx4g -XX:ParallelGCThreads=12"

I never have myself, but you can see the comment here , https://github.com/groupon/Selenium-Grid-Extras/issues/101#issuecomment-146579482 . And the PR that added it : https://github.com/groupon/Selenium-Grid-Extras/pull/160