SeleniumHQ / selenium

A browser automation framework and ecosystem.
https://selenium.dev
Apache License 2.0
30.67k stars 8.19k forks source link

High CPU processor usage with firefox-v40,gecko-0.23 and selenium-v3.14.0 #8299

Closed siddhant42 closed 4 years ago

siddhant42 commented 4 years ago

description:: I am using selenium with firefox-v61 and geckodriver-0.23 for crawling. I am running it with heavy scale -around 50 simultaneous processes of firefox every time in a new window. but firefox is using very high processor usage- around 3 processes(including 2 child-Id) for every site opening and rendering . And my total cpu usage of machine is becoming around 60 to 70. I am opening every time a new firefox instance to open the site due to some project criteria.

Earlier i used the older version of firefox-v40 and for opening a site only one process(no child-Id) is used per site for opening in firefox.And total processor usage in that case was around 25 to 30. Now i have upgraded selenium and firefox in my project and due to this issue i am not able to go ahead with my project. Can you please let me know how can i overcome the process usage of firefox with such scenario?

Detailed steps to reproduce the behavior: If you open mulitple firefox with v40 and v61, you can see the difference in processor usage.

Expected behavior: The processor usage should be same or somewhat more than what it used to be at firefox-v40, but not such a heavy difference.

Script: my $max_processes = 50; my $disply = 100 + (int(rand 30) * 3); my $ROOT_PATH = /home/developer/crawler; my $JSpiderVersion = v4.1.0; my cmd="xvfb-run -n $disply -a -s '-screen 0 1024x768x24' $ROOT_PATH/jdk1.8.0_181/bin/java -Xmx2g -XX:ParallelGCThreads=2 -Dwebdriver.gecko.driver=$ROOT_PATH/etc/geckoDriver_0.23/geckodriver -cp $ROOT_PATH/release/$JSpiderVersion/JSpider-$JSpiderVersion.jar com.naukri.crawler.driver.Srp --config $ROOT_PATH/release/$JSpiderVersion/etc/crawler.srp.config";

Environment::

OS: ubunt 18.0.4 Browser: firefox Browser version: 61,0 Browser Driver version: GeckoDriver 0.23 Language Bindings version: Java 8 Selenium Grid version (if applicable): 3.14.0

ghost commented 4 years ago

šŸ‘‹ Hi there! Thank you for creating this issue.

I am the Selenium Assistant Bot šŸ¤–, I triage issues in this repository. If I can't do it, I label it to help maintainers identify issues that need triaging.

I am an Open Source project šŸ™Œ, post bugs or ideas here!

ghost commented 4 years ago

ā—ļø It seems this issue is not using any of the supported templates

šŸ’” Supported issue types are (they start with):

Issue templates help this project to stay in shape, please use them and fill them out completely. By doing that you are helping the project because the community and maintainers can provide prompt feedback, and potentially solve the issue.

If you are asking a question, a better way to address this is:

If you think this is incorrect, please feel free to open a new issue.

Thank you for your contributions.