catchpoint / WebPageTest.agent

Cross-platform WebPageTest agent
Other
213 stars 139 forks source link

WPTagent on CentOS 7 unable to run tests on Chrome #281

Open PrinceMedium opened 5 years ago

PrinceMedium commented 5 years ago

I'm having huge issues running tests on Google Chrome, Firefox is running fine.

Using CentOS 7, webserver version 19.04 and the newest WPTagent. Webserver and WPTagent is on the same server. Currently running the server on VMware, but have tried physicals servers also and getting the same problem.

While trying to fix the issue, I've disabled to wptagent systemctl and running one of these manually from the command line instead (difference is the ---xvfb option):

python /var/www/webpagetest/wptagent/wptagent.py --server http://localhost/work/ --location WPT1_CPH -vvvv --name wptagent --shaper netem,ens192

python /var/www/webpagetest/wptagent/wptagent.py --server http://localhost/work/ --location WPT1_CPH -vvvv --name wptagent --xvfb --shaper netem,ens192

Subset of the WPTagent log:

11:05:16.020 - Job: {"processResults":1,"runs":1,"bwIn":5000,"plr":0,"latency":28,"url":"https:\/\/dr.dk","Test ID":"190827_GM_S","Capture Video":1,"IPAddr":"109.238.48.21","bwOut":1000,"orientation":"default","fvonly":1,"timeline":1,"lighthouseTrace":1,"timelineStackDepth":0,"browser":"Chrome"} 11:05:16.020 - [{'record': True, 'command': 'navigate', 'target': u'https://dr.dk'}] 11:05:16.024 - Preparing browser 11:05:16.024 - Terminating all instances of chrome 11:05:16.057 - Flushing DNS Redirecting to /bin/systemctl restart dnsmasq.service Failed to restart dnsmasq.service: Unit not found. sudo: rndc: command not found sudo: systemd-resolve: command not found 11:05:16.102 - Clearing profile /var/www/webpagetest/wptagent/work/wptagent/browser.190827_GM_S.1 11:05:16.158 - /opt/google/chrome/chrome --disable-background-networking --no-default-browser-check --no-first-run --new-window --disable-infobars --disable-translate --disable-notifications --disable-desktop-notifications --disable-save-password-bubble --allow-running-insecure-content --disable-component-update --disable-background-downloads --disable-add-to-shelf --disable-client-side-phishing-detection --disable-datasaver-prompt --disable-default-apps --disable-device-discovery-notifications --disable-domain-reliability --safebrowsing-disable-auto-update --disable-background-timer-throttling --disable-browser-side-navigation --net-log-capture-mode=IncludeCookiesAndCredentials --load-media-router-component-extension=0 --host-resolver-rules="MAP cache.pack.google.com 127.0.0.1","MAP clients1.google.com 127.0.0.1" --window-position="0,0" --window-size="1024,768" --remote-debugging-port=9222 --user-data-dir="/var/www/webpagetest/wptagent/work/wptagent/browser.190827_GM_S.1" --disable-gpu --disable-setuid-sandbox --enable-features=NetworkService,NetworkServiceInProcess,SecMetadata --enable-blink-features=LayoutInstabilityAPI about:blank 11:05:16.165 - Starting new HTTP connection (1): localhost:9222 11:05:16.166 - Connect to dev tools Error: HTTPConnectionPool(host='localhost', port=9222): Max retries exceeded with url: /json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb01cde1850>: Failed to establish a new connection: [Errno 111] Connection refused',))

.... (the last lines are repeated every few seconds until timeout)

kgrz commented 5 years ago

Okay, from what I debugged, this happens when the above chrome process crashes immediately after invoking. And this may be related to the sandboxing changes that happened in the recent past. I'll add a tl;dr version here, but will add a more detailed log in a separate issue:

Note that this issue won't show up if you're running the agent using docker because the --no-sandbox flag is being passed. Relevant portions in code:

https://github.com/WPO-Foundation/wptagent/blob/ec8a7229e6fafb41db5d33a00daf32638fbec7f5/internal/chrome_desktop.py#L94-L97