Agent should not attempt traffic-shaping on Mac when specifying native

[Krinkle]

The traffic-shaping feature doesn't work on Mac under Docker directly right now. However, even when submitting a test with "Connection: Native connection (No traffic shaping)", the agent still tries to set up something related to it and fails the run early.

10:22:50.818 - http://192.168.65.1:4001 "GET /work/getwork.php?f=json&shards=1&location=Test&pc=7010319d4402&screenwidth=1920&screenheight=1200&freedisk=55.338 HTTP/1.1" 200 236
10:22:50.819 - Job: {"processResults":1,"runs":3,"bwIn":0,"plr":0,"latency":0,"url":"https:\/\/test.wikipedia.org\/wiki\/Sandbox","Test ID":"170914_XY_3","Capture Video":1,"IPAddr":"172.17.0.1","bwOut":0,"orientation":"default","fvonly":1,"timeline":1,"lighthouseTrace":1,"timelineStackDepth":0,"browser":"Chrome"}
10:22:50.819 - [{'record': True, 'command': 'navigate', 'target': u'https://test.wikipedia.org/wiki/Sandbox'}]
10:22:50.820 - Preparing browser
10:22:50.820 - Terminating all instances of chrome
10:22:50.826 - Flushing DNS
dnsmasq: unrecognized service
rndc: neither /etc/bind/rndc.conf nor /etc/bind/rndc.key was found
10:22:50.843 - Clearing profile /wptagent/work/7010319d4402/browser.170914_XY_3.1
10:22:50.844 - /opt/google/chrome/chrome --disable-background-networking --no-default-browser-check --no-first-run --process-per-tab --new-window --disable-infobars --disable-translate --disable-notifications --disable-desktop-notifications --disable-save-password-bubble --allow-running-insecure-content --disable-component-update --disable-background-downloads --disable-add-to-shelf --disable-client-side-phishing-detection --disable-datasaver-prompt --disable-default-apps --disable-domain-reliability --safebrowsing-disable-auto-update --disable-background-timer-throttling --host-resolver-rules="MAP cache.pack.google.com 127.0.0.1","MAP clients1.google.com 127.0.0.1" --window-position="0,0" --window-size="1024,768" --remote-debugging-port=9228 --user-data-dir="/wptagent/work/7010319d4402/browser.170914_XY_3.1" --disable-gpu --no-sandbox about:blank
10:22:50.847 - Starting new HTTP connection (1): localhost
10:22:50.848 - Connect to dev tools Error: HTTPConnectionPool(host='localhost', port=9228): Max retries exceeded with url: /json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0af59f0410>: Failed to establish a new connection: [Errno 111] Connection refused',))
[1581:1590:0914/102250.965725:ERROR:bus.cc(395)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory

(chrome:1581): LIBDBUSMENU-GLIB-WARNING **: Unable to get session bus: Unknown or unsupported transport 'disabled' for address 'disabled:'
10:22:51.351 - Starting new HTTP connection (1): localhost
10:22:51.353 - http://localhost:9228 "GET /json HTTP/1.1" 200 2189
10:22:51.354 - Dev Tools tabs: [{"description":"","title":"CryptoTokenExtension","url":"chrome-extension:\/\/kmendfapggjehodndflmmgagdbamhnfd\/_generated_background_page.html","webSocketDebuggerUrl":"ws:\/\/localhost:9228\/devtools\/page\/f90cfffc-50d5-40f7-adbd-2d4f5d6a5527","type":"background_page","id":"f90cfffc-50d5-40f7-adbd-2d4f5d6a5527","devtoolsFrontendUrl":"\/devtools\/inspector.html?ws=localhost:9228\/devtools\/page\/f90cfffc-50d5-40f7-adbd-2d4f5d6a5527"},{"description":"","title":"Google Network Speech","url":"chrome-extension:\/\/neajdppkdcdipfabeoofebfddakdcjhd\/_generated_background_page.html","webSocketDebuggerUrl":"ws:\/\/localhost:9228\/devtools\/page\/9eb6a2b1-cdf9-4186-bfd7-71ec5fbd09b3","type":"background_page","id":"9eb6a2b1-cdf9-4186-bfd7-71ec5fbd09b3","devtoolsFrontendUrl":"\/devtools\/inspector.html?ws=localhost:9228\/devtools\/page\/9eb6a2b1-cdf9-4186-bfd7-71ec5fbd09b3"},{"description":"","title":"Feedback","url":"chrome-extension:\/\/gfdkimpbcpahaombhbimeihdjnejgicl\/_generated_background_page.html","webSocketDebuggerUrl":"ws:\/\/localhost:9228\/devtools\/page\/0ac98e70-c662-46d0-8013-30d290dedc68","type":"background_page","id":"0ac98e70-c662-46d0-8013-30d290dedc68","devtoolsFrontendUrl":"\/devtools\/inspector.html?ws=localhost:9228\/devtools\/page\/0ac98e70-c662-46d0-8013-30d290dedc68"},{"description":"","title":"Google Hangouts","url":"chrome-extension:\/\/nkeimhogjdpnpccoofpliimaahmaaome\/background.html","webSocketDebuggerUrl":"ws:\/\/localhost:9228\/devtools\/page\/89bf1205-755c-498b-bb52-288634ab5788","type":"background_page","id":"89bf1205-755c-498b-bb52-288634ab5788","devtoolsFrontendUrl":"\/devtools\/inspector.html?ws=localhost:9228\/devtools\/page\/89bf1205-755c-498b-bb52-288634ab5788"},{"description":"","title":"about:blank","url":"about:blank","webSocketDebuggerUrl":"ws:\/\/localhost:9228\/devtools\/page\/3b53b6d4-4fec-4a32-a20c-303d020236c3","type":"page","id":"3b53b6d4-4fec-4a32-a20c-303d020236c3","devtoolsFrontendUrl":"\/devtools\/inspector.html?ws=localhost:9228\/devtools\/page\/3b53b6d4-4fec-4a32-a20c-303d020236c3"}]
10:22:51.356 - DevTools websocket connected
10:22:51.356 - Devtools connected
10:22:51.357 - Sending: {"params":{"returnByValue":true,"expression":"[window.innerWidth, window.innerHeight]"},"id":1,"method":"Runtime.evaluate"}
10:22:51.365 - {"id":1,"result":{"result":{"type":"object","value":[1024,702]}}}
10:22:51.365 - Sending: {"params":{"returnByValue":true,"expression":"window.devicePixelRatio"},"id":2,"method":"Runtime.evaluate"}
10:22:51.366 - {"id":2,"result":{"result":{"type":"number","value":1,"description":"1"}}}
10:22:51.367 - Sending: {"params":{},"id":3,"method":"Network.clearBrowserCache"}
10:22:51.370 - {"id":3,"result":{}}
10:22:51.370 - Sending: {"params":{},"id":4,"method":"Network.clearBrowserCookies"}
10:22:51.387 - {"id":4,"result":{}}
10:22:51.388 - Sending: {"params":{"returnByValue":true,"expression":"navigator.userAgent"},"id":5,"method":"Runtime.evaluate"}
10:22:51.396 - {"id":5,"result":{"result":{"type":"string","value":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"}}}
10:22:51.396 - Sending: {"params":{"waitForDebuggerOnStart":true,"autoAttach":true},"id":6,"method":"Target.setAutoAttach"}
10:22:51.396 - Sending: {"params":{},"id":7,"method":"Target.getTargets"}
10:22:51.398 - {"id":7,"result":{"targetInfos":[{"targetId":"f90cfffc-50d5-40f7-adbd-2d4f5d6a5527","type":"background_page","title":"CryptoTokenExtension","url":"chrome-extension://kmendfapggjehodndflmmgagdbamhnfd/_
10:22:51.398 - {u'url': u'chrome-extension://kmendfapggjehodndflmmgagdbamhnfd/_generated_background_page.html', u'type': u'background_page', u'attached': False, u'targetId': u'f90cfffc-50d5-40f7-adbd-2d4f5d6a5527', u'title': u'CryptoTokenExtension'}
10:22:51.398 - {u'url': u'chrome-extension://neajdppkdcdipfabeoofebfddakdcjhd/_generated_background_page.html', u'type': u'background_page', u'attached': False, u'targetId': u'9eb6a2b1-cdf9-4186-bfd7-71ec5fbd09b3', u'title': u'Google Network Speech'}
10:22:51.399 - {u'url': u'chrome-extension://gfdkimpbcpahaombhbimeihdjnejgicl/_generated_background_page.html', u'type': u'background_page', u'attached': False, u'targetId': u'0ac98e70-c662-46d0-8013-30d290dedc68', u'title': u'Feedback'}
10:22:51.399 - {u'url': u'chrome-extension://nkeimhogjdpnpccoofpliimaahmaaome/background.html', u'type': u'background_page', u'attached': False, u'targetId': u'89bf1205-755c-498b-bb52-288634ab5788', u'title': u'Google Hangouts'}
10:22:51.399 - {u'url': u'about:blank', u'type': u'page', u'attached': True, u'targetId': u'3b53b6d4-4fec-4a32-a20c-303d020236c3', u'title': u'about:blank'}
10:22:51.399 - Sending: {"params":{"url":"data:text\/html,%3Chtml%3E%0D%0A%3Chead%3E%0D%0A%3Cstyle%3E%0D%0Abody%20%7Bbackground-color%3A%20white%3B%20margin%3A%200%3B%7D%0D%0A%23o%20%7Bwidth%3A100%25%3B%20height%3A%20100%25%3B%20background-color%3A%20%23DE640D%3B%7D%0D%0A%3C%2Fstyle%3E%0D%0A%3Cscript%3E%0D%0Awindow.addEventListener%28%27beforeunload%27%2C%20function%28%29%20%7B%0D%0A%20%20var%20o%20%3D%20document.getElementById%28%27o%27%29%0D%0A%20%20o.parentNode.removeChild%28o%29%3B%0D%0A%7D%29%3B%0D%0A%3C%2Fscript%3E%0D%0A%3C%2Fhead%3E%0D%0A%3Cbody%3E%3Cdiv%20id%3D%27o%27%3E%3C%2Fdiv%3E%3C%2Fbody%3E%0D%0A%3C%2Fhtml%3E"},"id":8,"method":"Page.navigate"}
10:22:51.400 - {"id":6,"result":{}}
10:22:51.403 - {"id":8,"result":{"frameId":"1687.1"}}
10:22:51.403 - Waiting for Idle...
10:22:52.016 - Configuring traffic shaping: 0/0 - 0 ms, 0.00% plr
10:22:52.016 - sudo tc qdisc add dev ifb0 root handle 1:0 netem delay 0ms
Cannot find device "ifb0"
10:22:52.025 - Resetting traffic shaping
Cannot find device "ifb0"
10:22:52.034 - DevTools websocket disconnected
10:22:52.035 - Starting new HTTP connection (1): localhost
10:22:52.036 - http://localhost:9228 "GET /json/close/3b53b6d4-4fec-4a32-a20c-303d020236c3 HTTP/1.1" 200 17
10:22:52.040 - Stopping browser
10:22:52.040 - Terminating all instances of chrome
[0914/102252.045258:ERROR:nacl_helper_linux.cc(310)] NaCl helper process running without a sandbox!
Most likely you need to configure your SUID sandbox correctly
10:22:52.051 - Waiting up to 30 seconds for chrome to exit
10:22:52.088 - Test run time: 1.269 sec
10:22:52.088 - Uploading result
10:22:52.089 - Uploading result zip
10:22:52.089 - http://192.168.65.1:4001/work/workdone.php?run=1&cached=0&pc=7010319d4402&location=Test&error=Error+configuring+traffic-shaping&id=170914_XY_3&
10:22:52.117 - http://192.168.65.1:4001 "POST /work/workdone.php?run=1&cached=0&pc=7010319d4402&location=Test&error=Error+configuring+traffic-shaping&id=170914_XY_3& HTTP/1.1" 200 0
10:22:52.118 - [{'record': True, 'command': 'navigate', 'target': u'https://test.wikipedia.org/wiki/Sandbox'}]
10:22:52.119 - Preparing browser
10:22:52.119 - Terminating all instances of chrome

Environment:

# Server
docker run -p 4001:80 --rm webpagetest/server

# Agent
docker run --rm \
-e SERVER_URL="http://192.168.65.1:4001/work/" \
-e LOCATION="Test" \
--cap-add=NET_ADMIN \
webpagetest/agent

Steps to reproduce:

1. http://localhost:4001/
2. Submit test with connection set to native.
3. Run result is empty, agent console showing above error.

[pmeenan]

You need to also pass EXTRA_ARGS="--shaper none" to tell the agent that shaping isn't possible, otherwise it will default to the netem shaper on Linux (in Docker) and still try to set up the configuration.

[Krinkle]

@pmeenan Where can I pass that with docker run? It seems the current entry point defines this variable locally, thus not allowing it to be defined via an environment variable through docker run.

[pmeenan]

Argh, thanks - Looks like it needs an update or I need to teach the agent to autodetect if shaping is possible or not.

[pmeenan]

I don't have a docker container handy to test but you should be able to pass -e SHAPER="none" now and it also passes arbitrary command-line options through if you supply EXTRA_ARGS

[Krinkle]

Thanks. Works great!