catchpoint / WebPageTest

Official repository for WebPageTest
Other
3.05k stars 709 forks source link

Move agent's traffic shaping to shell script #147

Closed zwri closed 8 years ago

zwri commented 10 years ago

The agent's traffic_shaper.js is currently hard-coded to use ipfw. Instead, the ipfw-specific details should be factored out into a user-modifiable script.

Current impl:

  1. Agent started with deviceAddr=1.2.3.4, gets job with "bwIn":5000,"bwOut":1000,"latency":28,"plr":0
  2. scheduleStart():
    1. call below "scheduleStop()"
    2. allocate unique pipeIds, e.g. "42" (via socket bind)
    3. ipfw add 42 pipe 42 ip from any to 1.2.3.4 in
    4. ipfw pipe 42 config bw 5000Kbit/s delay 14ms, ...
  3. Run job iteration
  4. scheduleStop():
    1. ipfw list then ifw deiete any rules/pipes that match our deviceAddr
    2. release pipeIds (via socket close)

The proposed solution is similar to our existing video_hdmi.js + "capture" script, but with more flexibility to support custom start/stop values:

  1. Add new --trafficShaper PATH arg with default value "lib/traffic_shaper/ipfw.sh"
  2. Alias current --deviceAddr 1.2.3.4 to optional --trafficShaperArg 1.2.3.4, where the value must match "^[^-\s]\S*$".
  3. Modify scheduleStart() to check if the script file exists, execute the shaper with args -s myDeviceId start 1.2.3.4 --bwIn 5000 --bwOut 1000 --latency 28 --plr 0, read and skip lines except an option line matching "^stop=([^-\s]\S*)$" (e.g. stop=foo_1.2.3.4), and return when it reads a line matching "^(.*\s)?started\." (or EOF or timeout after 10 seconds).
  4. Modify scheduleStop() to execute the shaper with args -s myDeviceId stop foo_1.2.3.4, return when it reads a line matching "^(.*\s)?stopped\." (or EOF or timeout after 10 seconds).

One awkward issue is recreating the dynamic, unique pipeId assignment in Bash. At worst we can loop on r=$RANDOM; nc -l -k $r. I'll have to think about this a bit...

zwri commented 10 years ago

Pushed to new branch wrightt:topic/ipfw_script commit https://github.com/wrightt/webpagetest/commit/b82f7020e7b3250b2aac086d003c73e31254fae9 Code review is on https://codereview.appspot.com/38470044/

zwri commented 10 years ago

Some changes re: original design:

  1. Added "--arg" after start/stop command, e.g.:

     myScript -s myDeviceId start --arg 1.2.3.4 --bwIn ...
  2. Script doesn't have to print "started." or "stopped."
  3. Added "_arg" to stdout pattern, e.g.

    stop_arg=foo_1.2.3.4
  4. Didn't fully port ipfw to script in this pass; see notes at end of traffic_shaper_ipfw.js for details
pmeenan commented 8 years ago

Done