Figure out a sane functional testing suite

[shawnbot]

I like Nightwatch, but I want to run and configure it like zuul. I'm wondering if there's room for a wrapper CLI that will run the Nightwatch runner and merge one of the configured "environments" (e.g. one local w/Selenium Server, another on Sauce Labs) with browser-specific configurations, then run itself once for each variation. Something like:

# implies -e default
nightwatcher -c nightwatch.json -b phantomjs
# test on Sauce with IE 8 & 9
nightwatcher -c nightwatch.json -e sauce -b ie@8..9

or you could specify one or more configurations that list which browsers to test, zuul-style:

# ie.yaml
browsers:
- name: ie
  version: 8..9

nightwatcher -c nightwatch.json -e sauce --browsers-from ie.yaml

Off the top of my head, the most important features would be:

1. Reasonable browser defaults so that people don't have to configure each one to test. Being able to do a one-off remote IE9 test, for instance, should be possible out of the box.
2. Sauce Labs support out of the box with auth configured automatically via environment variables (SAUCE_USERNAME and SAUCE_ACCESS_KEY are pretty much the standard).
3. Built-in launchers for Selenium Server for local testing, and Sauce Connect for remote testing of private URLs (namely, localhost) and CI services like Travis.

[shawnbot]

Use cases:

• Run Nightwatch tests locally against any available browser (default: PhantomJS).
• Run a one-off (either suite or individual) test against a single remote browser.
• Configure cross-browser testing runs either locally or remotely.

[shawnbot]

Also to consider: WebdriverIO, which has multi-browser support built in. (See the example config, where they set capabilities to an array. This would be a good place to use desired-capabilities.