Document how to setup a SauceLabs account and configure for testing

[tkalfigo]

Running grunt does the build. But grunt test fails at the intern:functional task. It seems that it fails when opening the tunnel to SauceLabs due to missing credentials. The grunt test output is:

Running "intern:functional" (intern) task
Verifying property intern.functional exists in config...OK
File: [no files]
Options: runType="runner", config="tests/intern.js", functionalSuites=["tests/functional/detectors.js","tests/functional/helpers.js"]
Listening on 0.0.0.0:9000
Error: Tunnel failed to start: Exit code: 1
  at handleChildExit  <node_modules/digdug/Tunnel.js:336:16>
  at Socket.<anonymous>  <node_modules/digdug/Tunnel.js:379:6>
  at emitOne  <events.js:101:20>
  at Socket.emit  <events.js:191:7>
  at Pipe._handle.close [as _onclose]  <net.js:511:12>
Warning: Test failure; check output above for details. Use --force to continue.

and in the generated ./SauceLabsTunnel.log the last few entries are:

20170823 123219.209 [22439] ***********************************************************
20170823 123219.209 [22439] A newer version of Sauce Connect (build 3688) is available!
20170823 123219.209 [22439] Download it here:
20170823 123219.209 [22439] https://saucelabs.com/downloads/sc-4.4.9-linux.tar.gz
20170823 123219.209 [22439] ***********************************************************
20170823 123219.209 [22439] PROXY *** scproxy started ***
20170823 123219.210 [22439] PROXY found DNS server 127.0.1.1
20170823 123219.259 [22439] Started scproxy on port 40559.
20170823 123219.259 [22439] Please wait for 'you may start your tests' to start your tests.
20170823 123219.259 [22439] Starting secure remote tunnel VM...
20170823 123220.359 [22439] GET https://saucelabs.com/rest/v1/undefined/tunnels?full=1: HTTP response code indicated failure.
20170823 123220.359 [22439] Response: {"message": "Not authorized"}.
20170823 123220.359 [22439] Error checking overlapping tunnels.
20170823 123220.359 [22439] Cleaning up.
20170823 123220.360 [22439] Sauce Connect could not establish a connection.
20170823 123220.360 [22439] Please check your firewall and proxy settings.
20170823 123220.360 [22439] You can also use sc --doctor to launch Sauce Connect in diagnostic mode.
20170823 123220.360 [22439] Goodbye.

Tried this both locally and on travis-ci.org, thinking that perhaps the .travis.yml env section would help:

env:
  global:
  # SAUCE_USERNAME
  - secure: S/6Ild1n9SmW8xYqarjkigDqmyFgWS4Hi0M07e/ryPeS1MNQdiS5+WuSmlSTevM/wU/oSCvzXnKhO9W/1tvz4qj/BMQYxYWFjwWAXJrkjJPvWoa5ogYkR3ETSYrqdhkT+8LVof8iLPlpHjS1y2fozyJbYf1YNSkcVCLxMdIwnk0=
...

but got same output on both.

I'm doing this with latest version 2.8.2.

How do I get the functional tests on SauceLab to run?

[BenFradet]

When opening a PR, the tests should run with the correct credentials, unless I'm mistaken?

[jbeemster]

Hey @tkalfigo @BenFradet only PRs from within Snowplow get access to those credentials. Otherwise you could have someone open a PR with malicious changes to dump those credentials to plain text.

For more information: https://docs.travis-ci.com/user/environment-variables/#Defining-encrypted-variables-in-.travis.yml

I believe this issue can be closed?

[alexanderdean]

Agree, closing

[christoph-buente]

How do you expect contributors test their implementation before sending a pull request? Could you provide a wiki document, how to setup a saucelabs account and how to configure all moving parts? We do not feel confident sending a PR withouth prior successfully running the test suite. @alexanderdean, what do you think?

[alexanderdean]

Good idea, re-opened and renamed ticket.

[chuwy]

Some initial documentation is already on wiki: https://github.com/snowplow/snowplow/wiki/4-For-developers#54-tests, though requiring step-by-step guide and some rewording.

[tkalfigo]

@chuwy I don't see how that wiki page helps devs who want to run the tests on a fork of the project as @christoph-buente requested. Also it seems outdated or inaccurate. I don't see a file tests/integration/request_recorder.js mentioned therein. And of course doesn't explain what would need to be adopted to run the tests, locally or through travis (e.g. AWS and NPM deployment among others I see in travis.yml).