johanneswuerbach / saucie

This library allows you to integrate results of your frontend JavaScript tests into a [Sauce jobs results page](https://saucelabs.com/docs/javascript-unit-tests-integration).
26 stars 18 forks source link

Support for tap output #13

Closed Joris-van-der-Wel closed 9 years ago

Joris-van-der-Wel commented 9 years ago

I was playing around a bit with this library & testem. I added a launcher to my testem.json:

"launchers": {
    "SL_Chrome_latest": {
      "command": "\"node_modules/.bin/saucie\" -u http://localhost:7357/ -b googlechrome",
      "protocol": "tap"
    },
}

I noticed that I only got a singular pass/fail result in testem, instead of a total count. So I added an argument to saucie to output the results of all test cases in TAP format:

"launchers": {
    "SL_Chrome_latest": {
      "command": "\"node_modules/.bin/saucie\" --tap -u http://localhost:7357/ -b googlechrome",
      "protocol": "tap"
    },
}

This is the first time I am trying out SauceLabs so there might be some mistakes.

I used a library to intercept output (console.log, etc) so that I can that output them as TAP diagnostic lines (beginning with a #) instead. This works for me, but it might not be considered clean enough to merge. I wanted to get this working quickly (at least for now).

Here is a sample of my TAP output:

## Testing tunnel ready
## Started Sauce Connect Process
## Starting run all tests against Sauce Labs
## Updating job status
## not ok
## Closed browser.
## Closing Sauce Connect Tunnel
## Closed Sauce Connect process
1..1145
not ok 1 test case that always fails
ok 2 foo bar test case
ok 3 1 < 2
ok 4 1/0 === Infinity

I am outputting a double # because some diagnostic lines have special meaning in TAP

johanneswuerbach commented 9 years ago

Hey, thanks for this PR. You can get test details already in testem, when you change the protocol from tap to browser.

Example:

"launchers": {
    "SL_Chrome_latest": {
      "exe": "node_modules/.bin/saucie",
      "args": [
        "-b",
        "googlechrome",
        "--at",
        "--u"
      ]
      "protocol": "browser"
    },
}

Does that work for you? If you still need this functionality, could you add tests?

Joris-van-der-Wel commented 9 years ago

Ah, I read a lot of docs but I never came across the "protocol": "browser" option. The purpose of saucie --attach now also makes more sense. This could use some documentation.

It seems to work, I can see the results of the tests in testem. But I did have some issues:

Here's the test session: https://saucelabs.com/tests/5c19dfd257c34cb184798133fc70bd55 (I am using mocha)