johanneswuerbach / saucie

This library allows you to integrate results of your frontend JavaScript tests into a [Sauce jobs results page](https://saucelabs.com/docs/javascript-unit-tests-integration).
26 stars 18 forks source link

Full test results. #24

Closed AsaAyers closed 7 years ago

AsaAyers commented 9 years ago

I'm using saucie through testem to run tape tests on SauceLabs.

Using my workaround in https://github.com/johanneswuerbach/saucie/issues/5#issuecomment-147855384 I have been able to make my tests run and provide a pass/fail. When this runs I just have a pass/fail for the whole browser:

ok 1 SL_Chrome - https://saucelabs.com/jobs/...
ok 2 SL_IE_11 - https://saucelabs.com/jobs/...

1..2
# tests 2
# pass  2
# fail  0

# ok

Is there any way to get this to report the actual tests?

johanneswuerbach commented 9 years ago

I'm not sure whether this works with tape, but for qunit & mocha this launcher in your testem.json prints all your tests:

"launchers": {
 "SL_chrome": {
   "exe": "./node_modules/.bin/saucie",
   "args": [
     "-b", "chrome", "--at", "--u"
   ],
   "protocol": "browser"
 }
}
AsaAyers commented 9 years ago

If you can help point me to how saucie passes the results back to testem, I can help make it work with the tap protocol. Maybe I'm missing something, but it seems like it only cares about passed or failedCount https://github.com/johanneswuerbach/saucie/blob/master/lib/sauce-update-job-status.js#L34-L39

johanneswuerbach commented 9 years ago

Saucie isn't returning the results back to testem, but is actually collecting them itself when running with protocol: browser.

You can find a basic example here: https://github.com/airportyh/testem/blob/master/views/taprunner.mustache#L11-L16