Closed flore77 closed 8 years ago
@flore77
I think there is also an issue with the number of tests reported by the qunit adapter.
I am running these qunit sample tests on BrowserStack using the master
branch of browserstack-runner
and fix-qunit-adapter
branch of js-reporters
. I changed the test to make it fail and tested on only 2 browsers. Here is the diff
diff --git a/browserstack.json b/browserstack.json
index 39809c5..45ea097 100644
--- a/browserstack.json
+++ b/browserstack.json
@@ -11,22 +11,10 @@
"os_version": "Lion"
},
{
- "browser": "safari",
- "browser_version": "latest",
- "os": "OS X",
- "os_version": "Mountain Lion"
- },
- {
"browser": "chrome",
"browser_version": "latest",
"os": "OS X",
"os_version": "Mountain Lion"
- },
- {
- "browser": "firefox",
- "browser_version": "latest",
- "os": "Windows",
- "os_version": "7"
}
]
}
diff --git a/tests/test.js b/tests/test.js
index 6fbea02..02e5109 100644
--- a/tests/test.js
+++ b/tests/test.js
@@ -1,7 +1,7 @@
QUnit.module('Odd Tests');
QUnit.test('isOdd()', function(assert) {
- assert.ok(isOdd(1), 'One is an odd number');
- assert.ok(isOdd(3), 'Three is an odd number');
- assert.ok(!isOdd(0), 'Zero is not odd number');
+ assert.ok(isOdd(10), 'One is an odd number');
+ assert.ok(isOdd(30), 'Three is an odd number');
+ assert.ok(!isOdd(11), 'Zero is not odd number');
});
Here is the screenshot of the test on master branch of browserstack-runner
Here is the screenshot of the test on v0.4.4 tag of browserstack-runner
As we can see, both the screenshots show 0 assertions of 3 passed, 3 failed
, but the data posted to the _report
endpoint is -
For the master branch - {"runtime":6,"total":1,"passed":0,"failed":1,"url":"/sample.html"}
For the v0.4.4 branch - {"failed":3,"passed":0,"total":3,"runtime":20,"url":"/sample.html"}
Also the data posted to _progress
endpoint are of this form -
For the master branch - {"actual":false,"expected":true,"message":"Three is an odd number","source":" at Object.<anonymous> (http://localhost:8888/tests/test.js:5:10)"}
For the v0.4.4 branch - {"actual":false,"expected":true,"message":"Three is an odd number","source":" at Object.<anonymous> (http://localhost:8888/tests/test.js:5:10)","testName":"Odd Tests: isOdd()"}
Can you do something about the testName
key? We used that to identify QUnit
tests
As we can see, both the screenshots show 0 assertions of 3 passed, 3 failed
It says assertions not tests, js-reporters counts tests, there is only one test, so what js-reporters reports is correct. I found it more consistent this way, because only for QUnit you were showing assertions and for the others frameworks you were showing tests. If you really want to display the count of assertions, we have introduced a new property called assertions
, which contains all the assertions emitted by Qunit.
The testName
will not appear, because we have changed the property from testName
to name
, after the release, the reporter inside browserstack will need some update, which I will be more than happy to provide 😃
@flore77 Nice! Got it. Thanks!
👍
This fixes the Suite instantiation when there is no global module.