qunitjs / js-reporters

📋 Common Reporter Interface (CRI) for JavaScript testing frameworks.
MIT License
60 stars 18 forks source link

Fix QUnit adapter #86

Closed flore77 closed 8 years ago

flore77 commented 8 years ago

This fixes the Suite instantiation when there is no global module.

vedharish commented 8 years ago

@flore77

I think there is also an issue with the number of tests reported by the qunit adapter.

I am running these qunit sample tests on BrowserStack using the master branch of browserstack-runner and fix-qunit-adapter branch of js-reporters. I changed the test to make it fail and tested on only 2 browsers. Here is the diff

diff --git a/browserstack.json b/browserstack.json
index 39809c5..45ea097 100644
--- a/browserstack.json
+++ b/browserstack.json
@@ -11,22 +11,10 @@
             "os_version": "Lion"
         },
         {
-            "browser": "safari",
-            "browser_version": "latest",
-            "os": "OS X",
-            "os_version": "Mountain Lion"
-        },
-        {
             "browser": "chrome",
             "browser_version": "latest",
             "os": "OS X",
             "os_version": "Mountain Lion"
-        },
-        {
-            "browser": "firefox",
-            "browser_version": "latest",
-            "os": "Windows",
-            "os_version": "7"
         }
     ]
 }
diff --git a/tests/test.js b/tests/test.js
index 6fbea02..02e5109 100644
--- a/tests/test.js
+++ b/tests/test.js
@@ -1,7 +1,7 @@
 QUnit.module('Odd Tests');

 QUnit.test('isOdd()', function(assert) {
-  assert.ok(isOdd(1), 'One is an odd number');
-  assert.ok(isOdd(3), 'Three is an odd number');
-  assert.ok(!isOdd(0), 'Zero is not odd number');
+  assert.ok(isOdd(10), 'One is an odd number');
+  assert.ok(isOdd(30), 'Three is an odd number');
+  assert.ok(!isOdd(11), 'Zero is not odd number');
 });

Here is the screenshot of the test on master branch of browserstack-runner Here is the screenshot of the test on v0.4.4 tag of browserstack-runner As we can see, both the screenshots show 0 assertions of 3 passed, 3 failed, but the data posted to the _report endpoint is -

For the master branch - {"runtime":6,"total":1,"passed":0,"failed":1,"url":"/sample.html"}

For the v0.4.4 branch - {"failed":3,"passed":0,"total":3,"runtime":20,"url":"/sample.html"}

Also the data posted to _progress endpoint are of this form -

For the master branch - {"actual":false,"expected":true,"message":"Three is an odd number","source":" at Object.<anonymous> (http://localhost:8888/tests/test.js:5:10)"}

For the v0.4.4 branch - {"actual":false,"expected":true,"message":"Three is an odd number","source":" at Object.<anonymous> (http://localhost:8888/tests/test.js:5:10)","testName":"Odd Tests: isOdd()"}

Can you do something about the testName key? We used that to identify QUnit tests

flore77 commented 8 years ago

As we can see, both the screenshots show 0 assertions of 3 passed, 3 failed

It says assertions not tests, js-reporters counts tests, there is only one test, so what js-reporters reports is correct. I found it more consistent this way, because only for QUnit you were showing assertions and for the others frameworks you were showing tests. If you really want to display the count of assertions, we have introduced a new property called assertions, which contains all the assertions emitted by Qunit.

The testName will not appear, because we have changed the property from testName to name, after the release, the reporter inside browserstack will need some update, which I will be more than happy to provide 😃

vedharish commented 8 years ago

@flore77 Nice! Got it. Thanks!

jzaefferer commented 8 years ago

👍