Add optional property for source file to event data (suiteStart or testStart?)

[muthu90ec]

Hi, I have a use case where I want to print the name of the testfile before and after the test run. For eg: if I have two test files testFIle1.js QUnit.test("point1", function(assert){ assert.ok(true); });

testFile2.js QUnit.test("point1", function(assert) { assert.ok(true); });

when I run qunit.js --reporter=myreporter test*.js ...I want the following printed in the console.

testStart=>filename: testFile1.js progress: point1 pass testEnd=>filename: testFile1.js result: pass

testStart=>filename: testFile2.js progress: point1 pass testEnd=>filename: testFile2.js result: pass

[trentmwillis]

The js-reporter itself doesn't know about files, but this would make sense for QUnit to include in the event data it emits.

[Krinkle]

The mechanism for grouping tests in the js-reporter spec language is "suites". Suites can be stacked or nested and may have any arbitrary string as their name.

Personally, I prefer choosing simple strings for suites and write them explicitly in the code. E.g. using module() in QUnit, or describe() in Mocha.

// qunit-example/test/cat.test.js

QUnit.module('Cat', () => {
  test('walk', (assert) => {
    assert.equal(…, …);
  });
});

// qunit-example/test/mouse.test.js

QUnit.module('Mouse', () => {
  test('walk', (assert) => {
    assert.equal(…, …);
  });
});

// mocha-example/test/cat.test.js

const assert = require('assert');

describe('Cat', () => {
  it('walk', () => {
    assert.equal(…, …);
  });
});

// mocha-example/test/mouse.test.js

const assert = require('assert');

describe('Mouse', () => {
  it('walk', () => {
    assert.equal(…, …);
  });
});

However, it would equally be valid to set these to a filename instead. Either manually via descibe( __filename, … ), or through a plugin for QUnit or Mocha that wraps every file it loads implicitly also in an extra suite named after the file.

Or, for perhaps better reuse and flexibility, such plugin could be a proxy JSReporter - in which case it would need to know the source file from the test framework indeed, which this issue asks for.

I think this information should be optional so that JSReporter doesn't require a file-based source for suite and test definitions, which can be hard to track at run-time. For most frameworks this is easy to do on the CLI where the CLI runner for QUnit and Mocha is naturally in charge of loading the JS files that define the tests.

However, in a web browser this is usually not the case. There it tends to be either the end-user declaring the JS files as scriopt in an HTML page, or dynamically through something like Karma runner.

[Krinkle]

I've adjusted the title to propose adding this to testStart or suiteStart. The runStart event is only fired once even if there are multiple test files loaded.

See also current event names.

@muthu90ec Could you describe more about the bigger picture for you and how/why this is useful? Also, do you think this would be better on "suiteStart" or better for every individual test via "testStart"?

[Krinkle]

I'm freezing the CRI event model data for now as per https://github.com/js-reporters/js-reporters/issues/133.

Within TAP (TestAnything protocol), the tools available are test names and comment data. If the information needs to be machine-readable with other reporters and such, then you'd be best off using the equivalent of a "test" or "suite" for this purpose. In other words, consider the file as an implicit "suite" that wraps the tests within.

For the specific use case of doing this with QUnit, I'd recommend using QUnit as described above which is generally how projects do this today already.