events details types

[leobalter]

On the README doc I miss the types for the event details, so I'm assuming:

• Test: object - A test holds basic information on a single test/spec.
  • testName: string - name of the test
  • suiteName: string - name of the suite the test belongs to
  • status: string - result of the test. Can "passed", "failed" or "skipped". A skipped test is disabled, i.e. it will not be executed.
  • runtime: number - execution time in milliseconds
  • errors: array of string only | string - Depending on the framework, this is a single exception or a list of failed assertions. Will be an empty array for statuses other than failed.
• Suite: A suite is a collection of tests and potentially other suites.
  • name: string - name of the suite
  • childSuites: array - array with all direct subsuites (objects or string names?)
  • tests: array - array containing all tests that directly belong to the suite (but not to a child suite)
  • runtime: number - execution time of the whole suite in milliseconds (including child suites)
  • status: object - summarized status of the suite (should it appear on suiteStart?)
  • failed, number - if at least one test failed how many tests failed? from 0 to n?
  • skipped: number: if all tests in the suite are skipped (and there is at least one skipped test) how many skipped tests? from 0 to n?
  • passed: number if there is at least one passed test and all other tests are skipped or if there are no tests in the suite. how many passed tests? from 0 to n?

Should we have an status total as well?

[jzaefferer]

errors is an array of Error objects with (optional?) additional properties, see later part of https://github.com/js-reporters/js-reporters/issues/12#issuecomment-120483356 - need to add that to the spec.

[jzaefferer]

For Suite:

• childSuites: array of Suite
• tests: array of Test
• status: 0 to n sounds right, for all three
• adding status.total as failed + skipped + passed makes sense, since that seems like a pretty common usecase

[jzaefferer]

@fcarstens your help getting this into the spec would be much appreciated!

[jzaefferer]

Btw. the current draft was partially developed in this Google Doc, might still be useful: https://docs.google.com/document/d/1E9X90ODgnMW8bTssIAkfRIQiGX0n0FaGI1vk3geviIs/edit?usp=sharing

[jzaefferer]

Also need to add fullName

[flore77]

I have noticed now this Suite property:

status: object: failed, number skipped, number passed number

Currently status is only a string with values: failed, passed and skipped.

We could make an object with the following properties:

var details = {
status: String,
passed: Number,
failed: Number,
skipped: Number
total: Number
};

The suite property could be called details, status will be a string representing the summarized status, all tests passed, or at least one failed etc, and the other properties will give a proper report of how many passed, how many failed etc.

I think it is valuable information, what do you think @jzaefferer

[jzaefferer]

Making status a string, consistent with Test.status makes sense.

As for the other properties, nesting them seems fine as well. Though details isn't very useful. Since we're counting tests, maybe something like testCounts?

[flore77]

Sounds cool. So we could strap out of the nesting, the status prop, so we will have:

var Suite = {
  name: String,
  ...
  status: String,
  testCounts: {
    passed: Number,
    failed: Number,
    skipped: Number,
    total: Number
  }
};

[flore77]

The testCounts property will count also the tests of the childSuites.

[flore77]

Fixed by #84 and #85.