search

tape-testing / faucet

human-readable TAP summarizer
MIT License
551 stars 25 forks source link

Does not report skipped tests? #8

Open davidmason opened 9 years ago

davidmason commented 9 years ago

I am piping output from tape 4.2.0 into faucet, I'm not sure if this issue is caused by tape output or by faucet.

According to what appears to be the tap spec, lines like this should count as skipped:

ok 23 # skip Insufficient flogiston pressure.

If the whole test file succeeds, the count of skipped tests is included in the generated output. The harness should report the text after # SKIP\S*\s+ as a reason for skipping.

Tape does not appear to output anything for skipped tests, and skipped assertions have # SKIP appended to the message rather than prepended.

faucet does not display tests that have skipped assertions any differently, even if I edit the output to make the skipped test lines look like the one in the spec (move # skip to just after the assertion number) and has nothing in the final lines about skipped tests.

awlayton commented 8 years ago

@davidmason tape is doing the right thing (or at least a compliant thing) for skipped tests.

From the TAP spec:

To summarize: - ok/not ok (required) - Test number (recommended) - Description (recommended) - Directive (only when necessary)

There can still be a description before the directive. You can also have an explanation/reason after the start of the directive which tape does not support, but the output is still spec compliant.

I would also like faucet to report my tests with skip (and todo) directives in some way.

awlayton commented 8 years ago

It seems that now it reports skip tests as passing, which I _really_ want to stop...