Closed bcomnes closed 2 years ago
thanks @bcomnes - interesting I thought spec did but dot didn't - the idea is to sort of have a sort of spectrum from coarse to fine grained output via reporters - e.g use dot for absolute minimalism, if a test fails, maybe switch to spec (get some failure output) if you need more than that, switch to tap reporter for full visibility (e.g. any t.comment calls you've dotted around will show up etc)
Got around to looking into this a little more, and may come down to a misunderstanding I have about TAP. The spec reporter does have a diff output, but the TAP being generated by brittle doesn't include a diff block, and tap generated by node-tap does.
So the reporters show a diff, but only if the TAP contains a diff.
tap-test % npm test
> scratch@0.0.0 test
> tap --reporter tap
TAP version 13
not ok 1 - tap.test.js # time=16.969ms
---
env: {}
file: tap.test.js
timeout: 30000
command: /opt/homebrew/Cellar/node/17.2.0/bin/node
args:
- tap.test.js
stdio:
- 0
- pipe
- 2
cwd: /Users/bret/scratch/tap-test
exitCode: 1
...
{
# Subtest: exports
not ok 1 - a failed test
---
compare: ===
at:
line: 4
column: 5
file: tap.test.js
type: Test
stack: >
Test.<anonymous> (tap.test.js:4:5)
Object.<anonymous> (tap.test.js:3:5)
Module.replacementCompile (/Users/bret/Developer/node-tap/node_modules/append-transform/index.js:60:13)
Object.<anonymous> (/Users/bret/Developer/node-tap/node_modules/append-transform/index.js:64:4)
source: |
tap.test('exports', async (t) => {
t.equal('beep', 'boop', 'a failed test')
----^
t.test('subtest', async (t) => {
t.equal('beep', 'boop', 'a failed test')
diff: |
--- expected
+++ actual
@@ -1,1 +1,1 @@
-boop
+beep
...
# Subtest: subtest
not ok 1 - a failed test
---
compare: ===
at:
line: 6
column: 7
file: tap.test.js
type: Test
stack: >
Test.<anonymous> (tap.test.js:6:7)
Test.<anonymous> (tap.test.js:5:5)
Object.<anonymous> (tap.test.js:3:5)
Module.replacementCompile (/Users/bret/Developer/node-tap/node_modules/append-transform/index.js:60:13)
Object.<anonymous> (/Users/bret/Developer/node-tap/node_modules/append-transform/index.js:64:4)
source: |2
t.test('subtest', async (t) => {
t.equal('beep', 'boop', 'a failed test')
------^
})
})
diff: |
--- expected
+++ actual
@@ -1,1 +1,1 @@
-boop
+beep
...
1..1
# failed 1 test
not ok 2 - subtest # time=3.607ms
1..2
# failed 2 of 2 tests
not ok 1 - exports # time=15.024ms
1..1
# failed 1 test
# time=16.969ms
}
1..1
# failed 1 test
# time=126.918ms
----------|---------|----------|---------|---------|-------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------|---------|----------|---------|---------|-------------------
All files | 0 | 0 | 0 | 0 |
----------|---------|----------|---------|---------|-------------------
Spec reporter output with diff in the TAP
So we can close this out. Do you want to add diff generation to brittle TAP?
Ran into this issue the other weekend, didn't get a chance to dig in, but I figured I should open an issue sooner than later..
Basically, on a test failure when running with spec or dot reporters, it only lists the test that failed, not the expected/actual value. It's in the tap report, but the other reporters aren't including that.
I can look into it more soon, but if anyone knows a quick fix in the meantime, feel free.