spec, dot reporters not including expected/actual output

[bcomnes]

Ran into this issue the other weekend, didn't get a chance to dig in, but I figured I should open an issue sooner than later..

Basically, on a test failure when running with spec or dot reporters, it only lists the test that failed, not the expected/actual value. It's in the tap report, but the other reporters aren't including that.

I can look into it more soon, but if anyone knows a quick fix in the meantime, feel free.

[davidmarkclements]

thanks @bcomnes - interesting I thought spec did but dot didn't - the idea is to sort of have a sort of spectrum from coarse to fine grained output via reporters - e.g use dot for absolute minimalism, if a test fails, maybe switch to spec (get some failure output) if you need more than that, switch to tap reporter for full visibility (e.g. any t.comment calls you've dotted around will show up etc)

[bcomnes]

Got around to looking into this a little more, and may come down to a misunderstanding I have about TAP. The spec reporter does have a diff output, but the TAP being generated by brittle doesn't include a diff block, and tap generated by node-tap does.

So the reporters show a diff, but only if the TAP contains a diff.

tap-test % npm test

> scratch@0.0.0 test
> tap --reporter tap

TAP version 13
not ok 1 - tap.test.js # time=16.969ms
  ---
  env: {}
  file: tap.test.js
  timeout: 30000
  command: /opt/homebrew/Cellar/node/17.2.0/bin/node
  args:
    - tap.test.js
  stdio:
    - 0
    - pipe
    - 2
  cwd: /Users/bret/scratch/tap-test
  exitCode: 1
  ...
{
    # Subtest: exports
        not ok 1 - a failed test
          ---
          compare: ===
          at:
            line: 4
            column: 5
            file: tap.test.js
            type: Test
          stack: >
            Test.<anonymous> (tap.test.js:4:5)

            Object.<anonymous> (tap.test.js:3:5)

            Module.replacementCompile (/Users/bret/Developer/node-tap/node_modules/append-transform/index.js:60:13)

            Object.<anonymous> (/Users/bret/Developer/node-tap/node_modules/append-transform/index.js:64:4)
          source: |
            tap.test('exports', async (t) => {
              t.equal('beep', 'boop', 'a failed test')
            ----^
              t.test('subtest', async (t) => {
                t.equal('beep', 'boop', 'a failed test')
          diff: |
            --- expected
            +++ actual
            @@ -1,1 +1,1 @@
            -boop
            +beep
          ...

        # Subtest: subtest
            not ok 1 - a failed test
              ---
              compare: ===
              at:
                line: 6
                column: 7
                file: tap.test.js
                type: Test
              stack: >
                Test.<anonymous> (tap.test.js:6:7)

                Test.<anonymous> (tap.test.js:5:5)

                Object.<anonymous> (tap.test.js:3:5)

                Module.replacementCompile (/Users/bret/Developer/node-tap/node_modules/append-transform/index.js:60:13)

                Object.<anonymous> (/Users/bret/Developer/node-tap/node_modules/append-transform/index.js:64:4)
              source: |2
                  t.test('subtest', async (t) => {
                    t.equal('beep', 'boop', 'a failed test')
                ------^
                  })
                })
              diff: |
                --- expected
                +++ actual
                @@ -1,1 +1,1 @@
                -boop
                +beep
              ...

            1..1
            # failed 1 test
        not ok 2 - subtest # time=3.607ms

        1..2
        # failed 2 of 2 tests
    not ok 1 - exports # time=15.024ms

    1..1
    # failed 1 test
    # time=16.969ms
}

1..1
# failed 1 test
# time=126.918ms
----------|---------|----------|---------|---------|-------------------
File      | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
----------|---------|----------|---------|---------|-------------------
All files |       0 |        0 |       0 |       0 |                   
----------|---------|----------|---------|---------|-------------------

Spec reporter output with diff in the TAP

So we can close this out. Do you want to add diff generation to brittle TAP?