Failing tests don't output the actual value

[mttkay]

If you have a failing test, there isn't really anything actionable in the test output to go on, since the only thing bats recognizes is that one of the assertions failed. So it will print something like:

`[ "$status" -eq 0 ]' failed

But what was the actual status?

Since the run function already captures the output of the given command, is it possible to pass this along and report it together with an "assertion"?

Maybe this is more something for an "assertion functions" layer on top of bats? In any way, I would expect from a test suite to be able to tell me what output the unit under test produced.

[ztombol]

Bats is intentionally kept simple. Only the developer knows what information is useful in debugging a given test case. Showing everything automatically would be just as bad.

So if you want feedback on a failing test, you have to make that happen yourself. Which is pretty easy, given the variables Bats exposes. For example:

@test 'test-a' {
  run false
  if [ "$status" -ne 0 ]; then
    echo "ERROR: status = $status"
    return 1
  fi
}

This actually came up many times before. With help from others, I wrote libraries that implement common assertions and provide useful feedback when they fail. See #110 for its design.

• bats-assert - common assertions
• bats-file - filesystem related assertions

See also bats-docs for more information.

[mttkay]

Ah great--this is exactly what I was looking for. 👍 So these projects are unrelated to the core project? Again, linking to them from the main README would be ace, since adding proper assertions is probably the first thing users would want to do.

Only the developer knows what information is useful in debugging a given test case. Showing everything automatically would be just as bad.

Every unit test framework I've ever used had some standard way of informing you what the failing result actually was. But I completely agree this can or should be factored out into an assertions module, it's just that it's kinda hard to discover this when turning to bats for the first time as there's no mention of it anywhere here.

Anyway, thanks for pointing it out!