sstephenson / bats

Bash Automated Testing System
MIT License
7.12k stars 518 forks source link

Bats swallows output in tests #191

Closed mttkay closed 7 years ago

mttkay commented 7 years ago

Because of https://github.com/sstephenson/bats/issues/190, I tried to do the obvious and print $output to standard out, but echo won't actually print anything:

@test "where's my output" {
  echo "echo"
}

this prints nothing to standard out.

Am I missing something? How do you inspect $output etc. when a test fails?

ztombol commented 7 years ago

This is intentional. From the wiki:

Anything written to stdout or stderr in setup, teardown, or a test function is captured by Bats. If the test case fails, the error trap prints the output to the TAP stream as a comment. Outside a test case, though, stdout is the TAP stream, so you'll want to specifically redirect any output to stderr.

You can use this to show relevant information on failure.

@test 'test-a' {
  run bash -c 'echo ERROR; false'
  echo "status = ${status}"
  echo "output = ${output}"
  [ "$status" -eq 0 ]
}

Bats will show all output made before the failing assertion. In this case, $output and $status.

 ✗ test-a
   (in test file /tmp/test.bats, line 5)
     `[ "$status" -eq 0 ]' failed
   status = 1
   output = ERROR

Aslo see my answer to your previous question #190.

mttkay commented 7 years ago

Thanks for the quick reply! This is really helpful 👍 Why not make this a bit more visible and/or add your example to the wiki? I'm sure I'm not the first one to run head first into this.

Closing this.

Zearin commented 7 years ago

Agreed!

This issue is awesome for:

The only thing that’s missing is a high-visibility place for other newcomers to see the solution before they have the same problem.

STDOUT and STDERR are important for just about any program that’s likely to use BATS. I think it’s worth doing the equivalent of a “pull quote” from the normal docs. :)

ztombol commented 7 years ago

Yes, this should be more visible. Along with a dozen other gotchas and best practices. But the maintainer does not have enough time. See #150. A number of us volunteered to help out, but no one serious has stepped forward to take over maintenance. I'm not blaming anyone. After all, I'm still in debt with my long list of proposed improvements that I promised. Bats is an important piece of many projects' infrastructure. Too bad that it does not receive enough attention.