Adding `assert_equals_golden` and `assert_output_equals_golden`

[CauhxMilloy]

Adding support for "golden files". Golden files are (often text) files that are checked into a repo and are used in testing for asserting strings against their content. They have a number of useful properties that make them a great alternative to single (possibly repetitive) assertions in files. Some benefits include:

• WYSIWYG plaintext output assertions (separating asserted output from test case logic).
• Test failure output that is searchable (asserted output is checked into repo as files).
• Clear file diffs of test assertions during merge / code review.
• Terse assertions in test cases (single assert instead of many verbose assert_line and refute_line for every line of output).
• Reusable golden files (declared once, used for many test cases).
• Clear intention (same exact expected output) when asserted against same goldens in multiple test cases.
• Can be clearly diff'd across multiple lines in failure message(s).
• Easily updated.

This PR adds support to bats for golden files with the assert_equals_golden and assert_output_equals_golden test helper functions. assert_equals_golden takes 2 arguments, and , to assert is the same as the contents in . assert_output_equals_golden takes 1 argument, , to assert output is the same as the contents in .

These functions support a number of features, including --regexp and automatic updating of golden files (via setting the BATS_ASSERT_UPDATE_GOLDENS_ON_FAILURE environment variable). Golden file assertions will properly account for empty lines or trailing newlines (e.g. when asserting against output obtained by run --keep-empty-lines).

This PR adds assert_equals_golden.bats which contains test cases for the newly added functionality.

Note that the output of these functions (that is asserted on in tests) has incorrect "lines" count. This is due to an incompatibility between run --keep-empty-lines and how various bats-support helper functions work. See https://github.com/bats-core/bats-support/issues/11.

[jasonkarns]

Is there a strong case for having a dedicated matcher instead of using $(cat fixtures/expected.txt) with whatever assertion helper best fits?

I recognize some other conveniences still need built out (better diffing with the assert_output, and more consistent options like partial/regex for assert_equals.

But I would propose we beef up the core assertions and then use them for what is still a text matching assertion.

If we do want some additional helper, I could see a fixture helper of some kind that had conventional pathing built-in. But the assertion side of the equations doesn't feel dissimilar enough from the core methods to necessitate a new one. (Some evidence to support this is that many other test utilities in other frameworks have plenty of helpers for fixtures in general; but none that I've seen ever combine fixture support with the matcher.)

[CauhxMilloy]

Using $(cat expected.txt) has in fact been something that I've used quite a bit and was the motivator to create these functions. Handling trailing new lines, protecting against authoring errors, etc, leads to a large chunk of code at every assertion. Having that wrapped up into a simple function really does help clean up the actual test case logic (easier to see what is actually tested). In addition to that, having the ability to automatically update goldens (i.e. when output purposefully changes) is really useful as a codebase changes over time (exhaustive assertions, instantly updated). Being able to get that functionality, but very succinctly, is really nice.

I made these into separate assertions to both allow for clarity of how the assertion is done and to keep existing functions from being bloated. When I had added in bats_pipe (https://github.com/bats-core/bats-core/pull/663, me from another account), I had started off making it part of run. It then got moved out into its own helper. assert_output already has many bells and whistles (and can actually lead to some authoring errors when accidentally used slightly incorrectly, but that's a separate topic) and I didn't want to lump this golden file functionality on top. Given that there's many flavors of asserts in bats, each to simplify writing of tests, I felt like this would follow suit. Can this be handled by combinations of other functions? Maybe (but not as clearly/nicely), and the same goes for "cant assert_output 'want' just be assert_equals "$output" 'want'?".

I've seen libraries with golden file support and sometimes automatic updates (which must be based on the text asserted) is done via built-in functionality or via scripts. But it's usually a common desire when working with them.