Zearin
commented
7 years ago FANTASTIC post!
@ztombol: Keep up the excellent work—not just in the code, but in top-notch posts like this to rally the troops. Excellent technical writing like this is rare (probably more than it should be!), but powerful.
I currently don’t have time to contribute—but if and when that changes, I will certainly sign up for this.
Until then, if you’re reading this and have the time, please contribute! Bats improves life for anyone developing or using bash scripts and programs everywhere. But its benefits go beyond the borders of current bash code; all the “glue” scripts written in bash-flavored shell scripts benefit as well.
Bats also makes bash a more attractive place to program, as other languages have multiple mature testing tools and frameworks at their disposal. It’s long past time that the most popular and widespread dialect of shell scripting had something similar!
Let’s iron out the remaining wrinkles of bats and improve the coding hygiene of bash developers everywhere. ☻
vanosg
Sylvain303
btamayo
jasonkarns
What is this?
This is a list containing popular feature requests, common misconceptions and long standing bugs. All of which should be addressed on the way to
v1.0.Not all of these will become part of Bats. Some may be rejected, some may become a footnote in the documentation, while some may be compiled into a best practice guide. What they have in common is that they are all popular and recurring topics, and so they must be addressed in some form.
You can contribute by:
All in all, this is an effort to clean-up the issue tracker as well. Also see #150.
The list
Documentation
Topics in which the documentation should be expanded.
[ ] Add an example on the proper use of
runwith piped commands.[ ] Add warning and workaround for
[[not triggering test failure on some Bash versions.[ ] Add warning about test descriptions (i.e. function names) having to be unique.
[ ] Add link to
bats-*helper libraries.[ ] Add detailed description of how Bats processes and executes tests.
The wiki entry is a good start, but without diving into the source, users don't understand why a
foraround a test case can't be used to programmatically generate tests, or what's the difference between doing something in global scope or setup. This should be a part of the official documentation, not just the wiki. ("I addressed this problem somewhat in #136." -- @ztombol)Fixes
Bugs that should be fixed.
[ ] Empty output lines are missing from
${lines[@]}.[ ] Use POSIX compliant options with
headfor better compatibility (e.g. Busybox support).[ ] Incorrect TAP format for skipped tests.
[ ]
set -o nounsetbreaksbats-exec-test.[ ]
bats-preprocessbreaks when translating a@testline that uses tabs for white space, instead of plain spaces.[ ] Bats failing on OS X 10.11.2.
Features
Popular feature requests.
[ ] Capture standard output and standard error separately as well, in addition to the combined output.
[ ] Running only a specific set of tests.
Tests may be selected by a regular expression matching their description.
Using numbers may be more convenient in some cases. For example, when running tests on Travis CI, Bats automatically uses TAP format output, which includes the test numbers. Using only the number of failing tests speeds up debugging and testing fixes. Tests may be specified as a list of numbers, e.g.
6,8, number ranges1-4,10-13, or the mixture of both, e.g.1-4,6,8,10-13.[ ] Create and document new public variables that can be used in
teardownto determine whether the test has succeeded, failed or has been skipped.This can be used to preserve test artefacts on failure to help debugging, e.g. what
bats-file'stemp_deldoes.This most likely involves making
BATS_TEST_COMPLETEDandBATS_TEST_SKIPPEDpublic.[ ] Log output or TAP output to a file.
[ ] Test files including other test files.
[ ] Per file
setupandteardownfunctions that are run only once per test file.May look something like Perl's interface.
startup- before the first testsetup- before every testteardown- after every testshutdown- after the last test[ ] Stop
load <path>from automatically appending.bashwhen<path>is relative.While files with arbitrary extension can be loaded with using the absolute path
BATS_TEST_DIRNAME. Consistent behaviour in case of absolute and relative paths would be easier to use.