Closed jotaen4tinypilot closed 8 months ago
Sure, this sounds good.
It's hard for me to say no to more tests! And as we write more bash scripts, it would be nice to have more automated testing of them.
It looks pretty lightweight and straightforward, and doing it in conjunction with #1710 sounds like a great test of it.
In the spirit of potential innovation (a topic we discussed in a recent dev meeting, related to us introducing hurl in the same spirit), I wanted to bring up the idea of writing automated tests for bash scripts.
I’ve used a bash testing utility called bats in one of my side-projects (more on that below), and found that it works somewhat nicely. (Well, as nice as it gets on bash.)
Tests with bats are basically shell script files that follow a specific notation/structure. Say, your tool under test is called
my-cmd
, and you wanted to test what invokingmy-cmd --some-flag
yields, you’d do:For executing the tests, you’d store that snippet in a file (e.g.,
test.bats
) and runbats tests.bats
on the terminal.bats in a nutshell:
.bats
extension)#@test
annotation comment@bats
prefix, though then you break out of the regular bash realm, so e.g. you cannot shellcheck anymore.run
. E.g., for a command/scriptmy-cmd
, you’d dorun my-cmd
.$output
and$status
variables, which you can assert on via regular shell conditionals.The side-project, in which I used bats, is a task runner CLI tool implemented in bash script. The full test-suite is in this folder; for isolation, I run the entire suite inside a disposable docker container, so I can safely mess with the file system. Note that the tool under test is also called
run
(like the bats built-in for invoking the command under test), which might cause confusion; therefore, my tool is aliased asmain
in my tests.Discussion
If we are interested in exploring this, I thought that https://github.com/tiny-pilot/tinypilot/issues/1710 could be one possible opportunity that might lend itself for exploration, as it’s relatively simple and self-contained. We have other such scripts, though, and we could also add these tests in hindsight.
One downside that I’d see is that we’d increase the complexity of our tool chain.
There might be other bash testing utilities – which I haven’t looked into, though.