Open dragon788 opened 6 years ago
Upon reflecting on this I also realized that you need to use an additional $(create_mock)
call for each binary you want to stub, otherwise you won't be tracking the right number of calls and args, so making a helper function like what jasonkarns/bats-mock uses for stub
is probably the way to go.
@dragon788 thanks very much for your feedback. I was thinking about possibilities of mocking executables by their real names myself. And yes, symlink-ing + PATH manipulations occurred to me as well. However, the issue with this approach is that the executable can be specified with a path component (like /usr/local/bin/psql
) in the script and so the idea breaks here. Not speaking about possible PATH manipulations by the script itself.
I have a hint of idea of solving that by chroot-ing into a disposable tree and doing unsafe things there but nothing is clear here yet, so I'm not sure if it's going to be the solution or there will be something else. I was a little bit hesitant about this enhancement, but after your message I started seeing a clear demand in it, so I scheduled work on it in terms of v2.0.
Was interesting checking the implementation of stub.sh after discovering it in the question referenced above. https://github.com/jimeh/stub.sh/blob/master/stub.sh Haven't fully digested whether it can handle full paths and still stub the binary or not.
@dragon788 interesting tool, thank you. However, I'm a little bit afraid of stub/restore approaches. From my previous experience it's an easy way to shoot oneself in the foot.
I definitely think your idea of doing it in a chroot is the safest route, that way it can be destroyed between tests to prevent side effects and can't alter the host and cause any issues.
I just found this bat-mock
tool and it looks very promising for unit testing bash code. I wonder if there is any progress with this issue?
@pszalko unfortunately it appeared to be harder that though and I just don't have enough time currently to address it properly. However, I would highly appreciate any PRs and/or ideas.
One idea and what I have used in tests is to create a function that overrides the actual executable. Something like:
@test "my override test" {
mock_curl="$(mock_create)"
curl() {
"${mock_curl}" "$@"
}
run something_that_call_curl
[ "$(mock_get_call_num "${mock_curl}")" -eq 1 ]
}
In fact I had been using just regular functions to do all my mocking for quite a while until I ran into issues of not being able to get call counts or easily access multiple calls arguments.
Solution from @dampcake is not working for me. It seems that function defined in @test is not passed to run: my script, which calls curl, takes it from system instead of mock.
To verify it, use another non-existing command (e.g. carl): myscript.sh:
#!/usr/bin/env bash
carl --help
exit 0
and test:
@test "test mocked curl" {
mock_curl="$(mock_create)"
carl() {
"${mock_curl}" "$@"
}
run myscript.sh
[ "$(mock_get_call_num "${mock_curl}")" -eq 1 ]
}
When running test I get:
myscript.sh: line 3: carl: command not found
Did you create the carl script in a folder that is present in the PATH variable? Some distributions include the current directory (.) in the PATH, that has become less common since it can be a security issue, it is better to temporarily replace or extend the PATH variable explicitly for your test execution.
This is the complete example with curl. Both files are in the same dir. myscript.sh:
#!/usr/bin/env bash
curl
exit 0
test.bats:
#!/usr/bin/env bats
setup() {
# https://bats-core.readthedocs.io/en/stable/
# modules installed with nvm and global npm:
# npm install -g bats-support
# npm install -g bats-assert
load ${NVM_BIN}/../lib/node_modules/bats-support/load.bash
load ${NVM_BIN}/../lib/node_modules/bats-assert/load.bash
# module installed with:
# https://github.com/grayhemp/bats-mock#installation
# ./build install
load /usr/local/lib/bats-mock.bash
# get the containing directory of this file
# use $BATS_TEST_FILENAME instead of ${BASH_SOURCE[0]} or $0,
# as those will point to the bats executable's location or the preprocessed file respectively
DIR="$( cd "$( dirname "$BATS_TEST_FILENAME" )" >/dev/null 2>&1 && pwd )"
# make executables in . visible to PATH
PATH="$DIR:$PATH"
}
@test "test mocked curl" {
mock_curl="$(mock_create)"
mock_set_output ${mock_curl} "this is the output from mock"
curl() {
"${mock_curl}" "$@"
}
# run something_that_call_curl
echo "-- running myscript.sh" >&3
run myscript.sh
echo "${output}" >&3
# this check will fail
#[ "$(mock_get_call_num "${mock_curl}")" -eq 1 ]
# run local function
echo "-- running local function" >&3
run curl
echo "${output}" >&3
# this check will succeed
[ "$(mock_get_call_num "${mock_curl}")" -eq 1 ]
}
Myscript is not seeing the mocked version of curl, but the system real version of it. As expected, direct run of curl function from test.bats is working fine:
$ bats test.bats
✓ test mocked curl
-- running myscript.sh
curl: try 'curl --help' or 'curl --manual' for more information
-- running local function
this is the output from mock
1 test, 0 failures
Would using a symlink to the mocked version of curl and prepending it to the PATH inside the test work?
@test "test mocked curl" {
mock_curl="$(mock_create)"
mock_set_output ${mock_curl} "this is the output from mock"
ln -s "${mock_curl}" $BATS_RUN_TMPDIR/curl
PATH="$BATS_RUN_TMPDIR:$PATH"
# run something_that_call_curl
echo "-- running myscript.sh" >&3
run myscript.sh
echo "${output}" >&3
# this check should pass now
[ "$(mock_get_call_num "${mock_curl}")" -eq 1 ]
# run local function
echo "-- running local function" >&3
run curl
echo "${output}" >&3
# this check will succeed
}
PR #20 contains a tentative implementation (and would also solve #17).
It doesn't use chown
but it is good enough for my test setup.
Restriction to PR #20: it doesn't work for testing shell scripts with hard-coded absolute paths.
@cescp I am pretty sure you will need to do an export -f curl
after you declare it as a mock function in order for it to be usable in the downstream script under test. You can avoid that by sourcing the script you are testing, but in this case i think exporting the mocked function makes more sense
I love the syntax of your implementation and it makes a lot of sense to me that the mock is a test double/spy that watches all the invocations and allows you to assert that it was called in specific ways, without necessarily having to write out an assertion for every single call.
While trolling StackOverflow and related sites I came across a question about mocking a script and checking the calls using
bats-mock
, but they were using the other version and it doesn't make it quite as easy to assertmock_called_with_args
or similar, so I wrote up an answer with an example using your bats-mock, but I was having trouble emulating thestub
behavior of actually shadowing/preempting a binary in the PATH so that the${mock}
definition was called instead. https://stackoverflow.com/q/38315185/3794873I eventually came up with a hacky method by doing the mock creation in the
setup()
method or in each test and then by making a symlink to the mock's file path in the same directory and prepending that directory to the PATH variable and then unlinking and stripping the location back out of the PATH in the teardown or the end of the test. Is there a cleaner way to do this without repeating the code across all of the tests and in such a way that it allows for one or more binaries to be stubbed, in the case you want to avoid any external resources being accessed and you could stub success for the list?Perhaps this is something that could be added to this better maintained version of bats-mock? Maybe something like
mock_set_command_stub_name
.