search

nickjj / rolespec

A test library for testing Ansible roles
GNU General Public License v3.0
232 stars 20 forks source link

Why `assert_ansible_run()` doesn't assert 0 failed tasks? #41

Open ganto opened 8 years ago

ganto commented 8 years ago

When running a playbook through assert_playbook_runs and the playbook aborts with a failed task rolespec doesn't return an error. In my naive understanding assert_ansible_run() should result in an error. Otherwise nothing is asserted and the name is wrong.

For an example check this Travis run.

The used rolespec test listed the following assertions:

setup_ansible

[...]
#
# Rolespec
#

assert_playbook_runs

assert_group acmetiny
assert_user_in_group acmetiny acmetiny

assert_path         /etc/ssl/acme-tiny/example.com/example.com.key
[...]

Did I miss something?

ypid commented 8 years ago

I came a cross a similar issue I think. It seems that Rolespec only evaluates edit code 1 as error? Also, I was able to use a workaround as shown here: https://github.com/debops/test-suite/blob/3fa5b31099ed5159fec97d480857268eca6c097e/ansible-redis/test#L42-L47

My issue was that shellcheck exists with code 2 when the tested file can not be accessed and rolespec did not fail. I can confirm that the fail command of Rolespec does what it claims to do :wink: