neillturner / kitchen-ansible

Ansible Provisioner for Test Kitchen
Other
350 stars 134 forks source link

Per-suite host_vars cause Busser to break during the verify phase #136

Closed craigw closed 8 years ago

craigw commented 8 years ago

Hello!

First of all, thanks for your work on kitchen-ansible, it's kept us sane :)

For one of my Ansible roles, I'm trying to add a per-suite host_vars folder. It's a role that sets up a cluster of several nodes, and I want one test suite for a cluster master, and another suite for a cluster slave. I don't think they need to share the host_vars directory that I use for the slave suite to make sure it connects to a master.

The path lookup code seem stop support what I want to do by adding a directory e.g. test/integration/slave/ansible/host_vars: https://github.com/neillturner/kitchen-ansible/blob/master/lib/kitchen/provisioner/ansible/config.rb#L151-L152. The converge stage works as expected when I create this directory, which supports my feeling that this is what I should do.

Unfortunately, at verify time, any folders in the suite directory will be treated as requested Busser plugins by test-kitchen: https://github.com/test-kitchen/test-kitchen/blob/eecfaa5852baae8c69ae9f317892feebc2e546fe/lib/kitchen/verifier/busser.rb#L235-L241. They'll try to install them as gems of the format busser-<folder_name>. The busser-ansible gem doesn't exist, so Busser fails to run the verifier and spits out an error message:

[...]
       Finished converging <slave-centos-66> (0m15.29s).
-----> Setting up <slave-centos-66>...
       Finished setting up <slave-centos-66> (0m0.00s).
-----> Verifying <slave-centos-66>...
       Preparing files for transfer
-----> Installing Busser (busser)
Fetching: thor-0.19.0.gem (100%)
       Successfully installed thor-0.19.0
Fetching: busser-0.7.1.gem (100%)
       Successfully installed busser-0.7.1
       2 gems installed
       Installing Busser plugins: busser-ansible
       /tmp/verifier/gems/gems/busser-0.7.1/lib/busser/command/plugin_install.rb:67:in `install_plugin_gem': undefined method `version' for nil:NilClass (NoMethodError)
        from /tmp/verifier/gems/gems/busser-0.7.1/lib/busser/command/plugin_install.rb:53:in `install'
        from /tmp/verifier/gems/gems/busser-0.7.1/lib/busser/command/plugin_install.rb:43:in `block (2 levels) in install_all'
        from /tmp/verifier/gems/gems/busser-0.7.1/lib/busser/command/plugin_install.rb:43:in `each'
        from /tmp/verifier/gems/gems/busser-0.7.1/lib/busser/command/plugin_install.rb:43:in `block in install_all'
        from /tmp/verifier/gems/gems/busser-0.7.1/lib/busser/rubygems.rb:71:in `silence_gem_ui'
        from /tmp/verifier/gems/gems/busser-0.7.1/lib/busser/command/plugin_install.rb:42:in `install_all'
        from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/command.rb:27:in `run'
        from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/invocation.rb:126:in `invoke_command'
        from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/invocation.rb:133:in `block in invoke_all'
        from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/invocation.rb:133:in `each'
        from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/invocation.rb:133:in `map'
        from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/invocation.rb:133:in `invoke_all'
        from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/group.rb:232:in `dispatch'
        from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/invocation.rb:115:in `invoke'
        from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor.rb:40:in `block in register'
        from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/command.rb:27:in `run'
        from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/invocation.rb:126:in `invoke_command'
        from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor.rb:359:in `dispatch'
        from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/invocation.rb:115:in `invoke'
        from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor.rb:235:in `block in subcommand'
        from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/command.rb:27:in `run'
        from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/invocation.rb:126:in `invoke_command'
        from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor.rb:359:in `dispatch'
        from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/base.rb:440:in `start'
        from /tmp/verifier/gems/gems/busser-0.7.1/bin/busser:8:in `<top (required)>'
        from /tmp/verifier/bin/busser:23:in `load'
        from /tmp/verifier/bin/busser:23:in `<main>'
>>>>>> Verify failed on instance <slave-centos-66>.
>>>>>> Please see .kitchen/logs/slave-centos-66.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: SSH exited (1) for command: [sh -c '

BUSSER_ROOT="/tmp/verifier"; export BUSSER_ROOT
GEM_HOME="/tmp/verifier/gems"; export GEM_HOME
GEM_PATH="/tmp/verifier/gems"; export GEM_PATH
GEM_CACHE="/tmp/verifier/gems/cache"; export GEM_CACHE
ruby="/usr/bin/ruby"
gem="/usr/bin/gem"
version="busser"
gem_install_args="busser --no-rdoc --no-ri --no-format-executable -n /tmp/verifier/bin --no-user-install"
busser="sudo -E /tmp/verifier/bin/busser"
plugins="busser-ansible"

$gem list busser -i 2>&1 >/dev/null
if test $? -ne 0; then
  echo "-----> Installing Busser ($version)"
  $gem install $gem_install_args
else
  echo "-----> Busser installation detected ($version)"
fi

if test ! -f "$BUSSER_ROOT/bin/busser"; then
  $busser setup
fi

echo "       Installing Busser plugins: $plugins"
$busser plugin install $plugins
']
>>>>>> ----------------------

I suspect that the real fix here is to patch test-kitchen to support an Ansible (or indeed a verifier configurable) style directory structure for each suite, but there are several months worth of outstanding pull requests in the test-kitchen repo. I'll tackle that when I get time to understand how the test-kitchen code is meant to interact with the verifier. I'd love your thoughts on how you'd like it to work, if you have an opinion on it, so I can push in the right direction for that.

Presumably however, this has been done in another way by someone else, and I'm hoping that someone can point me in a suitable direction in the short term. Can you help?

Until I find a better way to work, I do have a sort of work-around: I'm adding the host_vars directory for this suite to my role directory root. That does mean the host_vars are shared with all the test suites - but they still pass, so it's an acceptable work-around for the moment.

Thanks for your time!

Craig

neillturner commented 8 years ago

I have created a verifier called serverspec which bypasses all the busser stuff. See https://github.com/neillturner/kitchen-verifier-serverspec. Currently it runs on the server but could easily be enhanced to support ssh running. I also plan to enhance it to have a runner.rb script that will run and can be customized. So you might find that interesting.
I have update kitchen-verifier-serverspec to support SSH. it will not figure out the roles and host names from the playbook yaml but you should be able to run your tests by specifying the host name and the test.

NOTE: you need to install gem kitchen-verifier-serverspec on your workstation and update the .kitchen.yml file:
i.e.

verifier:
  name: serverspec

suites:
  - name: base
    verifier:
      patterns:
      - roles/tomcat/spec/tomcat_spec.rb
      bundler_path: '/usr/local/bin'
      rspec_path: '/home/vagrant/bin'
      env_vars:
        TARGET_HOST: 172.28.128.7
        LOGIN_USER: vagrant
        SSH_KEY: 'spec/tomcat_private_key.pem'

NOTE: You then does need all the test/integration... stuff as long as your tests are in your ansible repo.

neillturner commented 8 years ago

I've deprecated busser-ansible-spec you can use https://github.com/neillturner/kitchen-verifier-serverspec. and there is a ansiblespec_runner.rb that can be easily customized.

closing for now.