DanHam commented 5 years ago

Version:

test-kitchen (2.2.5)
inspec (4.3.2)
kitchen-inspec (1.1.0)

Environment:

$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.14.5
BuildVersion:   18F132

$ rbenv --version
rbenv 1.1.2

$ ruby --version
ruby 2.5.3p105 (2018-10-18 revision 65156) [x86_64-darwin18]

Scenario:

$ kitchen list
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ClientError
>>>>>> Message: Could not load the 'inspec' verifier from the load path. Did you mean: busser, dummy, shell ? Please ensure that your verifier is installed as a gem or included in your Gemfile if using Bundler.
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

Steps to Reproduce:

$ ruby-build 2.5.3 ~/.rbenv/versions/test-kitchen
$ rbenv shell test-kitchen
$ gem install wirble bundler

Create a Gemfile with the following contents

source 'https://rubygems.org'
gem 'test-kitchen'
gem 'kitchen-ansible'
gem 'kitchen-vagrant'
gem 'kitchen-inspec'

Run:

$ bundle install
$ rbenv rehash
$ gem list --local --quiet tty-box tty-prompt
tty-box (0.3.0)
tty-prompt (0.18.1)

Kitchen runs successfully:

$ kitchen list
Instance                        Driver   Provisioner      Verifier  Transport  Last Action    Last Error
default-centos-kitchen-ansible  Vagrant  AnsiblePlaybook  Inspec    Ssh        <Not Created>  <None>

I have another project using test-kitchen that uses serverspec as the verifier. The Gemfile is as follows:

gem 'test-kitchen'
gem 'kitchen-ansible'
gem 'kitchen-vagrant'
gem 'kitchen-verifier-serverspec'
gem 'serverspec'

After installing with bundler the tty-prompt and tty-box gems have been upgraded:

$ bundle install
$ gem list --local --quiet tty-box tty-prompt
tty-box (0.4.0, 0.3.0)
tty-prompt (0.19.0, 0.18.1)

Now kitchen list in the project using inspec as the verifier errors:

$ kitchen list
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ClientError
>>>>>> Message: Could not load the 'inspec' verifier from the load path. Did you mean: busser, dummy, shell ? Please ensure that your verifier is installed as a gem or included in your Gemfile if using Bundler.
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

If I manually downgrade just one of the upgraded gems, then kitchen runs again without error i.e. it is only when both tty-box 0.4.0 and tty-prompt 0.19.0 are installed that I see an error.

$ gem uninstall tty-box --version 0.4.0
$ gem list --local --quiet tty-box tty-prompt
tty-box (0.3.0)
tty-prompt (0.19.0, 0.18.1)

Uninstalling tty-box 0.4.0 allows kitchen to run normally again.

$ kitchen list
Instance                        Driver   Provisioner      Verifier  Transport  Last Action    Last Error
default-centos-kitchen-ansible  Vagrant  AnsiblePlaybook  Inspec    Ssh        <Not Created>  <None>

Similarly, if I reinstall tty-box 0.4.0, but this time remove tty-prompt 0.19.0, kitchen again runs normally:

$ gem install tty-box --version 0.4.0
$ gem uninstall tty-prompt --version 0.19.0
$ gem list --local --quiet tty-box tty-prompt
tty-box (0.4.0, 0.3.0)
tty-prompt (0.18.1)
$ kitchen list
Instance                        Driver   Provisioner      Verifier  Transport  Last Action    Last Error
default-centos-kitchen-ansible  Vagrant  AnsiblePlaybook  Inspec    Ssh        <Not Created>  <None>

Kitchen runs OK in the project using serverspec as the verifier regardless of what versions tty-box or tty-prompt are at.

Expected Result:

kitchen should be able to run normally with the required gems for running with serverspec and inspec installed.

Actual Result:

tty-box	tty-prompt	kitchen list
0.3.0	0.18.1	OK
0.3.0	0.19.0	OK
0.4.0	0.18.1	OK
0.4.0	0.19.0	FAILS

clintoncwolfe commented 5 years ago

Thanks for the excellent report!

szemkhel commented 5 years ago

I have similar issue. But I can't find working combination at all (within mentioned gems):

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.2 LTS
Release:    18.04
Codename:   bionic

tty-box (0.4.0, 0.3.0)
tty-prompt (0.19.0, 0.18.1)
test-kitchen (2.2.5)
inspec (4.3.2)
kitchen-inspec (1.1.0)

Kitchen failing with:

 >>>>>> ------Exception-------
 >>>>>> Class: Kitchen::ClientError
 >>>>>> Message: Could not load the 'inspec' verifier from the load path. Did you mean: busser, dummy, shell ? Please ensure that your verifier is installed as a gem or included in your Gemfile if using Bundler.
 >>>>>> ----------------------
 >>>>>> Please see .kitchen/logs/kitchen.log for more details
 >>>>>> Also try running `kitchen diagnose --all` for configuration

Failing sets of gems:

tty-box (0.3.0)
tty-prompt (0.18.1)
test-kitchen (2.2.5)

Still failing with:

tty-box (0.4.0, 0.3.0)
tty-prompt (0.19.0, 0.18.1)
test-kitchen (2.2.5)

Or here:

tty-box (0.4.0, 0.3.0)
tty-prompt (0.18.1)
test-kitchen (2.2.5)

and here:

tty-box (0.3.0)
tty-prompt (0.19.0)
test-kitchen (2.2.5)

Dirc commented 5 years ago

For me it is failing also for the exact four cases of @szemkhel , also on Ubuntu 18.04.2 LTS. So far I did not find a working set of gems..

Rollback to test-kitchen version 1.24 resolves the "Could not load the 'inspec' verifier from the load path" issue for me.

test-kitchen (1.24.0) tty-box (0.3.0) tty-prompt (0.18.1) inspec (4.7.3) kitchen-inspec (1.1.0)

inspec / kitchen-inspec