rundeck_api_client library LoadError in dokken testing with ChefDK > 2.0.28

[gsreynolds]

With the stable ChefDK 2.1.11, any kitchen-dokken test and so Travis build fails with the below.

-----> Installing Serverspec..
-----> serverspec installed (version 2.40.0)
/opt/chef/embedded/bin/ruby -I/opt/verifier/suites/serverspec -I/opt/verifier/gems/gems/rspec-support-3.6.0/lib:/opt/verifier/gems/gems/rspec-core-3.6.0/lib /opt/chef/embedded/bin/rspec --pattern /opt/verifier/suites/serverspec/\*\*/\*_spec.rb --color --format documentation --default-path /opt/verifier/suites/serverspec
Building native extensions.  This could take a while...
An error occurred while loading /opt/verifier/suites/serverspec/apache_spec.rb.
Failure/Error: load '/opt/kitchen/cookbooks/rundeck/libraries/rundeck_api_client.rb'
LoadError:
  cannot load such file -- /opt/kitchen/cookbooks/rundeck/libraries/rundeck_api_client.rb

# /opt/verifier/suites/serverspec/spec_helper.rb:13:in `load'
# /opt/verifier/suites/serverspec/spec_helper.rb:13:in `rescue in <top (required)>'
# /opt/verifier/suites/serverspec/spec_helper.rb:8:in `<top (required)>'
# /opt/verifier/suites/serverspec/apache_spec.rb:1:in `<top (required)>'
# ------------------
# --- Caused by: ---
# LoadError:
#   cannot load such file -- /tmp/kitchen/cookbooks/rundeck/libraries/rundeck_api_client.rb
#   /opt/verifier/suites/serverspec/spec_helper.rb:10:in `load'

Relevant spec_helper code: https://github.com/sous-chefs/rundeck/blob/b91dddd452fcdbb318763998349d2dff718de1c4/test/integration/default/serverspec/spec_helper.rb#L7-L14

It seems that the kitchen_sandbox directory /opt/kitchen is now not available during the verify phase, which we need for loading the api library. Possibly related to recent kitchen-dokken changes that cleanup kitchen_sandbox?

ChefDK 2.1.11 and kitchen-dokken 2.6.1

Adding output of Dir["/opt/kitchen/*"] to server.rb recipe so it executes during converge phase:

["/opt/kitchen/cache",
 "/opt/kitchen/client.pem",
 "/opt/kitchen/client.rb",
 "/opt/kitchen/clients",
 "/opt/kitchen/cookbooks",
 "/opt/kitchen/data_bags",
 "/opt/kitchen/dna.json",
 "/opt/kitchen/nodes",
 "/opt/kitchen/validation.pem"]

Adding same to spec_helper so it executes during verify phase :

[]

Poking about inside the container shows:

root@dokken:/# ls -lh /opt
ls: cannot access /opt/kitchen: No such file or directory
total 4.0K
drwxrwxr-x 5 root root 4.0K Sep  6 12:13 chef
d????????? ? ?    ?       ?            ? kitchen
drwxr-xr-x 1 1000 ftp   170 Sep  6 13:30 verifier

ChefDK 2.2.1 and kitchen-dokken 2.6.5

Tested locally and looks to be slightly improved in current channel ChefDK 2.2.1 with kitchen-dokken 2.6.5 where at least the directory isn't borked, but is now just empty instead:

root@dokken:/# ls -lh /opt/kitchen/
total 0

ChefDK 2.0.28 and kitchen-dokken 2.5.1

Going back to ChefDK 2.0.28 fixes the problem entirely, and tests pass without the LoadError:

root@dokken:/# ls -lh /opt/kitchen/
total 16K
drwxr-xr-x 1 1000 staff  170 Sep  6 15:20 cache
-rw------- 1 1000 staff 1.7K Sep  6 15:20 client.pem
-rw-r--r-- 1 1000 staff  679 Sep  6 15:20 client.rb
drwxr-xr-x 1 1000 staff  102 Sep  6 15:20 clients
drwxr-xr-x 1 1000 staff  646 Sep  6 15:20 cookbooks
drwxr-xr-x 1 1000 staff  136 Sep  6 15:20 data_bags
-rw-r--r-- 1 1000 staff  241 Sep  6 15:20 dna.json
drwxr-xr-x 1 1000 staff  102 Sep  6 15:20 nodes
-rw-r--r-- 1 1000 staff 1.7K Sep  6 15:20 validation.pem

[gsreynolds]

We can:

1. start a conversation upstream in kitchen-dokken about the issue
2. pin to ChefDK 2.0.28 for now
3. avoid loading cookbook libraries into test-kitchen entirely

or some combination of the above

[gsreynolds]

Related: #179 is an example, first commit was fully green and was tested under ChefDK 2.0.28, a further commit tested with ChefDK 2.1.11 with ChefSpec only changes turned integration tests red whilst UNIT_AND_LINT stayed green. (Though the PR has since been rebased)

[mengesb]

I see ChefDK 2.2.1 now; would this be resolved with that version?

[gsreynolds]

Unfortunately not. 2.1.11 shipped a change in behaviour where the test-kitchen sandbox is deleted after converges, (but also introduced a bug with that behaviour that broke the mount of the sandbox)

2.2.1 ships a bug fix as noted above but the issue here is the new behaviour. We can't load the cookbook library into test-kitchen because the contents of the sandbox are deleted after converge. (sorry for confusing things, I started writing the issue thinking it was the bugged mount before coming across the behaviour change - 2.2.1 is only slightly better because instead of a broken mount it's now an empty directory)

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.