search

chef-boneyard / chef-dk

DEPRECATED: A streamlined development and deployment workflow for Chef Infra platform.
Apache License 2.0
380 stars 170 forks source link

Chef generate cookbook creates tests in a different location than where kitchen-inspec looks for tests #964

Closed ricardolupo closed 7 years ago

ricardolupo commented 8 years ago

Issue Description

The default cookbook generators in chef-dk do not create the necessary scaffolding to work with kitchen-inspec.

By default, kitchen-inspec looks for tests in the cookbooks in these locations, in order of preference:

However, chef generate cookbook lays down tests in /test/recipes/, without the default subdirectory.

Current behavior

In order to run Inspec tests, people must either: 1) manually move the files to the proper directories required to work with kitchen-inspec; 2) manually edit the .kitchen.yml file to point to the default created tests in ./test/recipes/default_test.rb

Desired Behavior

Chef Development Kit Version: 0.16.28

Platform Version

Mac OS/X latest

Replication Case

-----> Setting up <default-ubuntu-1604>...
       Finished setting up <default-ubuntu-1604> (0m0.00s).
-----> Verifying <default-ubuntu-1604>...
       Use `/Users/charlesjohnson/foo/test/recipes/default` for testing
Summary: 0 successful, 0 failures, 0 skipped
       Finished verifying <default-ubuntu-1604> (0m0.28s).
...
-----> Setting up <default-centos-72>...
       Finished setting up <default-centos-72> (0m0.00s).
-----> Verifying <default-centos-72>...
       Use `/Users/charlesjohnson/foo/test/recipes/default` for testing
Summary: 0 successful, 0 failures, 0 skipped
       Finished verifying <default-centos-72> (0m0.26s).
Chef Delivery
Running Smoke Phase
-----> Starting Kitchen (v1.10.2)
-----> Verifying <default-ubuntu-1604>...
       Use `/Users/charlesjohnson/foo/test/recipes/default` for testing

Target:  ssh://vagrant@127.0.0.1:2222

  ○  1 skipped
     This is an example test, replace with your own test.
  ○  1 skipped
     This is an example test, replace with your own test.

Summary: 2 successful, 0 failures, 2 skipped
       Finished verifying <default-ubuntu-1604> (0m0.32s).
-----> Verifying <default-centos-72>...
       Use `/Users/charlesjohnson/foo/test/recipes/default` for testing

Target:  ssh://vagrant@127.0.0.1:2200

  ○  1 skipped
     This is an example test, replace with your own test.
  ○  1 skipped
     This is an example test, replace with your own test.

Summary: 2 successful, 0 failures, 2 skipped
       Finished verifying <default-centos-72> (0m0.29s).
-----> Kitchen is finished. (0m1.67s)
charlesjohnson commented 8 years ago

Edit: Initially I closed this ticket, based on a false reading. After talking with @ricardolupo I've revised the description and initial report. Thanks Lupo!

ricardolupo commented 8 years ago

The challenge here is that kitchen-inspec is actually looking for tests in the ./test/recipes/default directory. chef generate cookbook foo creates a sample test in ./test/recipes, which still leaves the user to create the default subdirectory. As a user, I expect the chef command to create all the scaffolding necessary for my cookbook.

Test case: 

09:51:46-rlupo~/cookbooks/dev/foo (master)$ kitchen verify
-----> Starting Kitchen (v1.10.2)
-----> Setting up <default-centos-72>...
$$$$$$ Running legacy setup for 'Docker' Driver
       Finished setting up <default-centos-72> (0m0.00s).
-----> Verifying <default-centos-72>...
       Use `/Users/rlupo/cookbooks/dev/foo/test/recipes/default` for testing
Summary: 0 successful, 0 failures, 0 skipped
       Finished verifying <default-centos-72> (0m0.73s).
-----> Kitchen is finished. (0m2.65s)
09:51:57-rlupo~/cookbooks/dev/foo (master)$ mkdir /Users/rlupo/cookbooks/dev/foo/test/recipes/default
09:52:21-rlupo~/cookbooks/dev/foo (master)$ mv /Users/rlupo/cookbooks/dev/foo/test/recipes/default
default/         default_test.rb
09:52:21-rlupo~/cookbooks/dev/foo (master)$ mv /Users/rlupo/cookbooks/dev/foo/test/recipes/default
default/         default_test.rb
09:52:21-rlupo~/cookbooks/dev/foo (master)$ mv /Users/rlupo/cookbooks/dev/foo/test/recipes/default_test.rb /Users/rlupo/cookbooks/dev/foo/test/recipes/default/default_test.rb
/Users/rlupo/cookbooks/dev/foo/test/recipes/default_test.rb -> /Users/rlupo/cookbooks/dev/foo/test/recipes/default/default_test.rb
09:52:44-rlupo~/cookbooks/dev/foo (master)$ kitchen verify
-----> Starting Kitchen (v1.10.2)
-----> Verifying <default-centos-72>...
       Use `/Users/rlupo/cookbooks/dev/foo/test/recipes/default` for testing

Target:  ssh://kitchen@192.168.99.100:32778

  ○  User root should exist; User root This is an example test, r... (1 skipped)
     This is an example test, replace with your own test.
  ○  Port 80 should not be listening; Port 80 This is an example ... (1 skipped)
     This is an example test, replace with your own test.

Summary: 2 successful, 0 failures, 2 skipped
       Finished verifying <default-centos-72> (0m0.81s).
-----> Kitchen is finished. (0m2.32s)
charlesjohnson commented 8 years ago

Updated the title and description of this issue to be more descriptive. Thanks for reporting this, @ricardolupo!

ricardolupo commented 8 years ago

add trailing / to ensure no confusion that default is a directory

15:19:42-rlupo~/cookbooks/dev/foo (master)$ kitchen verify -----> Starting Kitchen (v1.10.2) -----> Verifying ... Use /Users/rlupo/cookbooks/dev/foo/test/recipes/default for testing

Target: ssh://kitchen@192.168.99.100:32778

qubitrenegade commented 8 years ago

Another side effect is #1027 which causes foodcritic to fail.

I think it makes the most sense for chef generate cookbook to put the inspec files in /test/integration/default/inspec/ as this will keep with the current structure allowing us to have different suites and will allow for easier transition from ServerSpec to InSpec.

cheeseplus commented 7 years ago

Now the generator explicitly generates an inspec_tests key with a location and we've reverted the changes that made this confusing in the first place, closing.