Reuse the same test for multiple cookbooks

inspec / kitchen-inspec

Test-Kitchen Plugin for InSpec

Other

109 stars 58 forks source link

Reuse the same test for multiple cookbooks #25

Closed chris-rock closed 5 years ago

chris-rock commented 8 years ago

@coderanger and @cheeseplus pointed out, that it is not possible to reuse test across different cookbooks easily. He used shared tests like: https://github.com/poise/poise-service-spechelper/blob/master/lib/poise_service/spec_helper/helper.rb in https://github.com/poise/poise-service/blob/master/test/integration/default/serverspec/default_spec.rb#L17

We need to find a way to solve this problem with kitchen-inspec. Possible ways:

write busser implementation
allow test-kitchen to load a remote test suite

Any other ideas?

coderanger commented 8 years ago

Where does the inspec code itself actually run right now? My understanding is it is local on my workstation. If so, I can do the same trick but in the local Gemfile.

chris-rock commented 8 years ago

@coderanger Inspec is running where test-kitchen runs, therefore on your workstation. Since we face the problem at other areas as well eg. https://github.com/hardening-io would like to use the same test for Chef, Puppet and Ansible implementation. They currently use a Thor task to fetch the tests.

The way going forward from my perspective:

Step 1: Merge https://github.com/chef/inspec/pull/252 into InSpec This eases the way to write standalone test suites
Step 2: Add a feature to test-kitchen or kitchen-inspec to be able to load tests from a local directory or a remote directory.

Until Step 2 is done, I recommend to add shared test via a git submodule. I know, not perfect.

coderanger commented 8 years ago

@chris-rock If you look at the linked stuff, the shared tests are already in an external gem that is shared between projects, so that isn't the issue. It sounds like this should work with the current kitchen-inspec structure, but it should probably be called out more that this is really different from busser-serverspec.

chris-rock commented 8 years ago

@coderanger I would like to ensure that we are able to solve your use case. I find your implementation is super cool, especially how you have done it with serverspec and rspec. We got a lot of requests regarding this use case to run shared tests. At the same time, I learned that most users are not advanced ruby developers. They just want to have one git repo with all the tests. They are not happy to being forced to push anything to ruby gems. On top the gem approach is may not be well suited for users who need to have everything in private.

I try to avoid to force users to write a gem for running shared tests. Would be very interested in your opinion.

coderanger commented 8 years ago

Sure, my solution was driven entirely by that being the only way to get it done with busser-serverspec. What I would want to see if possible is the ability to pull tests from another cookbook as a core feature, but I'm not really sure where that would start.

clintoncwolfe commented 5 years ago

We think this is almost certainly covered by inspec profile dependency resolution. Please re-open if discussion is needed!