Workflow experiment

[pikesley]

This PR is really just intended to initiate a discussion around some ideas I have for workflow. So, get this:

At the moment, we have a separate repo for our cucumber-chef stuff. Our features live over there, our site-cookbooks, our Cheffile, the whole thing.

And this feels like a bit of a non-DevOps silo. My thinking is that I would really like to run a cucumber-chef test-lab per-project, but use the cookbooks, environments etc from the existing repo (because these will be common across projects). So I would add cucumber-chef to the Gemfile of some-project and bundle it, then do

cucumber-chef init
cucumber-chef create some-project.feature

then fix up the Labfile and construct the feature as required, and

cucumber-chef setup
cucumber

which I guess will run all of the tests in that project, from the ground up, which strikes me as a Good Thing.

This PR represents my first cut at actually implementing this. We need to specify the location from which we'll pull the cookbooks, so we can now do things like this in .cucumber-chef/config.rb:

chef.merge!(
    :cookbook_paths => [
        "~/Github/cuke-chef/cookbooks",
        "~/Github/cuke-chef/site-cookbooks"
    ],
    :environment_paths => [
        "~/Github/cuke-chef/environments"
    ],
    :role_paths => [
        "~/Github/cuke-chef/roles"
    ]
)

which client.rb can now understand via this in config.rb

I'm now thinking, however, that further abstractions could be made and we could just have something like

chef.merge!(
  :chef_repo_path => "~/Github/cuke-chef/"
)

and work it all out from there.

So at the moment, I have this working for a very simple case.

Things to think about:

• I'm currently not handling data bags but that could be done in the same way
• We'd also want to merge in any stuff local to the project (particularly a project-specific role, I guess). My current implementation of this a bit crufty.

Does anybody else think this idea has legs? Is there something I've not thought of that's going to bite me on the bum?

[zpatten]

:+1: From a cursory glance I like all of this; just got into the office; but I will post some thoughts on this before EOD.

[anujbiyani]

I just put my cucumber-chef tests in the same repo as my chef-repo. Why do you think it's better to have your cucumber tests in a separate repository?

[zpatten]

Ya at present I always place everything directly in the chef-repo myself. I really like the configuration schema @pikesley has proposed thou; anything that adds flexibility and doesn't paint us into a corner is :+1: in my book. (still have not reviewed the PR in depth yet).

[anujbiyani]

Normally I'm in favor of adding flexibility when it doesn't paint us into a corner, but in this case I think divorcing the cucumber-chef tests from the chef-repo does more harm than good.

When I view a chef-repo, the features serve as documentation (in addition to their normal purpose of maintainability); they show me how to use the roles and environments and cookbooks in this repo. If the features are moved into a separate repo, then I either 1) won't know they exist, or 2) will have to hunt for them and keep two repositories open just to manage one. And because the features are separate, it's much easier to forget to update them when a role/whatever changes.

One last, smaller point: if two repositories are used, then two pull requests are required for one change-set. Not a problem for people who don't use pull requests, but seeing as it's a popular feature on GitHub I think encouraging its misuse is a bad idea.

[pikesley]

@anujbiyani Point taken, but my proposal would not break your current workflow, it just allows a bit of flexibility. Also, I'm finding that a lot of my Chef work these days consists of hunting down the appropriate community cookbooks and then setting a few attributes, either in an environment (which then belongs in the chef-repo) or in a role (which quite likely belongs firmly within a specific project), which has led me to this idea. If you're working on your own cookbooks, then yeah, I can see why this idea will not appeal to you.

I see this as a nice smooth path to getting dev thinking about ops (and the other way around). And combined with the Magic Space Future of Vagrant 1.1, we can ship pretty much entirely self-contained projects out of Github. This really appeals. But maybe that's just me ;)

There is at least one more potential banana skin here: creating a feature attempts to create features/support/env.rb, which will clash with whatever's already there. Maybe this is OK, I don't really understand enough about what that's doing.

And yeah, the code in this PR is bit lashed-together, I don't recommend merging it just yet :)

[anujbiyani]

Point taken, but my proposal would not break your current workflow, it just allows a bit of flexibility.

True.

Also, I'm finding that a lot of my Chef work these days consists of hunting down the appropriate community cookbooks and then setting a few attributes, either in an environment (which then belongs in the chef-repo) or in a role (which quite likely belongs firmly within a specific project), which has led me to this idea.

Not sure why this idea is any better or worse for working with community cookbooks and just setting attributes. Further, when I write my own cookbooks I don't put them in a site-cookbooks folder in my chef-repo; I treat them the same as community cookbooks (i.e. their own git repo, and downloaded via a Cheffile or Berksfile.

I see this as a nice smooth path to getting dev thinking about ops (and the other way around).

I think I take the opposite view point. Speaking as a dev who was brand new to ops 4 months ago (when I started learning about and working aggressively on Chef stuff), I'm used to seeing tests live next to code. The two (tests and implementation) would co-exist in every sense of the word; one commit would contain both implementation and tests, one pull request would show both, and one repo would contain both.

[zpatten]

Here's my thoughts on a few things:

• I like being able to configure the paths as laid out by @pikesley this definitely helps to make the framework more flexible and allows it to be used in "non-standard" setups.
• I agree strongly with @anujbiyani that the tests should live next to the code. I ultimately would like to see a features directory in all cookbooks. I also agree that tests serve as documentation. It is not always clear from reading code alone what the expected outcome is (especially in large, complex, meta-strewn, DSL driven code); tests on the other hand, especially integration tests, tend to concisely illustrate what is expected of the code.
• In the past I generally kept open sourced cookbooks in git repos while putting private, sensitive cookbooks in site-cookbooks (using librarian). Ultimately I feel that having a repo for each cookbook and handling all of them via Cheffile/Berksfile is the best practice and the way I prefer to do things now. (I tell people to think of cookbooks as gems and ask them if they would have fun running a Rails application without bundler and all of the gems vendored into the project; that tends to make the point for Rails devs at least.)
• The create task needs some lovin'. As is, it will attempt to seed the environment and hook source files (i.e. env.rb and cc-hooks.rb). We could split out the process into two tasks (one task that just creates feature templates and one task to seed the env.rb and cc-hooks.rb. To be honest, personally I don't ever use the create task, I just build my features up by hand. I can see how it is problematic if people are accidentally overwriting the env.rb changes one might of made.
• The test labs should support per project (i.e. git-repo) testing, that is if I am switching between separate chef-repo's, the tests should be running against their corresponding lab. I'm pretty sure with Vagrant this already happens; thou I have not tested this functionality to be honest; with AWS we will need some minor code changes in the provider to make this happen.

I briefly went down a road of making cucumber-chef "aware" of librarian for example, but I dropped that in favor of just defining cookbook_paths, that is cucumber-chef should be ignorant because at the end of the day it just wants to upload items from directories and run your tests. Adding internals logic (like saying oh we should look in "site-cookbooks" because we know we're using librarian) is just asking for trouble.

That being said, I am highly in favor of configuration options like the ones suggested by @pikesley. I feel it is great to allow this level of configuration flexibility, especially if it helps to bring non-standard chef-repo models into "the fold" as it were and allows them to pursue test driven infrastructure.

Bottom line; as long as the default out-of-the-box operation revolves around the standard best practice of running everything out of a single chef-repo and we continue to support that model as the default model I am happy. I am also happy if we can break down a barrier of entry for people with non-standard chef-repo setups and let them start doing test driven infrastructure (the whole time gently nudging them towards the best practice model of course ;)).

[anujbiyani]

with AWS we will need some minor code changes in the provider to make this happen.

I can confirm that this is necessary. I set up a new chef repo while testing out some things but connected it to the same AWS account as another repo, and sure enough the new chef repo tried to use the old cucumber-chef server.

(I tell people to think of cookbooks as gems and ask them if they would have fun running a Rails application without bundler and all of the gems vendored into the project; that tends to make the point for Rails devs at least.)

Heh I do the exact same thing.

The create task needs some lovin'. As is, it will attempt to seed the environment and hook source files (i.e. env.rb and cc-hooks.rb). We could split out the process into two tasks (one task that just creates feature templates and one task to seed the env.rb and cc-hooks.rb.

I think that a feature init tesk is necessary (one that just creates env.rb and cc-hooks.rb), and one that creates the template is optional. I, too, wouldn't use the latter task, but it might be useful. The former is obviously required. Splitting create into two sounds like the best approach.

[pikesley]

I'm about to disappear to Cornwall for a week, so there'll be no further contributions from me until at least the 22nd, but I'll clean the code up when I'm back and see how I get on...

[zpatten]

@pikesley Have a safe trip; I may get around to implementing some of the configuration changes (namely the path ones) while your out; if I fiddle with this I'll be sure to post here.