search

facebook / grocery-delivery

The Grocery Delivery utility for managing cookbook uploads to distributed Chef backends.
Apache License 2.0
154 stars 55 forks source link

support upload of cookbooks using berkshelf #7

Closed wjimenez5271 closed 9 years ago

wjimenez5271 commented 9 years ago

support for Berkshelf to resolve and upload cookbooks

wjimenez5271 commented 9 years ago

also see related: https://github.com/facebook/between-meals/pull/11

jaymzh commented 9 years ago

I've not used berkshelf - so I have a few questions that may be dumb.

wjimenez5271 commented 9 years ago

Thanks for all the feedback. Those are great questions. I'll try my best to answer.

I have heard Jamie's thoughts on roles, and from what I can tell, his issue with them is their inherit ephemerality and lack of versioning. I share the same concern, but do like them for the ability to use them in knife searches. The alternative to roles is the "role-cookbook", however I don't think they have to be mutually exclusive. I plan to have a "chef role" for each role-cookbook, where all the logic and attribute data is in the role-cookbook, and the "chef role" is simply a pointer to that role-cookbook. This achieves the goals of the role-cookbook, while giving it some of the power of the native role object provided by Chef.

Re. your 2nd question, I thought so too, but the more I've explored this and sought after the opinions of others (there was a great hangops session on this that touched on it a bit), the less I've seen it as a strict dichotomy. The chef community embraces a loose model of how things are to be done--the thought being one-size-fits-all doesn't yield the best results. That being said, the use case for combining the two here is to manage community cookbooks in sane way. I think the monolithic repo has many benefits: it clearly describes a changes to an environment (branch = delta, instead of multiple branches for each repo involved), versioning can be based on git hash which enforces accurate "versioning" of cookbook code (instead of versions defined by the user which can mean different things if not managed carefully), just to name a few. But for community cookbooks, maintaining a duplicate copy in a local repo solely for the purposes of deployment ins't ideal. Using Berkshelf to resolve these dependencies addresses this problem well, while not compromising the benefits we get from our own assets being in a single repo. We're not making any changes to the comm. cookbooks, if we need to set any attributes or override anything, we'll use wrapper cookbooks. Therefore our changes are confined to the domain of our repo, again supporting the goal that change-sets can be described with a single branch/pull request/commit.

Hope that helps explain the use case, and thanks again for considering this idea!

jaymzh commented 9 years ago

Thanks for the explanation. I think I get it. So the workflow is something like:

Something like:

roles ---------------------------> grocery-delivery ----------> Chef Server
community cookbooks ----------------------------> berkshelf-----^
your cookbooks -------------> grocery-delivery ------^

?

If you do this - especially given that you have roles that are pointers to cookbooks, how do you keep roles in sync with cookbooks since berkshelf will block things going to Chef but roles are instantly in Chef?

wjimenez5271 commented 9 years ago

Berkshelf should push cookbooks to Chef instantly as well I think, it just makes sure dependencies are pushed also. Perhaps there is a situation I'm not thinking of?

jaymzh commented 9 years ago

From Jamie's talk, that wasn't the impression I got. You had to "promote" them using some sort of test or manual process? As I understand it the whole point is that you can "safely" upload to berkshelf regularly without it immediately going to your chef server, so you can take a whole bundle of cookbooks, test them and then put that entire regression-tested set of cookbooks into prod at once.

wjimenez5271 commented 9 years ago

Hmm, can't say I've seen that, possibly that was the model in v 2.0 of Berkshelf as I know they made a lot of changes in the latest version (3.0). I'll see if I can get some more info/confirmation of the expected behavior of berks.

sciurus commented 9 years ago

@jaymzh there is a distinction between berks install, which copies a cookbook into your $HOME/.berkshelf/cookbooks, and berks upload, which uploads it to your chef server. Maybe that is what you were thinking of? In Berkshelf I'm not aware of anything that behaves quite like you describe. I think the "berkshelf way" to put a set of cookbooks into prod at once would be to upload them, then edit the version constraints in the chef environment for prod.

I use Berkshelf the same way that @wjimenez5271 does. It's purely for managing dependencies. I don't use environments or cookbook versioning at all. I have a homegrown program that runs in CI to upload changes from my single chef repo to the chef server. That program also runs Berkshelf to upload the community cookbooks I depend on to the chef server. Now that I'm aware of grocery-delivery, I may look into using it instead. Having it handle Berkshelf for me would be a plus.

jaymzh commented 9 years ago

Excellent. Thanks for the explanation. Sounds awesome. OK, I'm onboard with the change.

We just need to clean up the stuff from the code review, and you need to sign the CLA... I don't know why the Facebook bot didn't bug you... I'll have a poke at that.

jaymzh commented 9 years ago

Here's a link to the CLA: https://code.facebook.com/cla - @wjimenez5271 - you'll need to sign it before we can merge your changes.

wjimenez5271 commented 9 years ago

Thanks! CLA completed.

jaymzh commented 9 years ago

Your dependent change to BM has been merged. Minor style here to fix up and then I'll go ahead and merge, looks good. Thanks for contributing and for all your work during the review process!

wjimenez5271 commented 9 years ago

Awesome, thanks for considering this PR!