WIP: Berksfile support

[tildedave]

Aims at #166. Very much WIP, need to write tests/formal documentation, wanted to get a sense whether this was the right direction. (Wrote a bunch of tests for one approach that turned out not to work in parallel mode so backed them out.)

Workflow:

User updates the kitchen section of the littlechef.cfg:

[kitchen]
berksfile = Berksfile
berksfile_cookbooks_directory = berks-cookbooks

Before running any 'fix' commands littlechef does a berks vendor into that directory, removing it if it already exists. It does not remove the directory if the directory is newer than the Berksfile.

Had to do this in the node call because anything later runs into parallel execution issues, we only want to run berks vendor in single threaded mode

Still TODO:

• check for berkshelf 3 when config is specified, die if not there (not sure how much of this would work for berkshelf 2 as I've only ever used 3)
• check for existence of Berksfile
• berks vendor fails = fix should fail

(I decided I liked this approach better than pre-deploy hooks.)

[tobami]

As to implementing it in node(), that is in itself not a bad idea, except that there are commands that execute chef-solo and don't call node(), prominently recipe and role: https://github.com/tildedave/littlechef/blob/b8cd7e5d38c683dbb1405663b27d05af44a09f06/littlechef/runner.py#L228

Another possibility would be to do it in chef.sync_node, which will guarantee that berkshelf managed cookbooks are always there. I wonder though what will happen to commands like list_cookbooks. We would expect to see berkshelf cookbooks there right?

[tildedave]

@tobami yeah I saw recipe and role. However my understanding was that these commands require node to run in order to set the hosts that they are executed on. I'll double check that those execution paths go through it.

I avoided sync_node because of the thought that it would be executed in parallel if fabric is running in parallel, but all of that is theoretical -- will get some actual info. (My fabric knowledge is a little light.)

[tobami]

Actually, you are correct that sync_node would be executed in parallel, so bad suggestion.

It should happen at the very beginning... how about _readconfig()? There is always the problem of tool vs library usage, but it could even be executed at module level, before the if. I don't quite like the structure of all this, it grew out of a fabfile, which is not a good module/library organization for a bigger project.

[tildedave]

Sure, I'll take a look at that. I should be able to get this a little more solidified by the end of the weekend. Thanks for all the suggestions, your guidance is very helpful!

[tobami]

Great to be of help!

To clarify my comment above, what I meant I don't like is how the whole runner.py grew out of a fabfile into some kind of hybrid which is not quite a proper Python package/module, just in case I was not clear :smile:

[haad]

+1

This would be a very nice feature to have in littlechef.

[haad]

What needs to be done here to get this merged ?

[tobami]

More or less the "It should happen at the very beginning... how about _readconfig()" suggestion

[tildedave]

Haven't had much time to look at this lately, sorry. I'll try to bump this higher up on my mental priority list but if someone else did it and learned from this spike that would be okay too :)

[tobami]

Being reimplemented in #210, reusing this branch