A step for global provisioning actions

[robkinyon]

Please q.v. https://github.com/jimmycuadra/vagrant-librarian-chef/issues/16

I don't know if there is a step already built-in to vagrant for this. If there is, please comment in that issue and close this one.

[canausa]

I am not sure what you are asking, even with following the other ticket. Can you be more explicit?

[robkinyon]

There is a race condition in vagrant-librarian-chef when running with multiple AWS VMs. When librarian-chef runs, it deletes the directory it's managing and re-downloads all the cookbooks. The race condition is because librarian-chef is invoked on a per-VM basis. So, it deletes and re-downloads for VM1, VM2, and VM3 separately. This isn't a problem with Virtualbox because VB runs its provisioners in serial. However, AWS runs its provisioners in parallel, so we end up with some VMs "missing" files because another VM's provisioning step deleted them.

What vagrant-librarian-chef (and all the cookbook management plugins, cause vagrant-berkshelf has a similar issue) needs to do is manage the cookbook directories on a global basis, not when each VM is provisioned.

In other words, the action sequence should change from this:

• vagrant provision
  • For each VM:
  • vagrant-librarian-chef installs the cookbooks for the Cheffile thisVM uses
  • Do whatever sync'ing is needed (shared folders, rsync for vagrant-aws, etc)
  • chef-solo executes

to this:

• vagrant provision
  • Globally, vagrant-librarian-chef iterates over all Cheffiles
  • Install cookbooks for each Cheffile
  • Do whatever sync'ing is needed (shared folders, rsync for vagrant-aws, etc)
  • For each VM
  • chef-solo executes

[drpebcak]

@canausa Here's what I think is going on.

@robkinyon is trying to bring up three virtual machines. He has a provisioning step that should happen 'globally' to ALL the machines, and then some vm specific provisioning that should happen after that. He's running into an issue where the 'global' provisioning step happens every time a machine is brought up, which destroys the files on the other machines that have already been provisioned.

I think he is asking if Vagrant has any way of stopping that from happening.

[robkinyon]

@drpebcak Close. The vagrant-librarian-chef plugin is managing a list of cookbook dependencies for me. All three virtual machines are using the same list of community cookbooks. (Whether they have additional provisioning is irrelevant, though my specific case does not.) When vagrant-librarian-chef does its thing, librarian-chef deletes the directory it manages. This, in itself, is not a problem.

The problem is vagrant-librarian-chef isn't a provisioner. It's a provisioner-dependency-manager. It shoudn't invoked on a per-VM basis. Instead, it needs to be invoked at the beginning of provision, before with_target_vms() is invoked. So, I'm asking if there is a before_hook() these provisioner-management plugins can use.

[canausa]

That is an interesting predicament. I am sure there are things that can be done on both sides ( vagrant and vagrant-librarian-chef) in order to improve the experience for the user.

That being said there are definitely other hooks that the developer of vagrant-librarian-chef could hook into. I am not sure which is best but @tmatilai or @fgrehm might have a better idea on hooks.

[tmatilai]

To my understanding the middleware stacks are created per instance, so there might not be any suitable action where to hook now. And nothing stops different VMs from using different dependency managers or at least different configurations for them. So it might make more sense to try to handle this on the plugin side.

Maybe vagrant-librarian-chef should either use instance specific directories, or protect multiple action instances from running at the same time (and multiple times). The env instance could probably be used there.

@robkinyon maybe you already know, but as a workaround you should be able to force sequential action with --no-parallel option. Not optimal of course.

[robkinyon]

@tmatilai vagrant-berkshelf uses instance-specific directories, which seems to get around the problem.

[fgrehm]

IIRC there is no such thing as a global provisioning hook and if understood the problem right the only way I can think that this might work with the current code is to hook into :environment_load that is called right after the Vagrant::Environment has been created. From there you should be able to get hold of the active Vagrant::Machine instances to do your stuff within the hook.

I havent given enough thought whether we should have support for that but here's some pointers if you want to give it a go at using the hook I proposed:

• get hold of VM names / providers
• grab Machine instances
• list configured provisioners (not sure if you'll need this one but you can use it to get Chef related configs)

Hope that helps :smile:

[mitchellh]

@fgrehm Is correct.

I would fix this in the plugin itself with some clever locking when hooking into action_up. Since Vagrant rusn all in a single process, you should be able to lock and set a global variable saying that one of the provisioners did it.