Wish: Chef provisioners should be able to install and use ChefDK, berks

[phs]

Right now, if I simultaneously want to use vagrant as my app's dev environment and chef as my provisioning tool, then I need a way to run chef in the guest. Presto, the chef_zero provisioner comes to the rescue and all is well.

If, following the example of virtually all chef-provisioned services, I need to also install berks dependencies, I am suddenly up a creek without a paddle.

Why is that?

There is vagrant-berkshelf which at least at one point was the canonical answer to this need, but it has a problematic history. The show-stopping bug for me currently is the inability of my host ruby and guest ruby (embedded in chef) to coexist. The issues listing over there is littered with various permutations of this core problem. Apparently it is so bad that the plugin itself is being phased out.

That's unfortunate, but fine as far as it goes. I should just move onto the next thing. Except @sethvargo over here is telling me in that post that the next thing is to ditch vagrant and do everything in kitchen. (Am I actually reading that right?)

That is not fine.

Surely something can be done.

Since the core pain point is the dueling rubies, one's forced to ask why we need to have them both. "Well, berks isn't installed on the guest, but we know it's on the host since we force you to install the tool chain on the host OS. So we need the host ruby."

Why can't we have berks in the guest? If I'm willing to have chef installed within my guest to run the provisioner, I may well be willing to have chefdk. Then I (or rather, the provisioner) can use the full tool chain without any need for extra plugins (or extra host OS dependencies!)

It took me about 20 minutes to blast this out and see it work:

In Vagrantfile:

# Enables the chef provisioner below.
config.vm.provision :shell, privileged: false, path: 'scripts/bootstrap-vagrant'

config.vm.provision :chef_zero do |chef_zero|
  chef_zero.install = false
  chef_zero.binary_path = '/opt/chefdk/bin'
  chef_zero.add_recipe 'my_recipe'
end

In scripts/bootstrap-vagrant:

#! /bin/bash

set -e
set -x

# Run as part of the vagrant provisioning process.
# Installs ChefDK (as opposed to just chef), and installs berks dependencies.

sudo apt-get update

# Install latest chefdk
wget -qO - https://www.chef.io/chef/install.sh | sudo bash -s -- -P chefdk

# For git@ berks dependencies
sudo apt-get install -y git

sudo tee /tmp/git_ssh <<'EOF'
ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no "$@"
EOF
sudo chmod +x /tmp/git_ssh

# https://github.com/mitchellh/vagrant/issues/6110#issuecomment-152646177
sudo mkdir -p /tmp/vagrant-chef/cookbooks/nodes

sudo -E GIT_SSH=/tmp/git_ssh \
  berks vendor -b /vagrant/Berksfile /tmp/vagrant-chef/cookbooks/cookbooks

sudo rm /tmp/git_ssh

This yields a vagrant provisioned by my cookbook, dependencies and all, with a guest-only toolchain.

So, my proposal is some new "common" options for the chef provisioners that lets users opt-in to installing chefdk instead of just chef.

Following vagrant-berkshelf's behavior (and if the user chose to use chefdk), if a Berksfile is found or declared by the user, an appropriate berks call is made to install dependencies in the guest where they need to go before firing up chef-client.

Concretely the new options might look like this:

config.vm.provision :chef_zero do |chef_zero|
  # Chef package to install, default 'chef'
  chef_zero.package = 'chefdk' 

  # Try to install berks dependencies?
  # Default is true iff berks found in binary_path and Berksfile is found.
  chef_zero.berks = true

  # Where's the Berksfile?  default Berksfile in guest root.
  chef_zero.berksfile = 'Berksfile'
end

[sethvargo]

Hi @phs

Thank you for opening an issue. I am okay with allowing users to install chef-dk instead of chef. However, I do not think that the Berksfile is going to remain the canonical source for cookbooks and therefore it should be up to the user to install those cookbooks; there are just too many edge cases and customizations that a user might like; it's better solved via a shell script. Additionally, I think Chef is working on Policyfile things that will eventually replace the Berksfile.

[phs]

If berks' place in the ecosystem is or will be unstable, would you consider including the berks functionality here if its surface area was reduced and it were strictly opt-in? Perhaps the new options concretely appear as

config.vm.provision :chef_zero do |chef_zero|
  # Chef package to install, default 'chef'
  chef_zero.package = 'chefdk' 

  # Where's the Berksfile?  The default is no Berksfile.  If set, the provisioner will try
  # to install berks dependencies.  If set, it is an error for the file to be missing or
  # berks to not be found in binary_path.
  chef_zero.berksfile = 'Berksfile'
end

[sethvargo]

@phs

No, I'm sorry. I'm one of the core authors of Berksfile. I love the tool :smile:. But I don't think it's Vagrant's responsibility to push that tool onto users until Chef Software makes a decision about where the tool stands in their ecosystem.