Integration/functional tests

[jantman]

Look into whether there's any established work to do integration/functional tests of vagrant plugins. If so, start doing that, and also start running tests against multiple Vagrant versions.

[jantman]

vagrant-spec acceptance tests

How would I do this? testing all providers sounds pretty crazy, but germane to #7 and #8, which seem to be provider-specific issues. But how do I test them without functional providers for all of them?

Is there a dummy provider to start with, and come back to those other issues separately? I also don't think there's a terribly clear way to ensure that r10k only runs once, short of capturing the debug log output and inspecting it.

• in Vagrant master as of https://github.com/mitchellh/vagrant/commit/27599ee897da4a8145efb87e2a9294a1a0eacf8b none of the plugins have acceptance tests

Plugins that appear to have acceptance tests:

• chef/vagrant-omnibus (has acceptance tests but they just use system() and that's it)
• petems/vagrant-puppet-install
• BerlinVagrant/vagrant-dns
• fgrehm/vagrant-lxc
• Cimpress-MCP/vagrant-orchestrate
• fgrehm/docker-provider
• vikasmathod/VagrantTravis (its readme says vagrant-azure??)

I posted to the Google Group asking for advice on this: Advice on testing plugins?

[jantman]

Remaining code paths that might benefit from tests:

• hook Modulegetter before Provision
• hook Modulegetter before ConfigValidate
• Config (validate)
  • checks that directories exist (puppet_dir, puppetfile_path, puppetfile, module_path)
• Modulegetter
  • bail out if config not set
  • check that our config matches the puppet provisioner config
• module path explicitly set in r10k config, must match the puppet provisioner

[jantman]

see https://github.com/mitchellh/vagrant-spec/issues/2

[jantman]

Current WIP for this is in issues/12final_vmware; all other issues/12* branches can be disregarded.

[jantman]

Opened #20 to handle VMWare Workstation. Let's merge the Virtualbox stuff and get that out, and start working on other real issues.

[jantman]

Remaining code paths that might benefit from tests:

• hook Modulegetter before Provision
• hook Modulegetter before ConfigValidate
• Config (validate)
  • checks that directories exist (puppet_dir, puppetfile_path, puppetfile, module_path)
• Modulegetter
  • bail out if config not set
  • check that our config matches the puppet provisioner config
• module path explicitly set in r10k config, must match the puppet provisioner
• vagrant destroy -f shouldn't do an r10k deploy