search

jantman / vagrant-r10k

UNSUPPORTED - SEEKING MAINTAINER - Vagrant middleware plugin to retrieve puppet modules using r10k.
MIT License
35 stars 12 forks source link

Problems with plugin sync #2

Closed frizop closed 9 years ago

frizop commented 9 years ago

When trying to launch a vagrant up and using config elements like the following, I get the error in the debug log -

config.r10k.puppet_dir = "."
config.r10k.puppetfile_path = "Puppetfile"
config.r10k.module_path = "modules"

and the following Puppetfile

mod 'puppetlabs/ntp'
mod 'basti1302/windows_path'
mod 'rismoney/chocolatey'

The short of the error is the following

[R10K::TaskRunner - ERROR] Task #<R10K::Task::Module::Sync:0x00000102150498> failed while running: Command puppet module --modulepath
/Users/frizop/Documents/vagrant-windows/modules install --force puppetlabs/ntp exited with 1:
/Applications/Vagrant/embedded/gems/gems/bundler-1.6.6/lib/bundler/rubygems_integration.rb:256:in 
`block in replace_gem': puppet is not part of the bundle. Add it to Gemfile. (Gem::LoadError)
    from /usr/local/var/rbenv/versions/2.0.0-p247/bin/puppet:22:in `<main>'

So I'm pretty confused here, if I manually run the puppet module command above it works fine, if I run r10k puppetfile install it installs fine, but for some reason (even after uninstalling and reinstalling the vagrant r10k plugin) it still doesn't work, any help?

And the debug log:

    DEBUG subprocess: Waiting for process to exit. Remaining to timeout: 32000
DEBUG subprocess: Exit status: 0
 INFO warden: Calling OUT action: #<VagrantPlugins::ProviderVirtualBox::Action::Created:0x000001024a7a88>
 INFO runner: Preparing hooks for middleware sequence...
 INFO runner: 5 hooks defined.
 INFO runner: Running action: #<Vagrant::Action::Warden:0x00000102540f30>
 INFO warden: Calling IN action: #<Proc:0x0000010258bb98@/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/action/warden.rb:94 (lambda)>
 INFO warden: Calling IN action: #<Proc:0x00000102540e90@/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/action/warden.rb:94 (lambda)>
 INFO warden: Calling IN action: #<VagrantPlugins::R10k::Modulegetter:0x0000010207aca8>
 INFO interface: info: vagrant-r10k: Beginning r10k deploy of puppet modules into /Users/User1/Documents/vagrant-windows/modules using /Users/User1/Documents/vagrant-windows/Puppetfile
 INFO interface: info: ==> build: vagrant-r10k: Beginning r10k deploy of puppet modules into /Users/User1/Documents/vagrant-windows/modules using /Users/User1/Documents/vagrant-windows/Puppetfile
==> build: vagrant-r10k: Beginning r10k deploy of puppet modules into /Users/User1/Documents/vagrant-windows/modules using /Users/User1/Documents/vagrant-windows/Puppetfile
[R10K::Task::Puppetfile::Sync - INFO] Loading modules from Puppetfile into queue
[R10K::Task::Module::Sync - INFO] Deploying chocolatey into /Users/User1/Documents/vagrant-windows/modules
[R10K::Task::Module::Sync - INFO] Deploying windows_path into /Users/User1/Documents/vagrant-windows/modules
[R10K::Task::Module::Sync - INFO] Deploying ntp into /Users/User1/Documents/vagrant-windows/modules
[R10K::Module::Forge - DEBUG] Execute: puppet module --modulepath /Users/User1/Documents/vagrant-windows/modules install --force puppetlabs/ntp
[R10K::Module::Forge - DEBUG] [puppet module --modulepath /Users/User1/Documents/vagrant-windows/modules install --force puppetlabs/ntp] STDERR: /Applications/Vagrant/embedded/gems/gems/bundler-1.6.6/lib/bundler/rubygems_integration.rb:256:in `block in replace_gem': puppet is not part of the bundle. Add it to Gemfile. (Gem::LoadError)
    from /usr/local/var/rbenv/versions/2.0.0-p247/bin/puppet:22:in `<main>'
[R10K::TaskRunner - ERROR] Task #<R10K::Task::Module::Sync:0x00000102447048> failed while running: Command puppet module --modulepath /Users/User1/Documents/vagrant-windows/modules install --force puppetlabs/ntp exited with 1: /Applications/Vagrant/embedded/gems/gems/bundler-1.6.6/lib/bundler/rubygems_integration.rb:256:in `block in replace_gem': puppet is not part of the bundle. Add it to Gemfile. (Gem::LoadError)
    from /usr/local/var/rbenv/versions/2.0.0-p247/bin/puppet:22:in `<main>'

[R10K::Task::Puppetfile::Purge - DEBUG] No stale modules in /Users/User1/Documents/vagrant-windows/modules
ERROR warden: Error occurred: RuntimeError: Command puppet module --modulepath /Users/User1/Documents/vagrant-windows/modules install --force puppetlabs/ntp exited with 1: /Applications/Vagrant/embedded/gems/gems/bundler-1.6.6/lib/bundler/rubygems_integration.rb:256:in `block in replace_gem': puppet is not part of the bundle. Add it to Gemfile. (Gem::LoadError)
    from /usr/local/var/rbenv/versions/2.0.0-p247/bin/puppet:22:in `<main>'

 INFO warden: Beginning recovery process...
 INFO warden: Calling recover: #<Vagrant::Action::Builtin::Call:0x0000010207ae10>
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Recovery complete.
ERROR warden: Error occurred: RuntimeError: Command puppet module --modulepath /Users/User1/Documents/vagrant-windows/modules install --force puppetlabs/ntp exited with 1: /Applications/Vagrant/embedded/gems/gems/bundler-1.6.6/lib/bundler/rubygems_integration.rb:256:in `block in replace_gem': puppet is not part of the bundle. Add it to Gemfile. (Gem::LoadError)
    from /usr/local/var/rbenv/versions/2.0.0-p247/bin/puppet:22:in `<main>'

 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
ERROR warden: Error occurred: RuntimeError: Command puppet module --modulepath /Users/User1/Documents/vagrant-windows/modules install --force puppetlabs/ntp exited with 1: /Applications/Vagrant/embedded/gems/gems/bundler-1.6.6/lib/bundler/rubygems_integration.rb:256:in `block in replace_gem': puppet is not part of the bundle. Add it to Gemfile. (Gem::LoadError)
    from /usr/local/var/rbenv/versions/2.0.0-p247/bin/puppet:22:in `<main>'

 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO environment: Released process lock: machine-action-7158750c4ad0798b640156d433f681aa
 INFO environment: Running hook: environment_unload
 INFO runner: Preparing hooks for middleware sequence...
 INFO runner: 5 hooks defined.
 INFO runner: Running action: #<Vagrant::Action::Builder:0x0000010265b848>
ERROR vagrant: Vagrant experienced an error! Details:
ERROR vagrant: #<VagrantPlugins::R10k::ErrorWrapper: RuntimeError: Command puppet module --modulepath /Users/User1/Documents/vagrant-windows/modules install --force puppetlabs/ntp exited with 1: /Applications/Vagrant/embedded/gems/gems/bundler-1.6.6/lib/bundler/rubygems_integration.rb:256:in `block in replace_gem': puppet is not part of the bundle. Add it to Gemfile. (Gem::LoadError)
    from /usr/local/var/rbenv/versions/2.0.0-p247/bin/puppet:22:in `<main>'
>
ERROR vagrant: RuntimeError: Command puppet module --modulepath /Users/User1/Documents/vagrant-windows/modules install --force puppetlabs/ntp exited with 1: /Applications/Vagrant/embedded/gems/gems/bundler-1.6.6/lib/bundler/rubygems_integration.rb:256:in `block in replace_gem': puppet is not part of the bundle. Add it to Gemfile. (Gem::LoadError)
    from /usr/local/var/rbenv/versions/2.0.0-p247/bin/puppet:22:in `<main>'

ERROR vagrant: /Users/User1/.vagrant.d/gems/gems/vagrant-r10k-0.1.1/lib/vagrant-r10k/modulegetter.rb:113:in `block in call'
/Users/User1/.vagrant.d/gems/gems/vagrant-r10k-0.1.1/lib/vagrant-r10k/modulegetter.rb:112:in `each'
/Users/User1/.vagrant.d/gems/gems/vagrant-r10k-0.1.1/lib/vagrant-r10k/modulegetter.rb:112:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/action/warden.rb:34:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/action/warden.rb:95:in `block in finalize_action'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/action/warden.rb:34:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/action/warden.rb:34:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/action/warden.rb:95:in `block in finalize_action'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/action/warden.rb:34:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/action/warden.rb:34:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/action/builder.rb:116:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/action/runner.rb:66:in `block in run'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/util/busy.rb:19:in `busy'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/action/runner.rb:66:in `run'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/action/builtin/call.rb:53:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/action/warden.rb:34:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/plugins/providers/virtualbox/action/check_virtualbox.rb:17:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/action/warden.rb:34:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/action/builder.rb:116:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/action/runner.rb:66:in `block in run'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/util/busy.rb:19:in `busy'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/action/runner.rb:66:in `run'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/machine.rb:196:in `action_raw'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/machine.rb:173:in `block in action'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/environment.rb:474:in `lock'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/machine.rb:161:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/machine.rb:161:in `action'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.6.5/lib/vagrant/batch_action.rb:82:in `block (2 levels) in run'
/Applications/Vagrant/embedded/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `call'
/Applications/Vagrant/embedded/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context'
 INFO interface: error: RuntimeError: Command puppet module --modulepath /Users/User1/Documents/vagrant-windows/modules install --force puppetlabs/ntp exited with 1: /Applications/Vagrant/embedded/gems/gems/bundler-1.6.6/lib/bundler/rubygems_integration.rb:256:in `block in replace_gem': puppet is not part of the bundle. Add it to Gemfile. (Gem::LoadError)
    from /usr/local/var/rbenv/versions/2.0.0-p247/bin/puppet:22:in `<main>'

RuntimeError: Command puppet module --modulepath /Users/User1/Documents/vagrant-windows/modules install --force puppetlabs/ntp exited with 1: /Applications/Vagrant/embedded/gems/gems/bundler-1.6.6/lib/bundler/rubygems_integration.rb:256:in `block in replace_gem': puppet is not part of the bundle. Add it to Gemfile. (Gem::LoadError)
    from /usr/local/var/rbenv/versions/2.0.0-p247/bin/puppet:22:in `<main>'

 INFO interface: Machine: error-exit ["VagrantPlugins::R10k::ErrorWrapper", "RuntimeError: Command puppet module --modulepath /Users/User1/Documents/vagrant-windows/modules install --force puppetlabs/ntp exited with 1: /Applications/Vagrant/embedded/gems/gems/bundler-1.6.6/lib/bundler/rubygems_integration.rb:256:in `block in replace_gem': puppet is not part of the bundle. Add it to Gemfile. (Gem::LoadError)\n\tfrom /usr/local/var/rbenv/versions/2.0.0-p247/bin/puppet:22:in `<main>'\n"]
[User1@profail]~/Documents/vagrant-windows%
jantman commented 9 years ago

@frizop how is Puppet installed on your system, using the OS packes or via ruby gem?

I'll point you to the Installation section of README.md - https://github.com/jantman/vagrant-r10k#installation "Note that if you include modules from a (the) forge in your Puppetfile [...] instead of just git references, you will need the puppet rubygem installed and available. It is not included in the Gemfile so that users who only use git references won't have a new/possibly conflicting Puppet installation."

Puppet is specifically not included in the plugin itself, as that would conflict with any existing puppet installs on the system. If you have the puppet gem installed (what does the output of gem list | grep puppet show?) then this is either a bug or a regression. If not, then this is known behavior (I'm not sure if there's a way to avoid the conflicts, so I erred on the side of caution).

I can try to reproduce this on my own machine in the morning or tomorrow.

jantman commented 9 years ago

Ahh, I just saw what might be the key here:

/usr/local/var/rbenv/versions/2.0.0-p247/bin/puppet

I haven't tested this with rbenv, as I don't use it and don't believe I know anyone who does. I'll need some time tomorrow morning to try and reproduce this and see what I can do.

frizop commented 9 years ago

Yep, you got it. I end up just building everything via ruby gems because I find it a bit easier to manage that way rather than pulling down packages and installing them over each other. Just for clarity sake, here is a list of the gems installed on my system:

Oh and one note, using the system ruby on the mac os is just asking for pain, thus the use of rbenv. I'm tied to * 2.0.0-p247 (set by /usr/local/var/rbenv/version)

*** LOCAL GEMS ***

addressable (2.3.6)
archive-tar-minitar (0.5.2)
autoparse (0.3.3)
autorun (0.1.0)
aws-sdk (1.57.0, 1.42.0)
aws-sdk-v1 (1.57.0)
beaker (1.20.1)
beaker-rspec (3.0.0)
bigdecimal (1.2.5, 1.2.0)
blimpy (0.6.7)
builder (3.2.2)
bundle (0.0.1)
bundler (1.7.4)
CFPropertyList (2.2.8)
chronic_duration (0.10.6)
coderay (1.1.0)
colored (1.2)
cri (2.6.1, 2.5.0)
diff-lcs (1.2.5)
docile (1.1.5)
docker-api (1.13.6)
excon (0.40.0)
extlib (0.9.16)
facter (2.2.0 universal-darwin)
faraday (0.9.0, 0.8.9)
faraday_middleware (0.9.1)
faraday_middleware-multi_json (0.0.6)
fission (0.5.0)
fog (1.24.0, 1.22.1)
fog-brightbox (0.6.1)
fog-core (1.24.0)
fog-json (1.0.0)
fog-radosgw (0.0.3)
fog-sakuracloud (0.1.1)
fog-softlayer (0.3.22)
fog-xml (0.1.0)
formatador (0.2.5)
google-api-client (0.7.1)
hiera (1.3.4)
highline (1.6.21)
hocon (0.0.6)
inflecto (0.0.2)
inifile (3.0.0, 2.0.2)
io-console (0.4.2)
ipaddress (0.8.0)
json (1.8.1, 1.7.7)
json_pure (1.8.1)
jwt (1.0.0)
kwalify (0.7.2)
launchy (2.4.2)
log4r (1.1.10)
metaclass (0.0.4)
method_source (0.8.2)
mime-types (2.4.3, 1.25.1)
mini_portile (0.6.1, 0.6.0)
minitar (0.5.4)
minitest (5.4.2, 4.3.2)
mocha (1.1.0)
multi_json (1.10.1, 1.8.4)
multipart-post (2.0.0, 1.2.0)
net-scp (1.2.1)
net-ssh (2.9.1)
nokogiri (1.6.3.1, 1.5.11)
numerizer (0.2.0, 0.1.1)
power_assert (0.1.4)
pry (0.10.1)
psych (2.0.6, 2.0.0)
puppet (3.7.2)
puppet-lint (1.1.0)
puppet-syntax (1.3.0)
puppetlabs_spec_helper (0.8.2)
r10k (1.3.4)
rake (0.9.6)
rbvmomi (1.8.2, 1.8.1)
rdoc (4.1.2, 4.0.0)
retriable (1.4.1)
rgen (0.6.6)
rspec (2.99.0)
rspec-core (2.99.2)
rspec-expectations (3.1.2, 3.0.4, 2.99.2)
rspec-its (1.0.1)
rspec-mocks (3.1.3, 3.0.4, 2.99.2)
rspec-puppet (1.0.1)
rspec-support (3.1.2, 3.0.4)
rspec-system (2.8.0)
rspec-system-puppet (2.2.1)
serverspec (2.3.1, 1.16.0)
signet (0.5.1)
simplecov (0.9.0)
simplecov-html (0.8.0)
slop (3.6.0)
spec (5.3.4)
specinfra (2.4.0, 1.27.5)
systemu (2.6.4, 2.5.2)
test-unit (3.0.1, 2.0.0.0)
thor (0.19.1)
trollop (2.0)
unf (0.1.4)
unf_ext (0.0.6)
uuidtools (2.1.5)
jantman commented 9 years ago

Ok. I'll have to spend some time on this to see how to reproduce it and get a fix (not familiar with rbenv), but if you happen to have any suggestions on that, I'm open to them. If not, I'll see what I can come up with tomorrow. Thanks for taking the time to provide a thorough, detailed bug report.

jantman commented 9 years ago

@frizop I'm really sorry, but I haven't had time to really look into this yet, and I'm about to be leaving for a week without reliable Internet access. I can, however, provide you with a workaround that might work for your purposes; if you specify the modules in the Puppetfile using their git repos instead of Forge modules, git will be used to install them instead of puppet module, and these issues should go away.

So, this would mean taking your current Puppetfile:

mod 'puppetlabs/ntp'
mod 'basti1302/windows_path'
mod 'rismoney/chocolatey'

and changing it to use git repos as the module source:

mod 'ntp',
  :git => 'https://github.com/puppetlabs/puppetlabs-ntp.git',
  :ref => '3.2.1'
mod 'windows_path',
  :git => 'https://github.com/basti1302/puppet-windows-path.git'
mod 'chocolatey',
  :git => 'https://github.com/rismoney/puppet-chocolatey.git'

Usually I would specify the ref for each of them, to ensure that I use a tag that matches up to a specific released version of the module (and to prevent pulling in any possibly-un-released changes from master), but neither of the two non-puppetlabs modules you're using appear to be properly tagging their releases on GitHub.

frizop commented 9 years ago

Ah, right on. Tested and this does indeed work. Any chance you could help point to what's wrong and maybe I can also take a look? I'm no ruby dev but I'll give it the ol' college try.

Also, I was away on vacation myself so no worries about not having an immediate solution!

frizop commented 9 years ago

I did note one thing looking into this a little bit, when running a debug2 on the r10k module running a module install, and it's that the R10K::Task::Module::Sync gets called, where I only see a R10K::Task::Puppetfile::Sync getting called on the vagrant module. I'm having a little bit of trouble following the application flow of the r10k module, it might be a little over my head or at least I don't know the proper debugging tools for ruby to really get my teeth into it. 

[frizop@profail]~/Documents/vagrant-windows% r10k puppetfile --verbose=debug2 install
[R10K::Task::Puppetfile::Sync - INFO] Loading modules from Puppetfile into queue
[R10K::Task::Module::Sync - INFO] Deploying chocolatey into /Users/frizop/Documents/vagrant-windows/modules
[R10K::Module::Forge - DEBUG1] Execute: puppet module --modulepath /Users/frizop/Documents/vagrant-windows/modules install --force rismoney/chocolatey
[R10K::Module::Forge - DEBUG2] [puppet module --modulepath /Users/frizop/Documents/vagrant-windows/modules install --force rismoney/chocolatey] STDOUT: Notice: Preparing to install into /Users/frizop/Documents/vagrant-windows/modules ...
Notice: Downloading from https://forgeapi.puppetlabs.com ...
Notice: Installing -- do not interrupt ...
/Users/frizop/Documents/vagrant-windows/modules
└── rismoney-chocolatey (v0.0.3)
tremblaysimon commented 9 years ago

I've got a similar error and what I understand it is we can't have puppet and puppet gem installed at the same time?

The workaround for me is the same as mentionned above: avoid using Puppet Forge directly.

jantman commented 9 years ago

@tremblaysimon The puppet gem isn't included in this plugin as doing so can cause conflicts on systems where Puppet is already installed (i.e. via os packages).

For the time being, yes, if you specify the modules directly from git or other source control (instead of as forge modules), it will prevent this issue.

lorello commented 9 years ago

I've got the same problem, having three versions of puppet installed (3.6.2, 3.7.4, 3.7.5) with rubygem.

I've removed them and installed 3.7.5 from the puppetlabs debian repo, the problem disappeared.

Thanks L

frizop commented 9 years ago

@lorello I was using gems as well but ended up giving up and just installing the packages from puppetlans directly, no more problems. I'd love to know why but I don't have the time to debug this or really worry about it.

mmarod commented 9 years ago

Er so... If I am understanding this thread correctly -- Forge modules cannot be referenced from a Puppetfile still? I'm having this issue and do not want to reference the git repos directly. I have one version of the puppet gem (4.2.0) installed on my OSX 10.10.3 host.

mmarod commented 9 years ago

I seem to have been able to get it to work by doing..

vagrant plugin install puppet

Sorry for the interruption!

jantman commented 9 years ago

@mmarod, I'd completely forgotten that you can use vagrant plugin install to install any Gem into Vagrant's environment... thanks so much for reminding me of this. I'll update the docs and make sure a note is included in the next release (which should be coming soon, hopefully).