search

rodjek / librarian-puppet

http://librarian-puppet.com
MIT License
693 stars 209 forks source link

Dependency cycle when module referenced in Puppetfile using :git #313

Closed madAndroid closed 9 years ago

madAndroid commented 9 years ago

We're using librarian-puppet to manage profile module dependencies, stored in git; when one profile references another, which in turn references the module we're installing the dependencies for, a dependency cycle is picked up, and librarian-puppet won't continue.

Not sure if this is expected behaviour - there is a similar issue described in #191, but that was due to the module in question referencing itself in it's Puppetfile. In this case, each profile module does not reference itself, but only the other module.

I've been able to reproduce this in a set of test modules (created with puppet module generate, using the puppet-module-skeleton )

https://github.com/madAndroid/puppet-profile-libtest1 https://github.com/madAndroid/puppet-profile-libtest2

When running `librarian-puppet install --verbose', the following error is shown:

for puppet-profile-libtest1:

/s/d/r/m/T/puppet-profile-libtest1 git:master  ❯❯❯❯ librarian-puppet install --verbose
[Librarian] Ruby Version: 2.0.0
[Librarian] Ruby Platform: universal.x86_64-darwin14
[Librarian] Rubygems Version: 2.0.14
[Librarian] Librarian Version: 0.6.3
[Librarian] Librarian Adapter: puppet
[Librarian] Librarian Adapter Version: 2.2.1
[Librarian] Project: /srv/data/repos/madAndroid/TESTING/puppet-profile-libtest1
[Librarian] Specfile: Puppetfile
[Librarian] Lockfile: Puppetfile.lock
[Librarian] Git: /usr/local/bin/git
[Librarian] Git Version: 2.3.5
[Librarian] Git Environment Variables:
[Librarian]   (empty)
[Librarian] Pre-Cached Sources:
[Librarian] Post-Cached Sources:
[Librarian]   [:forge, "https://forgeapi.puppetlabs.com", {}]
[Librarian]   [:git, "git@github.com:madAndroid/puppet-profile-libtest2.git", {}]
[Librarian] Resolving puppetlabs-stdlib (>= 0) <https://forgeapi.puppetlabs.com>
[Librarian]   Checking manifests
[Librarian]   Module puppetlabs-stdlib found versions: 4.6.0, 4.5.1, 4.5.0, 4.4.0, 4.3.2, 4.3.0, 4.2.2, 4.2.1, 4.2.0, 4.1.0, 3.2.2, 3.2.1, 3.2.0, 3.1.1, 3.1.0, 3.0.1, 3.0.0, 2.6.0, 2.5.1, 2.5.0, 2.4.0, 2.3.3, 2.3.2, 2.3.1, 2.3.0, 2.2.1, 2.2.0, 2.1.3, 2.0.0, 1.1.0, 1.0.0, 0.1.7, 0.1.6, 0.1.5, 0.1.4, 0.1.3, 0.1.2, 0.1.1
[Librarian]     Checking puppetlabs-stdlib/4.6.0 <https://forgeapi.puppetlabs.com>
[Librarian]       Resolved puppetlabs-stdlib (>= 0) <https://forgeapi.puppetlabs.com> at puppetlabs-stdlib/4.6.0 <https://forgeapi.puppetlabs.com>
[Librarian]   Resolved puppetlabs-stdlib (>= 0) <https://forgeapi.puppetlabs.com>
[Librarian] Resolving libtest2 (>= 0) <git@github.com:madAndroid/puppet-profile-libtest2.git#master>
[Librarian]   Checking manifests
[Librarian]     --- No output
[Librarian]     --- No output
[Librarian]     --- No output
[Librarian]     --- No output
[Librarian]     --> origin
[Librarian]     -->   origin/HEAD -> origin/master
[Librarian]     -->   origin/master
[Librarian]     --> 84a3c0fb0280e66aab13cb05cad82b4577d5bb14
[Librarian]     --> eca525e7989cf5c941dca4065b5ec45946a50e14
[Librarian]     --> 84a3c0fb0280e66aab13cb05cad82b4577d5bb14
[Librarian]     --- No output
[Librarian]     --> 84a3c0fb0280e66aab13cb05cad82b4577d5bb14
[Librarian]     Checking libtest2/0.1.0 <git@github.com:madAndroid/puppet-profile-libtest2.git#master>
[Librarian] Pre-Cached Sources:
[Librarian] Post-Cached Sources:
[Librarian]   [:forge, "https://forgeapi.puppetlabs.com", {}]
[Librarian]   [:git, "git@github.com:madAndroid/puppet-profile-libtest1.git", {}]
Dependency 'puppetlabs-stdlib' duplicated for module libtest2, merging: ["puppetlabs-stdlib (>= 0) <https://forgeapi.puppetlabs.com>", "puppetlabs-stdlib (>= 4.0.0) <https://forgeapi.puppetlabs.com>"]
[Librarian]       Resolved libtest2 (>= 0) <git@github.com:madAndroid/puppet-profile-libtest2.git#master> at libtest2/0.1.0 <git@github.com:madAndroid/puppet-profile-libtest2.git#master>
[Librarian]   Resolved libtest2 (>= 0) <git@github.com:madAndroid/puppet-profile-libtest2.git#master>
[Librarian] Resolving libtest1 (>= 0) <git@github.com:madAndroid/puppet-profile-libtest1.git#master>
[Librarian]   Checking manifests
[Librarian]     --- No output
[Librarian]     --- No output
[Librarian]     --- No output
[Librarian]     --- No output
[Librarian]     --> origin
[Librarian]     -->   origin/HEAD -> origin/master
[Librarian]     -->   origin/master
[Librarian]     --> d7a6ac3525e74f913c5ef6c5d2f0dcae86dd12ca
[Librarian]     --> c69b2234f5dddf7d021e10e0a9d1e7ec1e3b427d
[Librarian]     --> d7a6ac3525e74f913c5ef6c5d2f0dcae86dd12ca
[Librarian]     --- No output
[Librarian]     --> d7a6ac3525e74f913c5ef6c5d2f0dcae86dd12ca
[Librarian]     Checking libtest1/0.1.0 <git@github.com:madAndroid/puppet-profile-libtest1.git#master>
[Librarian] Pre-Cached Sources:
[Librarian] Post-Cached Sources:
[Librarian]   [:forge, "https://forgeapi.puppetlabs.com", {}]
[Librarian]   [:git, "git@github.com:madAndroid/puppet-profile-libtest2.git", {}]
Dependency 'puppetlabs-stdlib' duplicated for module libtest1, merging: ["puppetlabs-stdlib (>= 0) <https://forgeapi.puppetlabs.com>", "puppetlabs-stdlib (>= 4.0.0) <https://forgeapi.puppetlabs.com>"]
[Librarian]       Cycle with libtest1/0.1.0 <git@github.com:madAndroid/puppet-profile-libtest1.git#master>
[Librarian]       Backtracking from libtest1/0.1.0 <git@github.com:madAndroid/puppet-profile-libtest1.git#master>
[Librarian]   Failed to resolve libtest1 (>= 0) <git@github.com:madAndroid/puppet-profile-libtest1.git#master>
Could not resolve the dependencies.
/Library/Ruby/Gems/2.0.0/gems/librarianp-0.6.3/lib/librarian/action/persist_resolution_mixin.rb:11:in `persist_resolution'
/Library/Ruby/Gems/2.0.0/gems/librarianp-0.6.3/lib/librarian/action/resolve.rb:32:in `run'
/Library/Ruby/Gems/2.0.0/gems/librarian-puppet-2.2.1/lib/librarian/puppet/action/resolve.rb:10:in `run'
/Library/Ruby/Gems/2.0.0/gems/librarian-puppet-2.2.1/lib/librarian/puppet/cli.rb:104:in `resolve!'
/Library/Ruby/Gems/2.0.0/gems/librarian-puppet-2.2.1/lib/librarian/puppet/cli.rb:68:in `install'
/Library/Ruby/Gems/2.0.0/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
/Library/Ruby/Gems/2.0.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
/Library/Ruby/Gems/2.0.0/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
/Library/Ruby/Gems/2.0.0/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
/Library/Ruby/Gems/2.0.0/gems/librarianp-0.6.3/lib/librarian/cli.rb:26:in `block (2 levels) in bin!'
/Library/Ruby/Gems/2.0.0/gems/librarianp-0.6.3/lib/librarian/cli.rb:31:in `returning_status'
/Library/Ruby/Gems/2.0.0/gems/librarianp-0.6.3/lib/librarian/cli.rb:26:in `block in bin!'
/Library/Ruby/Gems/2.0.0/gems/librarianp-0.6.3/lib/librarian/cli.rb:47:in `with_environment'
/Library/Ruby/Gems/2.0.0/gems/librarianp-0.6.3/lib/librarian/cli.rb:26:in `bin!'
/Library/Ruby/Gems/2.0.0/gems/librarian-puppet-2.2.1/bin/librarian-puppet:7:in `<top (required)>'
/usr/bin/librarian-puppet:23:in `load'
/usr/bin/librarian-puppet:23:in `<main>'

for puppet-profile-libtest2:

[Librarian] Ruby Version: 2.0.0
[Librarian] Ruby Platform: universal.x86_64-darwin14
[Librarian] Rubygems Version: 2.0.14
[Librarian] Librarian Version: 0.6.3
[Librarian] Librarian Adapter: puppet
[Librarian] Librarian Adapter Version: 2.2.1
[Librarian] Project: /srv/data/repos/madAndroid/TESTING/puppet-profile-libtest2
[Librarian] Specfile: Puppetfile
[Librarian] Lockfile: Puppetfile.lock
[Librarian] Git: /usr/local/bin/git
[Librarian] Git Version: 2.3.5
[Librarian] Git Environment Variables:
[Librarian]   (empty)
[Librarian] Pre-Cached Sources:
[Librarian] Post-Cached Sources:
[Librarian]   [:forge, "https://forgeapi.puppetlabs.com", {}]
[Librarian]   [:git, "git@github.com:madAndroid/puppet-profile-libtest1.git", {}]
[Librarian] Resolving puppetlabs-stdlib (>= 0) <https://forgeapi.puppetlabs.com>
[Librarian]   Checking manifests
[Librarian]   Module puppetlabs-stdlib found versions: 4.6.0, 4.5.1, 4.5.0, 4.4.0, 4.3.2, 4.3.0, 4.2.2, 4.2.1, 4.2.0, 4.1.0, 3.2.2, 3.2.1, 3.2.0, 3.1.1, 3.1.0, 3.0.1, 3.0.0, 2.6.0, 2.5.1, 2.5.0, 2.4.0, 2.3.3, 2.3.2, 2.3.1, 2.3.0, 2.2.1, 2.2.0, 2.1.3, 2.0.0, 1.1.0, 1.0.0, 0.1.7, 0.1.6, 0.1.5, 0.1.4, 0.1.3, 0.1.2, 0.1.1
[Librarian]     Checking puppetlabs-stdlib/4.6.0 <https://forgeapi.puppetlabs.com>
[Librarian]       Resolved puppetlabs-stdlib (>= 0) <https://forgeapi.puppetlabs.com> at puppetlabs-stdlib/4.6.0 <https://forgeapi.puppetlabs.com>
[Librarian]   Resolved puppetlabs-stdlib (>= 0) <https://forgeapi.puppetlabs.com>
[Librarian] Resolving libtest1 (>= 0) <git@github.com:madAndroid/puppet-profile-libtest1.git#master>
[Librarian]   Checking manifests
[Librarian]     --- No output
[Librarian]     --- No output
[Librarian]     --- No output
[Librarian]     --- No output
[Librarian]     --> origin
[Librarian]     -->   origin/HEAD -> origin/master
[Librarian]     -->   origin/master
[Librarian]     --> d7a6ac3525e74f913c5ef6c5d2f0dcae86dd12ca
[Librarian]     --> c69b2234f5dddf7d021e10e0a9d1e7ec1e3b427d
[Librarian]     --> d7a6ac3525e74f913c5ef6c5d2f0dcae86dd12ca
[Librarian]     --- No output
[Librarian]     --> d7a6ac3525e74f913c5ef6c5d2f0dcae86dd12ca
[Librarian]     Checking libtest1/0.1.0 <git@github.com:madAndroid/puppet-profile-libtest1.git#master>
[Librarian] Pre-Cached Sources:
[Librarian] Post-Cached Sources:
[Librarian]   [:forge, "https://forgeapi.puppetlabs.com", {}]
[Librarian]   [:git, "git@github.com:madAndroid/puppet-profile-libtest2.git", {}]
Dependency 'puppetlabs-stdlib' duplicated for module libtest1, merging: ["puppetlabs-stdlib (>= 0) <https://forgeapi.puppetlabs.com>", "puppetlabs-stdlib (>= 4.0.0) <https://forgeapi.puppetlabs.com>"]
[Librarian]       Resolved libtest1 (>= 0) <git@github.com:madAndroid/puppet-profile-libtest1.git#master> at libtest1/0.1.0 <git@github.com:madAndroid/puppet-profile-libtest1.git#master>
[Librarian]   Resolved libtest1 (>= 0) <git@github.com:madAndroid/puppet-profile-libtest1.git#master>
[Librarian] Resolving libtest2 (>= 0) <git@github.com:madAndroid/puppet-profile-libtest2.git#master>
[Librarian]   Checking manifests
[Librarian]     --- No output
[Librarian]     --- No output
[Librarian]     --- No output
[Librarian]     --- No output
[Librarian]     --> origin
[Librarian]     -->   origin/HEAD -> origin/master
[Librarian]     -->   origin/master
[Librarian]     --> 84a3c0fb0280e66aab13cb05cad82b4577d5bb14
[Librarian]     --> eca525e7989cf5c941dca4065b5ec45946a50e14
[Librarian]     --> 84a3c0fb0280e66aab13cb05cad82b4577d5bb14
[Librarian]     --- No output
[Librarian]     --> 84a3c0fb0280e66aab13cb05cad82b4577d5bb14
[Librarian]     Checking libtest2/0.1.0 <git@github.com:madAndroid/puppet-profile-libtest2.git#master>
[Librarian] Pre-Cached Sources:
[Librarian] Post-Cached Sources:
[Librarian]   [:forge, "https://forgeapi.puppetlabs.com", {}]
[Librarian]   [:git, "git@github.com:madAndroid/puppet-profile-libtest1.git", {}]
Dependency 'puppetlabs-stdlib' duplicated for module libtest2, merging: ["puppetlabs-stdlib (>= 0) <https://forgeapi.puppetlabs.com>", "puppetlabs-stdlib (>= 4.0.0) <https://forgeapi.puppetlabs.com>"]
[Librarian]       Cycle with libtest2/0.1.0 <git@github.com:madAndroid/puppet-profile-libtest2.git#master>
[Librarian]       Backtracking from libtest2/0.1.0 <git@github.com:madAndroid/puppet-profile-libtest2.git#master>
[Librarian]   Failed to resolve libtest2 (>= 0) <git@github.com:madAndroid/puppet-profile-libtest2.git#master>
Could not resolve the dependencies.
/Library/Ruby/Gems/2.0.0/gems/librarianp-0.6.3/lib/librarian/action/persist_resolution_mixin.rb:11:in `persist_resolution'
/Library/Ruby/Gems/2.0.0/gems/librarianp-0.6.3/lib/librarian/action/resolve.rb:32:in `run'
/Library/Ruby/Gems/2.0.0/gems/librarian-puppet-2.2.1/lib/librarian/puppet/action/resolve.rb:10:in `run'
/Library/Ruby/Gems/2.0.0/gems/librarian-puppet-2.2.1/lib/librarian/puppet/cli.rb:104:in `resolve!'
/Library/Ruby/Gems/2.0.0/gems/librarian-puppet-2.2.1/lib/librarian/puppet/cli.rb:68:in `install'
/Library/Ruby/Gems/2.0.0/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
/Library/Ruby/Gems/2.0.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
/Library/Ruby/Gems/2.0.0/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
/Library/Ruby/Gems/2.0.0/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
/Library/Ruby/Gems/2.0.0/gems/librarianp-0.6.3/lib/librarian/cli.rb:26:in `block (2 levels) in bin!'
/Library/Ruby/Gems/2.0.0/gems/librarianp-0.6.3/lib/librarian/cli.rb:31:in `returning_status'
/Library/Ruby/Gems/2.0.0/gems/librarianp-0.6.3/lib/librarian/cli.rb:26:in `block in bin!'
/Library/Ruby/Gems/2.0.0/gems/librarianp-0.6.3/lib/librarian/cli.rb:47:in `with_environment'
/Library/Ruby/Gems/2.0.0/gems/librarianp-0.6.3/lib/librarian/cli.rb:26:in `bin!'
/Library/Ruby/Gems/2.0.0/gems/librarian-puppet-2.2.1/bin/librarian-puppet:7:in `<top (required)>'
/usr/bin/librarian-puppet:23:in `load'
/usr/bin/librarian-puppet:23:in `<main>'

The Puppetfiles: puppet-profile-libtest1:

#!/usr/bin/env ruby
#^syntax detection

forge "https://forgeapi.puppetlabs.com"

# A module from the Puppet Forge
mod 'puppetlabs-stdlib'

# A module from git
mod 'libtest2',
  :git => 'git@github.com:madAndroid/puppet-profile-libtest2.git'

puppet-profile-libtest2:

#!/usr/bin/env ruby
#^syntax detection

forge "https://forgeapi.puppetlabs.com"

# A module from the Puppet Forge
mod 'puppetlabs-stdlib'

# A module from git
mod 'libtest1',
  :git => 'git@github.com:madAndroid/puppet-profile-libtest1.git'
carlossg commented 9 years ago

that's expected, you can't have dependency cycles A->B->A

madAndroid commented 9 years ago

that's expected, you can't have dependency cycles A->B->A

I thought that it might be .. however, we definitely need this functionality in order to effectively test our manifests - since both modules need to have resources available in the other in order to satisfy our testing. Is there any known work around, other than 'just don't do it' ? :)

This use case makes #245 quite appealing .. for the moment, in order to resolve the issue we're experiencing, we're going to move the Puppetfiles out of the root of the repos, and hopefully this should help getting our testing working again.

carlossg commented 9 years ago

No work arounds, you can't have a dependency resolution system with cycles. My suggestion is to put the resources somewhere else so you don't introduce cycles.

madAndroid commented 9 years ago

Okay thanks - will investigate alternatives