Pulling multiple modules with same :path argument causes duplicate clones

[erikwebb]

(This issue stems from the librarian-puppet project.)

It appears that when I try to install multiple modules, each with the same :path argument, that a unique clone is created each time.

For example -

mod "base",
  :git => "git@git.example.com:infrastructure/puppet-modules.git",
  :path => "base"

mod "couchbase",
  :git => "git@git.example.com:infrastructure/puppet-modules.git",
  :path => "couchbase"

This will actually create 2 entire copies of the same repo. Unfortunately my client has all Puppet modules under a single repo, but I don't plan on using all of them for my Vagrant setup, so I only choose to install the few that are necessary.

If the assumption is that multiple Puppet modules are available within one repo, wouldn't it make sense to use that Git repo as multiple :path arguments? If you don't, you're assuming that first module will always be included or else the Git repo will never be cloned, creating an implied dependency.

[yfeldblum]

That would certainly be ideal, but there are structural problems with it that conflict with other things Librarian needs to do. For example, Librarian needs to be able to update individual dependencies independently of each other.

What's Available Now

Two sources that have differing options (:path is an option, and in your example it differs) are different sources. Two sources that have the same options are treated as the same source.

There is one saving grace. If you do the following (in Librarian-Chef, at any rate):

cookbook "base", :git => "git@git.example.com:infrastructure/puppet-modules.git"
cookbook "couchbase", :git => "git@git.example.com:infrastructure/puppet-modules.git"

Without specifying a :path option, then Librarian will look in both the top-level and in ./base for a cookbook metadata file marking the root of a cookbook named base. Same for ./couchbase for the couchbase cookbook. Since the options are the same (:git with the same value), it is the same source and it is checked out once. When updating one, the other is updated too. This is generally bad form, though. I'm not sure how Puppet modules work and if there is any kind of marker file for the root of a Puppet module.

Possible Optimizations

It's possible to optimize Librarian's Git source internally to detect when there are two of the same repository with different options that are compatible with the same checkout, but that's not done.