Some valid Ruby Puppetfiles are not interpreted

[alex-harvey-z3q]

When executing g10k -puppetfile I get:

$ g10k -puppetfile 
Error: Missing :git url in ./Puppetfile for module ad line: mod 'ad',git: 'git@example.com:FOO/puppet-ad',ref: 'master'

My Puppetfile has modules declared as:

mod 'ad',
  git: 'git@example.com:FOO/puppet-ad',
  ref: 'master'
...

Workaround for me was to rewrite the Puppetfiles using a regexp:

:%s/^  \(...\): /  :\1 => 

[xorpaul]

Oh, I've never seen that Puppetfile format to be honest.

I'm always looking at https://github.com/puppetlabs/r10k/blob/master/doc/puppetfile.mkd

I don't know if I'm going to support that format.

I'd rather just use a YAML file or something similar.

[alex-harvey-z3q]

That's OK. This is very cool:

$ g10k -puppetfile 
Resolving Git modules (39/39) 2.436s [====================================================================] 100%
Synced ./Puppetfile with 39 git repositories and 0 Forge modules in 11.2s with git (10.8s sync, I/O 3.8s) and Forge (0.0s query+download, I/O 0.0s)

👍

[alex-harvey-z3q]

Puppetfile, by the way, is a Ruby DSL. mod is a method that optionally accepts a Hash of keys. Anything that is valid Hash syntax in Ruby is valid Puppetfile, http://breakthebit.org/post/8453341914/ruby-19-and-the-new-hash-syntax

[xorpaul]

Yeah, which makes it difficult for any other language than Ruby to parse a Puppetfile. Trust me I know :grinning:

Same problem as #36

[Gibheer]

Maybe you can take a ruby syntax parser, for example from grubby and get the real meaning out of it that way.

On the other hand, you would then have a ruby syntax parser and people would be able to write loops and stuff, which would open a whole new level of mess :D

[maxadamo]

@alexharv074 11 seconds is too much for 39 modules (but perhaps you have a high latency to your git server, or you disabled hardlinks). Look at my logs. 113 module in 5 seconds: 2017-07-05 18:58:16,838 INFO Synced ./Puppetfile with 78 git repositories and 35 Forge modules in 5.8s with git (5.4s sync, I/O 0.2s) and Forge (1.5s query+download, I/O 0.0s)

[ntnn]

@xorpaul In ntnn/puppetfiler I've written some code which parses a puppetfile to yaml. One possibility to resolve issues like these could be to extract that functionality to its own gem and to use that in g10k, e.g. by distributing it in the same tarball.

Enabling you to remove the regexes parsing g10k with all their blind spots.

What do you think of that?

[xorpaul]

I appreciate the suggestion, but bundling a ruby gem with g10k isn't really something that I want to do.

I'm testing a YAML based Puppetfile at the moment, which should resolve such issues.

[alex-harvey-z3q]

Not really sure how a YAML config file helps people who want to use this tool in the way that it is advertised, i.e. as a drop-in replacement for r10k. If the tool fails to read a Puppetfile, then it might be a useful alternative to r10k; but it is not a drop-in replacement. I for one would not be able to use it at all if it can't support real Puppetfiles because my code needs to be compatible with a real r10k.

[xorpaul]

@alexharv074 The r10k Puppetfile format parsing will remain in g10k. I'm just testing if I can provide advanced features like variable interpolation more easily inside a YAML based Puppetfile.

[ZeroPointEnergy]

As some already mentioned the issue is that the Puppetfile supports ruby syntax and people are doing all kinds of wild things with it.

A very nice solution would be to simply provide a hook for executing a script between when the control repository is checked out and g10k deploys the modules. This way we can transform whatever logic was in that old Puppetfile and just generate one with it g10k understands.

[kwisatz]

Just wanted to leave my 2 cents here that I also stumbled upon this by wondering what was wrong with my file when g10k claimed that Missing :git url in …. IMHO if g10k becomes more popular, and I guess it will, given that Puppetlabs now refer to it, many more people migrating from r10k will stumble over this.