thinking about .fixtures.yml vs metadata.json

[Dan33l]

Now unit tests and acceptance tests use .fixtures.yml.

But:

• Humans, on the forge, read information extracted from metadata.json
• puppet module install uses metata.json.

We have two sources. It would be greet if we can test that they are updated accordingly.

[ekohl]

I somewhat disagree for a number of reasons. These could be solved by unification in metadata.json.

metadata.json doesn't deal with soft dependencies, but you probably want to be able to test this. Note there's still the issue that you can't easily test with and without. Ideally metadata.json would actually support soft dependencies.

In acceptance tests you may want to test with other modules as well. https://github.com/voxpupuli/puppet-trusted_ca pulls in the Apache module to test whether it can deploy certificates in its acceptance tests. That does bring us to the point that there should be a fixtures conditional that it's only installed for acceptance tests because you don't want to burden the unit tests with additional modules. https://github.com/puppetlabs/puppetlabs_spec_helper#using-fixtures does mention the puppet_version conditional, an additional flag might be nice. These could be stored as testDependencies in metadata.json.

Git vs non-git. In tests it's great to be able to test with the latest versions. However, there's no reliable way to detect the git URL for a module from the forge. In fact, sometimes you want to diverge because you need a certain branch. Fixtures allows that. You can store it in metadata.json in the dependencies similar to how fixtures.yaml stores it. Ideally you would even have a flag/env var to choose whether to pull modules from the forge or git. That would allow you to add another worker to test with released versions.

[trevor-vaughan]

We currently use fixtures.yml for quite a bit more than metadata.json can (or probably should) handle.

The various current use cases are represented at https://github.com/puppetlabs/puppetlabs_spec_helper#fixtures-examples

Being unable to do these things would cause additional tooling to be written which would further fragment existing and future users and cause more confusion than already exists between metadata.json, Puppetfiles, and fixtures.yml.

[Dan33l]

the point proposed by @ekohl that there should be a fixtures conditional that a module is only installed for tests looks interesting.

My goal with this issue is to brainstorm about the fact that we have to maintain several sources (I even forgotten Puppetfiles with r10k). For instance, tests can be green and pass but because Puppetfile is different in production env the run on production nodes can fail. In my side it looks a problem since long years.

[npwalker]

https://tickets.puppetlabs.com/browse/PDK-1042

https://tickets.puppetlabs.com/browse/PDK-993

[ghoneycutt]

I disagree with @ekohl recommendation for testDependencies with regards to it being stored in metadata.json. The more appropriate place would be puppetlabs_spec_helper by adding an option that only installs the module if an environment variable is set. This is more in accordance to @trevor-vaughan response and how the rest of the tooling works.

[ekohl]

I disagree with @ekohl recommendation for testDependencies

Wouldn't exactly call a recommendation, more of a possible implementation.

[tphoney]

There is another possible way, https://puppet.com/docs/bolt/latest/installing_tasks_from_the_forge.html#task-8928. yeah i know its another way. but it works well with the litmus workflow. I have been speaking with a contributor about maybe adding this as a litmus rake task.

[tphoney]

Hey everyone i have "the fix" maybe :)

192

This rake task allows a user to build all the modules in a directory, copy the resultant tar files to the SUT, and force install them. By default it will look in the spec/fixtures/modules. folder. It will accept a folder path as a parameter too. (it ignores symlinked folders)

bundle exec rake litmus:install_modules_from_directory
# or 
bundle exec rake 'litmus:install_modules_from_directory[~/.puppetlabs/bolt/modules]'

Can we close this ticket, and open a new ticket if you have feedback with the new functionality

[pmcmaw]

@Dan33l @trevor-vaughan @ekohl @ghoneycutt @npwalker

Hey, im am doing a rough groom of this repo and wondering how yous would feel about closing this issue inline with Tps comment?

[ekohl]

I'd be ok with that. I never ended up using Litmus since it uses PDK which I refuse to use since it has telemetry on by default. In https://github.com/voxpupuli/voxpupuli-acceptance/blob/master/lib/voxpupuli/acceptance/fixtures.rb we have our own beaker based code that uses puppet-modulebuilder to create packages from the checked out fixtures. I've been very content with that.

[jordanbreen28]

closing inline with above comments.