Puppet 4: strict_variables defaults to true and spec_helper config option ignored

[newtonne]

We're finally(!) migrating our infrastructure from Puppet 3 to Puppet 4 and in preparation for this, we have started testing all our modules against the latest Puppet 4 version, 4.10.12.

We found that many of our tests that were passing against Puppet 3.8.7, now failed against Puppet 4.10.12 with errors similar to the following:

Puppet::PreformattedError:
       Evaluation Error: Unknown variable: '::lsbmajdistrelease'. at ...

By the way, we are using the same rspec-puppet version for Puppet 3 and Puppet 4 testing:

$ gem list | grep rspec-puppet
rspec-puppet (2.7.10)

It seemed to me that something was setting strict_variables to true, and so to test this theory, I ran the tests as:

STRICT_VARIABLES=no rake

and indeed they passed.

However, when I configured the spec_helper config option instead:

RSpec.configure do |c|
  c.strict_variables = false
end

the tests did not pass. What's more, the documentation suggests that this setting defaults to false anyway.

I then tried configuring strict_variables using Puppet.settings:

RSpec.configure do |c|
  c.before :each do
    Puppet.settings[:strict_variables]=false
  end
end

which caused the tests to pass once more.

So a couple of questions:

1. Why does it appear that strict_variables is defaulting to true when testing in Puppet 4? The Puppet 4 documentation states that this setting defaults to false.
2. Why does the c.strict_variables setting seem to have no effect in Puppet 4? When I set c.strict_variables=true in Puppet 3, the tests do fail as expected.

Thanks.

[DavidS]

In many cases rspec-puppet is used to verify modules for use by others. In this case the module needs to work on the strictest settings, since it has no control over other's environments who'd possibly be running with strict_variables on. Hence the default.

To address this for folks like yourself, we added the possibility to override this setting through the .sync.yml in the pdk-templates in https://github.com/puppetlabs/pdk-templates/pull/326.

Please follow the official docs for upgrading to puppet4/5 and the Updating 3->4 Manifests guide. Also understand that puppet5 is EOL this January, so don't stop updating until you're at puppet7. The updates from 5 through 7 will be a lot smoother, as there are no major changes like from 3 to 4.

[newtonne]

Thanks, @DavidS. Indeed, as mentioned in the PR you linked, that line in puppetlabs_spec_helper/module_spec_helper.rb confuses things somewhat.

Also, I'm sure you'll be pleased to learn that we have already upgraded through to Puppet 6 👍