search

voxpupuli / rspec-puppet-facts

Simplify your unit tests by looping on every supported Operating System and populating facts.
Apache License 2.0
61 stars 53 forks source link

Symbolized facts causes hiera to "ignore bad definition" in hierarchy #58

Closed dandunckelman closed 5 years ago

dandunckelman commented 7 years ago

Summary

When I try to perform a hiera.lookup request in a spec, the hierarchy doesn't find the kernel fact. Because of that, Hiera shows this message:

Ignoring bad definition in :hierarchy: 'kernel/'

And as a result, the data I need is not found.

spec/spec_helper.rb

require "puppetlabs_spec_helper/module_spec_helper"
require "rspec-puppet-facts"
include RspecPuppetFacts

# needed to perform lookups inside specs
require "hiera"

RSpec.configure do |c|
  c.default_facts = {
    "hostname"      => `hostname -f`.strip,
    "ipaddress"     => `hostname -i`.strip,
    "puppetversion" => "4.6.1",
  }

  c.fail_fast = true if ENV['FAIL_FAST'] == "true"

  # needed to evaluate hiera lookups in the puppet code
  c.hiera_config = "spec/fixtures/hiera/hiera.yaml"

  if ENV['COVERAGE'] == "true"
    c.after(:suite) do
      RSpec::Puppet::Coverage.report!
    end
  end
end

# Default to "no" as our specs fail on default facts (i.e. "$::networking")
ENV['STRICT_VARIABLES'] = "no" unless ENV['STRICT_VARIABLES']

spec/fixtures/hiera/hiera.yaml

:backends:
  - yaml
:yaml:
  :datadir: spec/fixtures/hiera
:hierarchy:
  - "fqdn/%{fqdn}"
  - "environment/%{environment}"
  - "hostgroup/%{hostgroup}"
  - "os/%{operatingsystem}/%{operatingsystemrelease}"
  - "os/%{operatingsystem}"
  - "kernel/%{kernel}"
  - "common"

spec/fixtures/hiera/os/Ubuntu.yaml

cron_service: 'cron'

spec/fixtures/hiera/kernel/Linux.yaml

cron_service: 'crond'

manifests/linux.pp

class ta_base::linux() {
  $cron_service = hiera('cron_service')

  service { $cron_service:
    enable     => true,
    ensure     => true,
    hasrestart => true,
    hasstatus  => true,
  }
}

spec/classes/linux_spec.rb

require "spec_helper"

hiera = Hiera.new(:config => "spec/fixtures/hiera/hiera.yaml")

describe "ta_base::linux" do
  on_supported_os.each do |os, facts|
    context "on #{os}" do
      let(:facts) { facts }

      cron_service = hiera.lookup("cron_service", nil, facts)

      it { is_expected.to compile.with_all_deps }

      it { is_expected.to contain_service(cron_service)
        .with(
          "enable"  => true,
          "ensure"  => true
        )
      }
    end
  end
end

Example run

$ SPEC=spec/classes/linux_spec.rb FAIL_FAST=true rake spec_standalone
/home/dunk/.rvm/rubies/ruby-2.3.1/bin/ruby -I/home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/rspec-core-3.6.0/lib:/home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/rspec-support-3.6.0/lib/home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/rspec-core-3.6.0/exe/rspec spec/classes/linux_spec.rb --color
DEBUG: 2017-08-23 17:38:05 -0600: Hiera YAML backend starting
DEBUG: 2017-08-23 17:38:05 -0600: Looking up cron_service in YAML backend
DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'fqdn/'
DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'environment/'
DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'hostgroup/'
DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'os//'
DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'os/'
DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'kernel/'
DEBUG: 2017-08-23 17:38:05 -0600: Looking for data source common
DEBUG: 2017-08-23 17:38:05 -0600: Looking up cron_service in YAML backend
DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'fqdn/'
DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'environment/'
DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'hostgroup/'
DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'os//'
DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'os/'
DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'kernel/'
DEBUG: 2017-08-23 17:38:05 -0600: Looking for data source common
DEBUG: 2017-08-23 17:38:05 -0600: Looking up cron_service in YAML backend
DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'fqdn/'
DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'environment/'
DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'hostgroup/'
DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'os//'
DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'os/'
DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'kernel/'
DEBUG: 2017-08-23 17:38:05 -0600: Looking for data source common

ta_base::linux
  on centos-6-x86_64
    should compile into a catalogue without dependency cycles
    should contain Service[] with enable => true and ensure => true (FAILED - 1)

Failures:

  1) ta_base::linux on centos-6-x86_64 should contain Service[] with enable => true and ensure => true
     Failure/Error:
       it { is_expected.to contain_service(cron_service)
         .with(
           "enable"  => true,
           "ensure"  => true
         )
       }

     ArgumentError:
       No title provided and "Service" is not a valid resource reference
     # /home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/puppet-4.7.0/lib/puppet/resource.rb:572:in `extract_type_and_title'
     # /home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/puppet-4.7.0/lib/puppet/resource.rb:557:in `type_and_title'
     # /home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/puppet-4.7.0/lib/puppet/resource/catalog.rb:355:in `resource'
     # /home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/rspec-puppet-2.6.8/lib/rspec-puppet/matchers/create_generic.rb:85:in `matches?'
     # ./spec/classes/linux_spec.rb:17:in `block (4 levels) in <top (required)>'

Finished in 3.93 seconds (files took 4.26 seconds to load)
2 examples, 1 failure

Failed examples:

rspec ./spec/classes/linux_spec.rb[1:1:2] # ta_base::linux on centos-6-x86_64 should contain Service[] with enable => true and ensure => true

/home/dunk/.rvm/rubies/ruby-2.3.1/bin/ruby -I/home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/rspec-core-3.6.0/lib:/home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/rspec-support-3.6.0/lib/home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/rspec-core-3.6.0/exe/rspec spec/classes/linux_spec.rb --color failed

From what I can see, this comes down to the facts hash having kernel as a Symbol instead of a String, because it properly loads spec/fixtures/hiera/kernel/Linux.yaml when I do this:

# manifest
facts['kernel'] = "Linux"
cron_service = hiera.lookup("cron_service", nil, facts)

# log output from running test
DEBUG: 2017-08-23 17:39:14 -0600: Looking up cron_service in YAML backend
DEBUG: 2017-08-23 17:39:14 -0600: Ignoring bad definition in :hierarchy: 'fqdn/'
DEBUG: 2017-08-23 17:39:14 -0600: Ignoring bad definition in :hierarchy: 'environment/'
DEBUG: 2017-08-23 17:39:14 -0600: Ignoring bad definition in :hierarchy: 'hostgroup/'
DEBUG: 2017-08-23 17:39:14 -0600: Ignoring bad definition in :hierarchy: 'os//'
DEBUG: 2017-08-23 17:39:14 -0600: Ignoring bad definition in :hierarchy: 'os/'
DEBUG: 2017-08-23 17:39:14 -0600: Looking for data source kernel/Linux
DEBUG: 2017-08-23 17:39:14 -0600: Found cron_service in kernel/Linux

So....

dandunckelman commented 7 years ago

Environment

Bundler   1.14.6
Rubygems  2.5.1
Ruby      2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux]
GEM_HOME  /home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing
GEM_PATH  /home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing:/home/dunk/.rvm/gems/ruby-2.3.1@global
RVM       1.29.1 (latest)
Git       1.9.1
Platform  x86_64-linux
rubygems-bundler (1.4.4)

Gemfile

source "https://rubygems.org"

gem 'puppet', '4.7.0'

group :test do
  gem 'puppet-lint', '2.3.0'
  gem 'puppetlabs_spec_helper', '2.2.0'
  gem 'rspec-puppet', '2.6.8'
  gem 'parallel_tests', '2.14.2'
  gem 'rspec-puppet-facts', '1.8.0', :require => false
  gem 'metadata-json-lint', '2.0.2'
  gem 'semantic_puppet', '1.0.1' # this gem is needed (for metadata-json-lint) until we move to Puppet 4.9.x
end

group :development, :test do
  gem 'beaker-rspec', '6.1.0', :require => false
  gem 'pry', '0.10.4'
end

Gemfile.lock

GEM
  remote: https://rubygems.org/
  specs:
    CFPropertyList (2.3.5)
    addressable (2.5.1)
      public_suffix (~> 2.0, >= 2.0.2)
    aws-sdk-v1 (1.67.0)
      json (~> 1.4)
      nokogiri (~> 1)
    beaker (3.20.0)
      aws-sdk-v1 (~> 1.57)
      beaker-hiera (~> 0.0)
      beaker-hostgenerator
      beaker-puppet (~> 0.0)
      beaker-vmpooler (~> 0.1)
      docker-api
      fission (~> 0.4)
      fog (~> 1.38)
      google-api-client (~> 0.9)
      hocon (~> 1.0)
      in-parallel (~> 0.1)
      inifile (~> 3.0)
      minitar (~> 0.5.4)
      minitest (~> 5.4)
      net-scp (~> 1.2)
      net-ssh (~> 4.0)
      open_uri_redirections (~> 0.2.1)
      rsync (~> 1.0.9)
      stringify-hash (~> 0.0)
      thor (~> 0.19)
      unf (~> 0.1)
    beaker-hiera (0.1.1)
      stringify-hash (~> 0.0.0)
    beaker-hostgenerator (0.10.1)
      deep_merge (~> 1.0)
      stringify-hash (~> 0.0.0)
    beaker-puppet (0.3.0)
      in-parallel (~> 0.1)
      stringify-hash (~> 0.0.0)
    beaker-rspec (6.1.0)
      beaker (~> 3.0)
      rspec (~> 3.0)
      serverspec (~> 2)
      specinfra (~> 2)
    beaker-vmpooler (0.1.0)
      rbvmomi (~> 1.9)
      stringify-hash (~> 0.0.0)
    builder (3.2.3)
    coderay (1.1.1)
    declarative (0.0.9)
    declarative-option (0.1.0)
    deep_merge (1.1.1)
    diff-lcs (1.3)
    docker-api (1.33.6)
      excon (>= 0.38.0)
      json
    excon (0.57.1)
    facter (2.4.6)
    facterdb (0.3.11)
      facter
      jgrep
    faraday (0.12.1)
      multipart-post (>= 1.2, < 3)
    fast_gettext (1.1.0)
    fission (0.5.0)
      CFPropertyList (~> 2.2)
    fog (1.40.0)
      fog-aliyun (>= 0.1.0)
      fog-atmos
      fog-aws (>= 0.6.0)
      fog-brightbox (~> 0.4)
      fog-cloudatcost (~> 0.1.0)
      fog-core (~> 1.43)
      fog-digitalocean (>= 0.3.0)
      fog-dnsimple (~> 1.0)
      fog-dynect (~> 0.0.2)
      fog-ecloud (~> 0.1)
      fog-google (<= 0.1.0)
      fog-json
      fog-local
      fog-openstack
      fog-powerdns (>= 0.1.1)
      fog-profitbricks
      fog-rackspace
      fog-radosgw (>= 0.0.2)
      fog-riakcs
      fog-sakuracloud (>= 0.0.4)
      fog-serverlove
      fog-softlayer
      fog-storm_on_demand
      fog-terremark
      fog-vmfusion
      fog-voxel
      fog-vsphere (>= 0.4.0)
      fog-xenserver
      fog-xml (~> 0.1.1)
      ipaddress (~> 0.5)
      json (>= 1.8, < 2.0)
    fog-aliyun (0.1.0)
      fog-core (~> 1.27)
      fog-json (~> 1.0)
      ipaddress (~> 0.8)
      xml-simple (~> 1.1)
    fog-atmos (0.1.0)
      fog-core
      fog-xml
    fog-aws (1.4.0)
      fog-core (~> 1.38)
      fog-json (~> 1.0)
      fog-xml (~> 0.1)
      ipaddress (~> 0.8)
    fog-brightbox (0.11.0)
      fog-core (~> 1.22)
      fog-json
      inflecto (~> 0.0.2)
    fog-cloudatcost (0.1.2)
      fog-core (~> 1.36)
      fog-json (~> 1.0)
      fog-xml (~> 0.1)
      ipaddress (~> 0.8)
    fog-core (1.44.3)
      builder
      excon (~> 0.49)
      formatador (~> 0.2)
    fog-digitalocean (0.3.0)
      fog-core (~> 1.42)
      fog-json (>= 1.0)
      fog-xml (>= 0.1)
      ipaddress (>= 0.5)
    fog-dnsimple (1.0.0)
      fog-core (~> 1.38)
      fog-json (~> 1.0)
    fog-dynect (0.0.3)
      fog-core
      fog-json
      fog-xml
    fog-ecloud (0.3.0)
      fog-core
      fog-xml
    fog-google (0.1.0)
      fog-core
      fog-json
      fog-xml
    fog-json (1.0.2)
      fog-core (~> 1.0)
      multi_json (~> 1.10)
    fog-local (0.3.1)
      fog-core (~> 1.27)
    fog-openstack (0.1.21)
      fog-core (>= 1.40)
      fog-json (>= 1.0)
      ipaddress (>= 0.8)
    fog-powerdns (0.1.1)
      fog-core (~> 1.27)
      fog-json (~> 1.0)
      fog-xml (~> 0.1)
    fog-profitbricks (3.0.0)
      fog-core (~> 1.42)
      fog-json (~> 1.0)
    fog-rackspace (0.1.5)
      fog-core (>= 1.35)
      fog-json (>= 1.0)
      fog-xml (>= 0.1)
      ipaddress (>= 0.8)
    fog-radosgw (0.0.5)
      fog-core (>= 1.21.0)
      fog-json
      fog-xml (>= 0.0.1)
    fog-riakcs (0.1.0)
      fog-core
      fog-json
      fog-xml
    fog-sakuracloud (1.7.5)
      fog-core
      fog-json
    fog-serverlove (0.1.2)
      fog-core
      fog-json
    fog-softlayer (1.1.4)
      fog-core
      fog-json
    fog-storm_on_demand (0.1.1)
      fog-core
      fog-json
    fog-terremark (0.1.0)
      fog-core
      fog-xml
    fog-vmfusion (0.1.0)
      fission
      fog-core
    fog-voxel (0.1.0)
      fog-core
      fog-xml
    fog-vsphere (1.11.2)
      fog-core
      rbvmomi (~> 1.9)
    fog-xenserver (0.3.0)
      fog-core
      fog-xml
    fog-xml (0.1.3)
      fog-core
      nokogiri (>= 1.5.11, < 2.0.0)
    formatador (0.2.5)
    gettext (3.2.4)
      locale (>= 2.0.5)
      text (>= 1.3.0)
    gettext-setup (0.26)
      fast_gettext (~> 1.1.0)
      gettext (>= 3.0.2)
      locale
    google-api-client (0.13.1)
      addressable (~> 2.5, >= 2.5.1)
      googleauth (~> 0.5)
      httpclient (>= 2.8.1, < 3.0)
      mime-types (~> 3.0)
      representable (~> 3.0)
      retriable (>= 2.0, < 4.0)
    googleauth (0.5.1)
      faraday (~> 0.9)
      jwt (~> 1.4)
      logging (~> 2.0)
      memoist (~> 0.12)
      multi_json (~> 1.11)
      os (~> 0.9)
      signet (~> 0.7)
    hiera (3.4.0)
    hocon (1.2.5)
    httpclient (2.8.3)
    in-parallel (0.1.17)
    inflecto (0.0.2)
    inifile (3.0.0)
    ipaddress (0.8.3)
    jgrep (1.4.1)
      json
    json (1.8.6)
    json-schema (2.8.0)
      addressable (>= 2.4)
    json_pure (1.8.6)
    jwt (1.5.6)
    little-plugger (1.1.4)
    locale (2.1.2)
    logging (2.2.2)
      little-plugger (~> 1.1)
      multi_json (~> 1.10)
    mcollective-client (2.11.0)
      json
      stomp
      systemu
    memoist (0.16.0)
    metaclass (0.0.4)
    metadata-json-lint (2.0.2)
      json-schema (~> 2.8)
      spdx-licenses (~> 1.0)
    method_source (0.8.2)
    mime-types (3.1)
      mime-types-data (~> 3.2015)
    mime-types-data (3.2016.0521)
    mini_portile2 (2.2.0)
    minitar (0.5.4)
    minitest (5.10.2)
    mocha (1.2.1)
      metaclass (~> 0.0.1)
    multi_json (1.12.1)
    multipart-post (2.0.0)
    net-scp (1.2.1)
      net-ssh (>= 2.6.5)
    net-ssh (4.1.0)
    net-telnet (0.1.1)
    nokogiri (1.8.0)
      mini_portile2 (~> 2.2.0)
    open_uri_redirections (0.2.1)
    os (0.9.6)
    parallel (1.11.2)
    parallel_tests (2.14.2)
      parallel
    pry (0.10.4)
      coderay (~> 1.1.0)
      method_source (~> 0.8.1)
      slop (~> 3.4)
    public_suffix (2.0.5)
    puppet (4.7.0)
      facter (> 2.0, < 4)
      hiera (>= 2.0, < 4)
      json_pure (~> 1.8)
    puppet-lint (2.3.0)
    puppet-syntax (2.4.1)
      rake
    puppetlabs_spec_helper (2.2.0)
      mocha (~> 1.0)
      puppet-lint (~> 2.0)
      puppet-syntax (~> 2.0)
      rspec-puppet (~> 2.0)
    rake (12.0.0)
    rbvmomi (1.11.3)
      builder (~> 3.0)
      json (>= 1.8)
      nokogiri (~> 1.5)
      trollop (~> 2.1)
    representable (3.0.4)
      declarative (< 0.1.0)
      declarative-option (< 0.2.0)
      uber (< 0.2.0)
    retriable (3.0.2)
    rspec (3.6.0)
      rspec-core (~> 3.6.0)
      rspec-expectations (~> 3.6.0)
      rspec-mocks (~> 3.6.0)
    rspec-core (3.6.0)
      rspec-support (~> 3.6.0)
    rspec-expectations (3.6.0)
      diff-lcs (>= 1.2.0, < 2.0)
      rspec-support (~> 3.6.0)
    rspec-its (1.2.0)
      rspec-core (>= 3.0.0)
      rspec-expectations (>= 3.0.0)
    rspec-mocks (3.6.0)
      diff-lcs (>= 1.2.0, < 2.0)
      rspec-support (~> 3.6.0)
    rspec-puppet (2.6.8)
      rspec
    rspec-puppet-facts (1.8.0)
      facter
      facterdb (>= 0.3.0)
      json
      mcollective-client
      puppet
    rspec-support (3.6.0)
    rsync (1.0.9)
    semantic_puppet (1.0.1)
      gettext-setup (>= 0.3)
    serverspec (2.39.1)
      multi_json
      rspec (~> 3.0)
      rspec-its
      specinfra (~> 2.68)
    sfl (2.3)
    signet (0.7.3)
      addressable (~> 2.3)
      faraday (~> 0.9)
      jwt (~> 1.5)
      multi_json (~> 1.10)
    slop (3.6.0)
    spdx-licenses (1.1.0)
    specinfra (2.69.0)
      net-scp
      net-ssh (>= 2.7, < 5.0)
      net-telnet
      sfl
    stomp (1.4.4)
    stringify-hash (0.0.2)
    systemu (2.6.5)
    text (1.3.1)
    thor (0.19.4)
    trollop (2.1.2)
    uber (0.1.0)
    unf (0.1.4)
      unf_ext
    unf_ext (0.0.7.4)
    xml-simple (1.1.5)

PLATFORMS
  ruby

DEPENDENCIES
  beaker-rspec (= 6.1.0)
  metadata-json-lint (= 2.0.2)
  parallel_tests (= 2.14.2)
  pry (= 0.10.4)
  puppet (= 4.7.0)
  puppet-lint (= 2.3.0)
  puppetlabs_spec_helper (= 2.2.0)
  rspec-puppet (= 2.6.8)
  rspec-puppet-facts (= 1.8.0)
  semantic_puppet (= 1.0.1)

BUNDLED WITH
   1.14.6
dandunckelman commented 7 years ago

Stepping thru the code brought me to here: https://github.com/puppetlabs/hiera/blob/3.4.0/lib/hiera/backend.rb#L332

where Hiera checks if fqdn is a key of the facts hash via value.include?(segment)

Because the hash has fqdn as a Symbol and not a String, the check fails, resulting in the Ignoring bad definition in :hierarchy: 'fqdn/' error.

dandunckelman commented 7 years ago

So...maybe this?

throw :no_such_key unless value.include?(segment) or value.include?(segment.to_sym)
dandunckelman commented 7 years ago

I think the question on this project is this: why are the fact keys symbols and not strings?

dandunckelman commented 7 years ago

Created a separate issue for the Hiera project: https://tickets.puppetlabs.com/browse/HI-578

blackknight36 commented 5 years ago

Has anybody ever found a solution for this? My hiera configuration works fine on the puppet master but every time I run rspec tests I get the ignoring bad definition error.