Using profiles with kitchen-inspec

[benjamink]

I'm not sure if this is a duplicate of #25 but I'm having trouble getting kitchen-inspec to work with profiles.

I have the following directory structure:

test
└── integration
    ├── linux
    │   ├── controls
    │   │   ├── ntp_spec.rb
    │   │   └── selinux_spec.rb
    │   └── inspec.yml
    └── pm-inbound
        └── default_spec.rb

My test/pm-inbound/default_spec.rb looks like this:

require_controls 'linux' do

  control 'ntp-config'

end

inspec seems to work (obviously tests fail here b/c I haven't passed a host):

[git] $ inspec exec test/integration/pm-inbound/default_spec.rb --profiles-path test/integration
FFFF

Failures:

  1) System Package ntp should be installed
     Failure/Error: DEFAULT_FAILURE_NOTIFIER = lambda { |failure, _opts| raise failure }
       expected that `System Package ntp` is installed
     # controls/ntp_spec.rb:8:in `block (3 levels) in load'

  2) Service ntpd should be enabled
     Failure/Error: DEFAULT_FAILURE_NOTIFIER = lambda { |failure, _opts| raise failure }
       expected that `Service ntpd` is enabled
     # controls/ntp_spec.rb:12:in `block (3 levels) in load'

  3) Service ntpd should be installed
     Failure/Error: DEFAULT_FAILURE_NOTIFIER = lambda { |failure, _opts| raise failure }
       expected that `Service ntpd` is installed
     # controls/ntp_spec.rb:13:in `block (3 levels) in load'

  4) Service ntpd should be running
     Failure/Error: DEFAULT_FAILURE_NOTIFIER = lambda { |failure, _opts| raise failure }
       expected that `Service ntpd` is running
     # controls/ntp_spec.rb:14:in `block (3 levels) in load'

Finished in 0.40791 seconds (files took 0.39686 seconds to load)
4 examples, 4 failures

Failed examples:

rspec  # System Package ntp should be installed
rspec  # Service ntpd should be enabled
rspec  # Service ntpd should be installed
rspec  # Service ntpd should be running

However, kitchen-inspec won't work:

[git] $ kitchen verify
-----> Starting Kitchen (v1.6.0)
-----> Verifying <pm-inbound-centos-67>...
       Search `/Users/bkrein/git/ansible/test/integration/pm-inbound` for tests
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: Failed to complete #verify action: [undefined method `control' for #<Inspec::ProfileContext:0x007fbe14677c38>]
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

zlib(finalizer): the stream was freed prematurely.

My .kitchen.yml:

---
driver:
  name: vagrant
  provision_command: sed -i '/tsflags=nodocks/d' /etc/yum.conf

provisioner:
  name: ansible_playbook
  hosts: localhost
  require_chef_for_busser: false
  require_ruby_for_busser: false
  requirements_path: requirements-test.txt

platforms:
  - name: centos-6.7
  - name: centos-7.1

verifier:
  name: inspec

suites:
  - name: pm-inbound
    provisioner:
      name: ansible_playbook
      hosts: pm-inbound-ord
      playbook: pm-inbound.yml
      additional_copy_path:
        - "."

[chris-rock]

@benjamink Thanks for reporting. The reason for that is, that profile inheritance is not supported in kitchen-inspec yet, but it should. We will add this feature. In the meantime, I recommend to test via inspec directly.

[jorhett]

Now that #75 is merged, is this issue resolved @benjamink ?

[nvtkaszpir]

System info

Test Kitchen version 1.11.1 inspec version 0.32.0 kitchen-inspec (0.15.0)

Issue

Cannot get it to work:

07:45:19 $ kitchen verify twemproxy-debian7-lxc
-----> Starting Kitchen (v1.11.1)
-----> Verifying <twemproxy-debian7-lxc>...
       Use `/<redacted>/test/integration/twemproxy` for testing
       Use `/<redacted>/test/integration/twemproxy` for testing
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>>     Failed to complete #verify action: [undefined method `list' for {}:Hash] on twemproxy-debian7-lxc
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

Debug output

Debug output, added two lines to see exackly values od local_suite_files and config[:inspec_tests]:

07:51:32 $ kitchen verify twemproxy-debian7-lxc -l debug
-----> Starting Kitchen (v1.11.1)
D      [Vagrant command] BEGIN (vagrant --version)
D      [Vagrant command] END (0m0.19s)
-----> Verifying <twemproxy-debian7-lxc>...
D      Initialize InSpec
       Use `/<redacted>/test/integration/twemproxy` for testing
D      local_suite_files: '["/<redacted>/test/integration/twemproxy"]'
D      config[:inspec_tests]: '[]'
       Use `/<redacted>/test/integration/twemproxy` for testing
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>>     Failed to complete #verify action: [undefined method `list' for {}:Hash] on twemproxy-debian7-lxc
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

D      ------Exception-------
D      Class: Kitchen::ActionFailed
D      Message: 1 actions failed.
>>>>>>     Failed to complete #verify action: [undefined method `list' for {}:Hash] on twemproxy-debian7-lxc
D      ----------------------
D      ------Backtrace-------
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/command.rb:187:in `report_errors'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/command.rb:178:in `run_action'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/command/action.rb:39:in `block in call'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/2.1.0/benchmark.rb:279:in `measure'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/command/action.rb:37:in `call'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/cli.rb:56:in `perform'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/cli.rb:185:in `block (2 levels) in <class:CLI>'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/thor-0.19.0/lib/thor/command.rb:27:in `run'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/thor-0.19.0/lib/thor/invocation.rb:126:in `invoke_command'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/cli.rb:326:in `invoke_task'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/thor-0.19.0/lib/thor.rb:359:in `dispatch'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/thor-0.19.0/lib/thor/base.rb:440:in `start'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/bin/kitchen:13:in `block in <top (required)>'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/errors.rb:174:in `with_friendly_errors'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/bin/kitchen:13:in `<top (required)>'
D      /home/kaszpir/.rbenv/versions/2.1.3/bin/kitchen:23:in `load'
D      /home/kaszpir/.rbenv/versions/2.1.3/bin/kitchen:23:in `<main>'
D      ----End Backtrace-----
D      -Composite Exception--
D      Class: Kitchen::ActionFailed
D      Message: Failed to complete #verify action: [undefined method `list' for {}:Hash] on twemproxy-debian7-lxc
D      ----------------------
D      ------Backtrace-------
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/inspec-0.32.0/lib/inspec/dsl.rb:37:in `load_spec_files_for_profile'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/inspec-0.32.0/lib/inspec/dsl.rb:15:in `include_controls'
D      /<redacted>/test/integration/twemproxy/controls/profile.rb:1:in `load'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/inspec-0.32.0/lib/inspec/profile_context.rb:82:in `instance_eval'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/inspec-0.32.0/lib/inspec/profile_context.rb:82:in `load'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/inspec-0.32.0/lib/inspec/runner.rb:174:in `add_test_to_context'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/inspec-0.32.0/lib/inspec/runner.rb:135:in `block in add_content'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/inspec-0.32.0/lib/inspec/runner.rb:135:in `each'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/inspec-0.32.0/lib/inspec/runner.rb:135:in `add_content'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/inspec-0.32.0/lib/inspec/runner.rb:110:in `add_profile'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/inspec-0.32.0/lib/inspec/runner.rb:73:in `add_target'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/kitchen-inspec-0.15.0/lib/kitchen/verifier/inspec.rb:72:in `block in call'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/kitchen-inspec-0.15.0/lib/kitchen/verifier/inspec.rb:72:in `each'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/kitchen-inspec-0.15.0/lib/kitchen/verifier/inspec.rb:72:in `call'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/instance.rb:423:in `block in verify_action'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/instance.rb:513:in `call'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/instance.rb:513:in `synchronize_or_call'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/instance.rb:478:in `block in action'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/2.1.0/benchmark.rb:279:in `measure'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/instance.rb:477:in `action'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/instance.rb:415:in `verify_action'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/instance.rb:348:in `block in transition_to'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/instance.rb:347:in `each'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/instance.rb:347:in `transition_to'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/instance.rb:160:in `verify'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/command.rb:201:in `public_send'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/command.rb:201:in `run_action_in_thread'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/command.rb:173:in `block (2 levels) in run_action'
D      ----End Backtrace-----
D      ---Nested Exception---
D      Class: Kitchen::ActionFailed
D      Message: Failed to complete #verify action: [undefined method `list' for {}:Hash]
D      ----------------------
D      ------Backtrace-------
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/inspec-0.32.0/lib/inspec/dsl.rb:37:in `load_spec_files_for_profile'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/inspec-0.32.0/lib/inspec/dsl.rb:15:in `include_controls'
D      /<redacted>/test/integration/twemproxy/controls/profile.rb:1:in `load'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/inspec-0.32.0/lib/inspec/profile_context.rb:82:in `instance_eval'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/inspec-0.32.0/lib/inspec/profile_context.rb:82:in `load'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/inspec-0.32.0/lib/inspec/runner.rb:174:in `add_test_to_context'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/inspec-0.32.0/lib/inspec/runner.rb:135:in `block in add_content'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/inspec-0.32.0/lib/inspec/runner.rb:135:in `each'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/inspec-0.32.0/lib/inspec/runner.rb:135:in `add_content'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/inspec-0.32.0/lib/inspec/runner.rb:110:in `add_profile'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/inspec-0.32.0/lib/inspec/runner.rb:73:in `add_target'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/kitchen-inspec-0.15.0/lib/kitchen/verifier/inspec.rb:72:in `block in call'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/kitchen-inspec-0.15.0/lib/kitchen/verifier/inspec.rb:72:in `each'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/kitchen-inspec-0.15.0/lib/kitchen/verifier/inspec.rb:72:in `call'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/instance.rb:423:in `block in verify_action'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/instance.rb:513:in `call'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/instance.rb:513:in `synchronize_or_call'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/instance.rb:478:in `block in action'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/2.1.0/benchmark.rb:279:in `measure'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/instance.rb:477:in `action'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/instance.rb:415:in `verify_action'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/instance.rb:348:in `block in transition_to'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/instance.rb:347:in `each'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/instance.rb:347:in `transition_to'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/instance.rb:160:in `verify'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/command.rb:201:in `public_send'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/command.rb:201:in `run_action_in_thread'
D      /home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/test-kitchen-1.11.1/lib/kitchen/command.rb:173:in `block (2 levels) in run_action'
D      ----End Backtrace-----

Running test directly in inspec

Running inspec works, becuase we point it into the profile directly, although looks like it really does not care about --profiles-path variable at all, because we point into profile and it has path: ../../xstrema-tools in dependencies (so i could add --profiles-path /dev/null and it still works):

07:50:56 $ inspec exec test/integration/twemproxy -t ssh://root@10.0.3.137 --password root
  ✔  package-bzip2: System Package bzip2 should be installed
        System Package bzip2 should be installed
  ✔  package-screen: System Package screen should be installed
        System Package screen should be installed
  ✔  package-mc: System Package mc should be installed
        System Package mc should be installed
  ✔  package-htop: System Package htop should be installed
        System Package htop should be installed
  ✔  package-ntp: System Package screen should be installed
        System Package screen should be installed
  ✔  service-ntp-enabled: Service ntp should be enabled
        Service ntp should be enabled
  ✔  service-ntp-running: Service ntp should be running
        Service ntp should be running

Profile: InSpec Profile for twemproxy/nutrcacker (twemproxy)
Version: 1.0.0
Target:  ssh://root@10.0.3.137:22

     No tests executed.

Summary: 7 successful, 0 failures, 0 skipped

Directory structure

test
├── integration
│   └── twemproxy
│       ├── controls
│       │   └── profile.rb
│       ├── inspec.yml
│       └── README.md
└── xstream-tools
    ├── controls
    │   ├── bzip2.rb
    │   ├── htop.rb
    │   ├── mc.rb
    │   ├── ntp.rb
    │   └── screen.rb
    ├── inspec.yml
    ├── libraries
    └── README.md

6 directories, 10 files

Contents of other files

Contents of the .kitchen.yml in the parent directory:

---
driver_plugin: lxc

driver:
  name: vagrant
  provider: lxc

transport:
  name: sftp # kitchen-sync does not work properly with test-kitchen 1.4.x
  compression_level: 9

provisioner:
  name: puppet_apply
  manifests_path: puppet/manifests
  modules_path: puppet/modules
  hiera_config_path: puppet/hiera.kitchen.yaml # different file mapping than files uploaded by boilerplate
  hiera_data_path: puppet/hiera
  puppet_verbose: true
  require_chef_for_busser: false

verifier:
  name: inspec
  # profiles_path: inspec/ # commented out: the goggles, they do nothing!

platforms:
# https://github.com/fgrehm/vagrant-lxc-base-boxes

- name: debian.7-lxc
  platform: linux
  driver:
    box: debian7lxc
  driver_config:
    username: root # defaults to "root"
    password: root # defaults to "root"

suites:
  - name: twemproxy
    driver:
    provisioner:
      manifest: twemproxy.pp

Contents of the test/integration/twemproxy/inspec.yml:

name: twemproxy
title: InSpec Profile for twemproxy/nutrcacker
maintainer: Michał Sochoń
copyright: Michał Sochoń
copyright_email: kaszpir@gmail.com
license: Apache 2 license
summary: profile for instance with twemproxy as standalone
version: 0.0.1
supports:
  - os-family: debian
    release: 7
  - platform: linux
depends:
  - name: xstream-tools
    path: ../../xstream-tools

Contents of the test/integration/twemproxy/controls/profile.rb:

include_controls 'xstream-tools' do
end

kitchen diagnose --all

---
timestamp: 2016-09-01 06:15:20 UTC
kitchen_version: 1.11.1
plugins:
  driver:
    Vagrant:
      class: Kitchen::Driver::Vagrant
      version: 0.20.0
      api_version: 2
  provisioner:
    PuppetApply:
      class: Kitchen::Provisioner::PuppetApply
      version:
      api_version:
  transport:
    Sftp:
      class: Kitchen::Transport::Sftp
      version:
      api_version:
  verifier:
    Inspec:
      class: Kitchen::Verifier::Inspec
      version: 0.15.0
      api_version: 1
loader:
  process_erb: true
  process_local: true
  process_global: true
  global_config:
  project_config:
    filename: "/<redacted>/.kitchen.yml"
    raw_data:
      driver_plugin: lxc
      driver:
        name: vagrant
        provider: lxc
      transport:
        name: sftp
        compression_level: 9
      provisioner:
        name: puppet_apply
        manifests_path: puppet/manifests
        modules_path: puppet/modules
        hiera_config_path: puppet/hiera.kitchen.yaml
        hiera_data_path: puppet/hiera
        puppet_verbose: true
        require_chef_for_busser: false
      verifier:
        name: inspec
      platforms:
      - name: debian.7-lxc
        driver:
          box: debian7lxc
        driver_config:
          username: root
          password: root
      - name: trusty-lxc
        driver:
          box: vagrant-lxc-trusty-amd64
        driver_config:
          username: vagrant
          password: vagrant
      suites:
      - name: twemproxy
        driver:
        provisioner:
          manifest: twemproxy.pp
        run_list:
        attributes:
        verifier:
          inspec_tests:
  local_config:
  combined_config:
    filename:
    raw_data:
      driver_plugin: lxc
      driver:
        name: vagrant
        provider: lxc
      transport:
        name: sftp
        compression_level: 9
      provisioner:
        name: puppet_apply
        manifests_path: puppet/manifests
        modules_path: puppet/modules
        hiera_config_path: puppet/hiera.kitchen.yaml
        hiera_data_path: puppet/hiera
        puppet_verbose: true
        require_chef_for_busser: false
      verifier:
        name: inspec
      platforms:
      - name: debian.7-lxc
        driver:
          box: debian7lxc
        driver_config:
          username: root
          password: root
      - name: trusty-lxc
        driver:
          box: vagrant-lxc-trusty-amd64
        driver_config:
          username: vagrant
          password: vagrant
      suites:
      - name: twemproxy
        driver:
        excludes:
        - precise-lxc
        - trusty-lxc
        provisioner:
          manifest: twemproxy.pp
        run_list:
        attributes:
        verifier:
          inspec_tests:
instances:
  twemproxy-debian7-lxc:
    platform:
      os_type: unix
      shell_type: bourne
    state_file:
      hostname: 10.0.3.137
      last_action: verify
      port: '22'
      ssh_key: "/<redacted>/..."
      username: root
    driver:
      boot_timeout:
      box: debian7lxc
      box_check_update:
      box_download_insecure:
      box_url:
      box_version:
      customize: {}
      gui:
      kitchen_root: "/<redacted>"
      linked_clone:
      log_level: :info
      name: vagrant
      network: []
      password: root
      pre_create_command:
      provider: lxc
      provision: false
      ssh: {}
      synced_folders: []
      test_base_path: "/<redacted>/test/integration"
      username: root
      vagrant_binary: vagrant
      vagrantfile_erb: "/home/kaszpir/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/kitchen-vagrant-0.20.0/templates/Vagrantfile.erb"
      vagrantfiles: []
      vm_hostname: twemproxy-debian7-lxc
    provisioner:
      chef_bootstrap_url: https://www.getchef.com/chef/install.sh
      command_prefix:
      custom_facts: {}
      custom_install_command:
      custom_pre_apply_command:
      custom_pre_install_command:
      facter_file:
      facter_version:
      facterlib:
      files_path: files
      fileserver_config_path:
      ftp_proxy:
      hiera_config_path: puppet/hiera.kitchen.yaml
      hiera_data_path: puppet/hiera
      hiera_data_remote_path: "/var/lib/hiera"
      hiera_deep_merge: false
      hiera_eyaml: false
      hiera_eyaml_key_path:
      hiera_eyaml_key_remote_path: "/etc/puppet/secure/keys"
      hiera_package: hiera-puppet
      hiera_version:
      ignore_spec_fixtures: false
      ignored_paths_from_root: []
      install_custom_facts: false
      install_hiera: false
      kitchen_root: "/<redacted>"
      librarian_puppet_ssl_file:
      manifest: twemproxy.pp
      manifests_path: puppet/manifests
      max_retries: 1
      metadata_json_path:
      modulefile_path:
      modules_path: puppet/modules
      name: puppet_apply
      platform: debian.7-lxc
      puppet_apply_command:
      puppet_apt_collections_repo: http://apt.puppetlabs.com/puppetlabs-release-pc1-wheezy.deb
      puppet_apt_repo: http://apt.puppetlabs.com/puppetlabs-release-precise.deb
      puppet_coll_remote_path: "/opt/puppetlabs"
      puppet_config_path:
      puppet_debug: false
      puppet_detailed_exitcodes:
      puppet_enc:
      puppet_environment:
      puppet_git_init:
      puppet_git_pr:
      puppet_logdest:
      puppet_no_sudo: false
      puppet_noop: false
      puppet_omnibus_url: https://raw.githubusercontent.com/petems/puppet-install-shell/master/install_puppet.sh
      puppet_verbose: true
      puppet_version:
      puppet_whitelist_exit_code:
      puppet_yum_collections_repo: http://yum.puppetlabs.com/puppetlabs-release-pc1-el-6.noarch.rpm
      puppet_yum_repo: https://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm
      puppetfile_path:
      remove_puppet_repo: false
      require_chef_for_busser: false
      require_puppet_collections: false
      require_puppet_omnibus: false
      require_puppet_repo: true
      resolve_with_librarian_puppet: true
      retry_on_exit_code: []
      root_path: "/tmp/kitchen"
      spec_files_path:
      spec_files_remote_path: "/etc/puppet/spec"
      sudo: true
      sudo_command: sudo -E
      test_base_path: "/<redacted>/test/integration"
      update_package_repos: true
      wait_for_retry: 30
    transport:
      compression: false
      compression_level: 9
      connection_retries: 5
      connection_retry_sleep: 1
      connection_timeout: 15
      keepalive: true
      keepalive_interval: 60
      kitchen_root: "/<redacted>"
      log_level: :info
      max_ssh_sessions: 9
      max_wait_until_ready: 600
      name: sftp
      port: 22
      ssh_gateway:
      ssh_gateway_username:
      ssh_key:
      test_base_path: "/<redacted>/test/integration"
      username: root
    verifier:
      chef_omnibus_root: "/opt/chef"
      command_prefix:
      ftp_proxy:
      http_proxy:
      https_proxy:
      inspec_tests:
      kitchen_root: "/<redacted>"
      log_level: :info
      name: inspec
      root_path: "/tmp/verifier"
      sudo: true
      sudo_command: sudo -E
      suite_name: twemproxy
      test_base_path: "/<redacted>/test/integration"

[chris-rock]

@nvtkaszpir I looks like this is the same error as https://github.com/chef/kitchen-inspec/issues/101

[nvtkaszpir]

I was wondering if to merge them, but so far I decided to split it.

101 should handle showing more user friendly message about missing suites/profiles - it's more like kitchen suites do not match inspec suites.

66 deals with profiles in general, where kitchen-inspec is unable to find dependencies within inspec profiles.

You can get workaround #66 with not using profiles, but if you got bad suites/platforms then you will hit #101

[chris-rock]

We are just finalizing the dependency management in InSpec, once its ready, it will be available in kitchen-inspec as well. This should cover this issue. For #101 we should output an error, once the transport is not supported.

[nvtkaszpir]

OK, I understand it, right now kitchen-inspect + inspect profiles are not compatible. For now the only workaround is to use multiple tests within suite, and avoiding inspec profiles in tests within suite.

[chris-rock]

@nvtkaszpir You can use InSpec profiles, only the dependency feature is not available yet

[nvtkaszpir]

Ah, that explains it :)

[chris-rock]

It should work, once you remove the depends from inspec.yml

[nvtkaszpir]

Not really, the Failed to complete #verify action: [undefined methodlist' for {}:Hash]` persists.

[chris-rock]

Which version of inspec and kitchen-inspec are you using?

[nvtkaszpir]

Test Kitchen version 1.11.1 inspec version 0.32.0 kitchen-inspec (0.15.0)

[chris-rock]

is the problem also happening if you use the example profile from inspec? https://github.com/chef/inspec/tree/master/examples/profile

[nvtkaszpir]

using chef/inspec/tree/master/examples/profile directly works.

kitchen.yml seciton:

      inspec_tests:
        - test/integration/profile

but if I take another profile (lets name it steve) which has that profile as include:

include_controls 'profile'do
end

and will change in kitchen.yml

      inspec_tests:
        - test/integration/steve

then it dies with error message

here is a package you may want to test: http://b.hlds.pl/inspec/test-kitchen-inspec-issues.zip

[chris-rock]

include_controls is part of the dependency management. The profile need to be available and kitchen inspec does not support the latest features yet. We'll add this feature, once its marked as stable. Can you work around that for now and use two profiles without dependency management?

[chris-rock]

I let you know, once this feature is available in kitchen-inspec so that you could try it.

[nvtkaszpir]

ok, so far I can work without dependency.

[nvtkaszpir]

btw, but it worked before inspec 0.32.0, I know cause I had working kitchen.yml files for aws ;)

[chris-rock]

okay, good to know. Can you provide the simplest example that breaks it. I'd like to add this to our inspec integration tests

[nvtkaszpir]

https://github.com/chef/kitchen-inspec/issues/66#issuecomment-244072665 see at the bottom of the post, unless you need something more?

[jayhendren]

Has there been any further movement on this issue?

[chris-rock]

This should work as expected and has been demonstrated at http://grdnr.io/inspec-and-chef-compliance-as-code/. Sample code is available here as https://github.com/chris-rock/acme-inspec-profile

[jayhendren]

Ok, why is this issue still open, then?

I am unable to write a repository that both runs an Inspec profile via kitchen and exports that profile so it can be included by other profiles in other repositories, and I assumed that was in scope of this issue. I can open a new issue if that is not the case.

[chris-rock]

@Poohblah Do you have your case somewhere open source? Or a gist where we can see that error?

[jayhendren]

I don't have an example ready, but it's pretty straightforward to explain. kitchen-inspec expects a profile to live in test/integration/$suite_name/inspec. However, when including a profile from an inspec.yml file, then inspec expects the included profile to live in the top-level directory of the repo/tarball/whatever. Therefore I am unable to write a profile that can be both run by kitchen-inspec and exported for inclusion in other profiles.

[adamleff]

The ability to have a profile embedded in a cookbook (or other directory structure) exported for inclusion in other profiles as a dependency is not supported, with the exception of the path fetcher and providing a path to the directory where the profile lives. This is obviously difficult to determine depending on how the profile is fetched initially.

The ability to supply a git repository where the profile is not the only contents and not at the top-level of the repo would be a feature enhancement to InSpec itself. If this feature is needed, please feel free to log an issue at chef/inspec. Thank you!