search

inspec / kitchen-inspec

Test-Kitchen Plugin for InSpec
Other
109 stars 58 forks source link

'junit' is not a valid reporter type in chef workstation 20.9.136 #260

Closed hangsu-ma closed 3 years ago

hangsu-ma commented 3 years ago

i suspect the problem is happening because junit reporter is a plugin now, and kitchen inspec create runner with reporter options before loading plugins: https://github.com/inspec/kitchen-inspec/blob/80ef8c11acf9fc2b48d24eb3f75c0fdebf9649f1/lib/kitchen/verifier/inspec.rb#L83-L91

Version:

all

Environment:

Ubuntu 18.04 LTS Chef Workstation version: 20.9.136 Chef Infra Client version: 16.4.41 Chef InSpec version: 4.22.22 Chef CLI version: 3.0.27 Chef Habitat version: 1.6.56 Test Kitchen version: 2.6.0 Cookstyle version: 6.16.10

Scenario:

generate junit report from kitchen verify, same kitchen file works with chef workstation 20.8.125

Steps to Reproduce:

---
driver:
  name: vagrant
  provider: virtualbox

provisioner:
  name: chef_zero
  chef_license: accept-silent
  product_version: 16
  always_update_cookbooks: true

verifier:
  name: inspec
  chef_license: accept-silent
  load_plugins: true
  reporter:
    - junit:./.kitchen/inspec.xml

transport:
  name: ssh
platforms:
  - name: ubuntu-18.04

suites:
  - name: default
    provisioner:
      policyfile: Policyfile.rb
    attributes:
      catme: 'set in kitchen'

Expected Result:

generating junit test report

Actual Result:

error message 

 Message: Failed to complete #verify action: ['junit' is not a valid reporter type.]

see log from below attached:

kitchen verify --debug --log-level=debug

default-ubuntu-1804.log

interesting part:


D      ------Exception-------
D      Class: Kitchen::ActionFailed
D      Message: 1 actions failed.
>>>>>>     Failed to complete #verify action: ['junit' is not a valid reporter type.] on default-ubuntu-1804
D      ----------------------
D      ------Backtrace-------
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/lib/kitchen/command.rb:183:in `report_errors'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/lib/kitchen/command.rb:174:in `run_action'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/lib/kitchen/command/action.rb:35:in `block in call'
D      /opt/chef-workstation/embedded/lib/ruby/2.7.0/benchmark.rb:293:in `measure'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/lib/kitchen/command/action.rb:33:in `call'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/lib/kitchen/cli.rb:51:in `perform'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/lib/kitchen/cli.rb:197:in `block (2 levels) in <class:CLI>'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/thor-1.0.1/lib/thor/command.rb:27:in `run'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/thor-1.0.1/lib/thor/invocation.rb:127:in `invoke_command'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/thor-1.0.1/lib/thor.rb:392:in `dispatch'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/thor-1.0.1/lib/thor/base.rb:485:in `start'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/bin/kitchen:11:in `block in <top (required)>'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/lib/kitchen/errors.rb:170:in `with_friendly_errors'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/bin/kitchen:11:in `<top (required)>'
D      /usr/bin/kitchen:354:in `load'
D      /usr/bin/kitchen:354:in `<main>'
D      ----End Backtrace-----
D      -Composite Exception--
D      Class: Kitchen::ActionFailed
D      Message: Failed to complete #verify action: ['junit' is not a valid reporter type.] on default-ubuntu-1804
D      ----------------------
D      ------Backtrace-------
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.22.22/lib/inspec/config.rb:360:in `block in validate_reporters!'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.22.22/lib/inspec/config.rb:359:in `each'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.22.22/lib/inspec/config.rb:359:in `validate_reporters!'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.22.22/lib/inspec/config.rb:479:in `finalize_parse_reporters'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.22.22/lib/inspec/config.rb:435:in `finalize_options'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.22.22/lib/inspec/config.rb:65:in `initialize'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.22.22/lib/inspec/runner.rb:49:in `new'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/inspec-core-4.22.22/lib/inspec/runner.rb:49:in `initialize'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/kitchen-inspec-2.0.0/lib/kitchen/verifier/inspec.rb:84:in `new'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/kitchen-inspec-2.0.0/lib/kitchen/verifier/inspec.rb:84:in `call'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/lib/kitchen/instance.rb:458:in `block in verify_action'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/lib/kitchen/instance.rb:551:in `synchronize_or_call'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/lib/kitchen/instance.rb:513:in `block in action'
D      /opt/chef-workstation/embedded/lib/ruby/2.7.0/benchmark.rb:293:in `measure'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/lib/kitchen/instance.rb:512:in `action'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/lib/kitchen/instance.rb:450:in `verify_action'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/lib/kitchen/instance.rb:381:in `block (2 levels) in transition_to'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/lib/kitchen/lifecycle_hooks.rb:44:in `run_with_hooks'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/lib/kitchen/instance.rb:380:in `block in transition_to'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/lib/kitchen/instance.rb:379:in `each'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/lib/kitchen/instance.rb:379:in `transition_to'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/lib/kitchen/instance.rb:161:in `verify'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/lib/kitchen/command.rb:197:in `public_send'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/lib/kitchen/command.rb:197:in `run_action_in_thread'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/test-kitchen-2.6.0/lib/kitchen/command.rb:168:in `block (2 levels) in run_action'
D      /opt/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/logging-2.3.0/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
D      ----End Backtrace-----
biz812 commented 3 years ago

I am experiencing the same issue.

I've had to pin back inspec as a short term fix. gem 'inspec', '~> 4.21.0' This looks to be the commit that removes it as a valid option: https://github.com/inspec/inspec/commit/98ae89e6cc747c2b749eb0e761cdc20e1aeb212b

I see it's now a plugin that "included" with inspec but I've no idea how to translate that to my kitchen.yml for use with chef...

djessich commented 3 years ago

I have the same problem. This breaks all my CI server builds unfortunately.

fozboz commented 3 years ago

The switch to the plugin has also broken kitchen-inspec when verifying multiple suites. This error is thrown https://github.com/inspec/inspec/blob/master/lib/inspec/plugin/v2/registry.rb#L75 on all but the first suite, because a unique instance of the Activator class gets created for every suite. The same actually applies to other plugins, too.

clintoncwolfe commented 3 years ago

@fozboz I have created a separate issue for the multiple reporters / multiple suites issue - #262.

clintoncwolfe commented 3 years ago

This issue was resolved by #261 and released in kitchen-inspec 2.6.1, included in Chef Workstation 20.9.158 - https://discourse.chef.io/t/chef-workstation-20-9-158-released/17758 .