search

neillturner / kitchen-verifier-serverspec

A Test Kitchen Serverspec Verifer without having to transit the Busser layer
Other
39 stars 19 forks source link

Windows verify #28

Closed mikef-nl closed 7 years ago

mikef-nl commented 7 years ago

Hi Neil,

Could you post an example for testing with Windows systems? From what I have found they need a slightly different spec_helper.rb:

require 'serverspec'
require 'winrm'

set :backend, :winrm
set :os, :family => 'windows'

user = vagrant
pass = vagrant
endpoint = "http://#{ENV['TARGET_HOST']}:5985/wsman"

winrm = ::WinRM::WinRMWebService.new(endpoint, :ssl, :user => user, :pass => pass, :basic_auth_only => true)
winrm.set_timeout 300 # 5 minutes max timeout for any operation
Specinfra.configuration.winrm = winrm
neillturner commented 7 years ago

good question. i asked matt wrock who is the chef windows guru and he said: Hi Neil, I have not run serverspec for at least a year. The snippit seems ok but I really don’t know if anything has happened in serverspec world that changes WinRM settings. Matt i think chef are using inspec more for testing now. I would like to get windows working with serverspec as i have windows working with test kitchen and puppet and ansible. my windows laptop is too underpowered to run a VM with windows server 2102 (Can only run windows nano really). so would need to test with a windows server on AWS. it should be able to to be made to work using in the .kitchen.yml remote_exec: false

mikef-nl commented 7 years ago

Thank for the reply Neil. I tried the suggestion and now receive this error:

-----> Verifying <default-w2k12>...
       [Serverspec] Verify on instance=#<Kitchen::Instance:0x00000000922818> with state={:hostname=>"127.0.0.1", :port=>"55985", :username=>"vagrant", :password=>"vagrant", :rdp_port=>"3389", :last_action=>"setup", :last_error=>nil}
       Environment variable KITCHEN_HOSTNAME value 127.0.0.1
       Environment variable KITCHEN_PORT value 55985
       Environment variable KITCHEN_USERNAME value vagrant
       Environment variable KITCHEN_PASSWORD value vagrant
       Environment variable KITCHEN_RDP_PORT value 3389
       Environment variable KITCHEN_LAST_ACTION value setup
       Environment variable KITCHEN_LAST_ERROR value 
       Transport Environment variable KITCHEN_USERNAME value administrator
       Transport Environment variable KITCHEN_PORT value 5985
       Installing bundler and serverspec locally on workstation
       Running Serverspec
       Running command: 
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>>     Failed to complete #verify action: [No such file or directory - ] on default-w2k12

my test in test/integration/default/serverspec/default_spec.rb is:

require 'spec_helper'

describe 'Temp folder check' do

  describe service('winlogbeat') do
    it { should be_running }
  end

  describe file('C:/tmp') do
    it { should be_directory }
  end

end
neillturner commented 7 years ago

run the kitchen command with -l debug and you will get more messages are you using default_pattern: true in .kitchen.yml if you show me your kitchen.yml file i see what it is doing. I think you are close to getting it to work.

mikef-nl commented 7 years ago

Here we go:


       [Serverspec] Verify on instance=#<Kitchen::Instance:0x0000000107e8f0> with state={:hostname=>"127.0.0.1", :port=>"55985", :username=>"vagrant", :password=>"vagrant", :rdp_port=>"3389", :last_action=>"setup", :last_error=>nil}
       Environment variable KITCHEN_HOSTNAME value 127.0.0.1
       Environment variable KITCHEN_PORT value 55985
       Environment variable KITCHEN_USERNAME value vagrant
       Environment variable KITCHEN_PASSWORD value vagrant
       Environment variable KITCHEN_RDP_PORT value 3389
       Environment variable KITCHEN_LAST_ACTION value setup
       Environment variable KITCHEN_LAST_ERROR value
       Transport Environment variable KITCHEN_USERNAME value administrator
       Transport Environment variable KITCHEN_PORT value 5985
D      Running Serverspec locally on workstation
       Installing bundler and serverspec locally on workstation
       Running Serverspec
       Running command:
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 2 actions failed.
>>>>>>     Converge failed on instance <default-ansibleserver>.  Please see .kitchen/logs/default-ansibleserver.log for more details
>>>>>>     Failed to complete #verify action: [No such file or directory - ] on default-w2k12
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

D      ------Exception-------
D      Class: Kitchen::ActionFailed
D      Message: 2 actions failed.
>>>>>>     Converge failed on instance <default-ansibleserver>.  Please see .kitchen/logs/default-ansibleserver.log for more details
>>>>>>     Failed to complete #verify action: [No such file or directory - ] on default-w2k12
D      ----------------------
D      ------Backtrace-------
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/command.rb:183:in `report_errors'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/command.rb:174:in `run_action'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/command/action.rb:36:in `block in call'
D      /usr/share/ruby/benchmark.rb:281:in `measure'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/command/action.rb:34:in `call'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/cli.rb:53:in `perform'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/cli.rb:187:in `block (2 levels) in <class:CLI>'
D      /usr/local/share/gems/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
D      /usr/local/share/gems/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/cli.rb:334:in `invoke_task'
D      /usr/local/share/gems/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
D      /usr/local/share/gems/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/bin/kitchen:13:in `block in <top (required)>'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/errors.rb:171:in `with_friendly_errors'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/bin/kitchen:13:in `<top (required)>'
D      /usr/local/bin/kitchen:23:in `load'
D      /usr/local/bin/kitchen:23:in `<main>'
D      ----End Backtrace-----
D      -Composite Exception--
D      Class: Kitchen::InstanceFailure
D      Message: Converge failed on instance <default-ansibleserver>.  Please see .kitchen/logs/default-ansibleserver.log for more details
D      ----------------------
D      ------Backtrace-------
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/provisioner/base.rb:83:in `rescue in call'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/provisioner/base.rb:86:in `call'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:384:in `block in converge_action'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:527:in `synchronize_or_call'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:489:in `block in action'
D      /usr/share/ruby/benchmark.rb:281:in `measure'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:488:in `action'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:380:in `converge_action'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:359:in `block in transition_to'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:358:in `each'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:358:in `transition_to'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:157:in `verify'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/command.rb:197:in `public_send'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/command.rb:197:in `run_action_in_thread'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/command.rb:169:in `block (2 levels) in run_action'
D      /usr/local/share/gems/gems/logging-2.2.1/lib/logging/diagnostic_context.rb:474:in `call'
D      /usr/local/share/gems/gems/logging-2.2.1/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
D      ----End Backtrace-----
D      ---Nested Exception---
D      Class: Kitchen::ActionFailed
D      Message: SSH exited (4) for command: [

          ANSIBLE_HOST_KEY_CHECKING=false ANSIBLE_ROLES_PATH=/tmp/kitchen/roles sudo -E -s  ansible-playbook -i /tmp/kitchen/hosts  -c winrm -M /tmp/kitchen/modules          /tmp/kitchen/default.yml
]
D      ----------------------
D      ------Backtrace-------
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/provisioner/base.rb:83:in `rescue in call'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/provisioner/base.rb:86:in `call'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:384:in `block in converge_action'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:527:in `synchronize_or_call'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:489:in `block in action'
D      /usr/share/ruby/benchmark.rb:281:in `measure'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:488:in `action'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:380:in `converge_action'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:359:in `block in transition_to'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:358:in `each'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:358:in `transition_to'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:157:in `verify'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/command.rb:197:in `public_send'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/command.rb:197:in `run_action_in_thread'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/command.rb:169:in `block (2 levels) in run_action'
D      /usr/local/share/gems/gems/logging-2.2.1/lib/logging/diagnostic_context.rb:474:in `call'
D      /usr/local/share/gems/gems/logging-2.2.1/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
D      ----End Backtrace-----
D      -Composite Exception--
D      Class: Kitchen::ActionFailed
D      Message: Failed to complete #verify action: [No such file or directory - ] on default-w2k12
D      ----------------------
D      ------Backtrace-------
D      /usr/local/share/gems/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout/unix.rb:338:in `exec'
D      /usr/local/share/gems/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout/unix.rb:338:in `block in fork_subprocess'
D      /usr/local/share/gems/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout/unix.rb:316:in `fork'
D      /usr/local/share/gems/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout/unix.rb:316:in `fork_subprocess'
D      /usr/local/share/gems/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout/unix.rb:93:in `run_command'
D      /usr/local/share/gems/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout.rb:259:in `run_command'
D      /home/gitlab-runner/.gem/ruby/gems/kitchen-verifier-serverspec-0.6.8/lib/kitchen/verifier/serverspec.rb:388:in `shellout'
D      /home/gitlab-runner/.gem/ruby/gems/kitchen-verifier-serverspec-0.6.8/lib/kitchen/verifier/serverspec.rb:123:in `serverspec_commands'
D      /home/gitlab-runner/.gem/ruby/gems/kitchen-verifier-serverspec-0.6.8/lib/kitchen/verifier/serverspec.rb:76:in `call'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:434:in `block in verify_action'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:527:in `synchronize_or_call'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:489:in `block in action'
D      /usr/share/ruby/benchmark.rb:281:in `measure'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:488:in `action'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:426:in `verify_action'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:359:in `block in transition_to'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:358:in `each'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:358:in `transition_to'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:157:in `verify'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/command.rb:197:in `public_send'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/command.rb:197:in `run_action_in_thread'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/command.rb:169:in `block (2 levels) in run_action'
D      /usr/local/share/gems/gems/logging-2.2.1/lib/logging/diagnostic_context.rb:474:in `call'
D      /usr/local/share/gems/gems/logging-2.2.1/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
D      ----End Backtrace-----
D      ---Nested Exception---
D      Class: Kitchen::ActionFailed
D      Message: Failed to complete #verify action: [No such file or directory - ]
D      ----------------------
D      ------Backtrace-------
D      /usr/local/share/gems/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout/unix.rb:338:in `exec'
D      /usr/local/share/gems/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout/unix.rb:338:in `block in fork_subprocess'
D      /usr/local/share/gems/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout/unix.rb:316:in `fork'
D      /usr/local/share/gems/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout/unix.rb:316:in `fork_subprocess'
D      /usr/local/share/gems/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout/unix.rb:93:in `run_command'
D      /usr/local/share/gems/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout.rb:259:in `run_command'
D      /home/gitlab-runner/.gem/ruby/gems/kitchen-verifier-serverspec-0.6.8/lib/kitchen/verifier/serverspec.rb:388:in `shellout'
D      /home/gitlab-runner/.gem/ruby/gems/kitchen-verifier-serverspec-0.6.8/lib/kitchen/verifier/serverspec.rb:123:in `serverspec_commands'
D      /home/gitlab-runner/.gem/ruby/gems/kitchen-verifier-serverspec-0.6.8/lib/kitchen/verifier/serverspec.rb:76:in `call'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:434:in `block in verify_action'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:527:in `synchronize_or_call'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:489:in `block in action'
D      /usr/share/ruby/benchmark.rb:281:in `measure'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:488:in `action'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:426:in `verify_action'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:359:in `block in transition_to'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:358:in `each'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:358:in `transition_to'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/instance.rb:157:in `verify'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/command.rb:197:in `public_send'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/command.rb:197:in `run_action_in_thread'
D      /usr/local/share/gems/gems/test-kitchen-1.16.0/lib/kitchen/command.rb:169:in `block (2 levels) in run_action'
D      /usr/local/share/gems/gems/logging-2.2.1/lib/logging/diagnostic_context.rb:474:in `call'
D      /usr/local/share/gems/gems/logging-2.2.1/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
D      ----End Backtrace-----
mikef-nl commented 7 years ago

Seems it couldn't find the spec_helper.rb. Now we're getting closer:

       Running Serverspec
       Using default pattern /home/gitlab-runner/builds/4efd7d24/0/delta_roles/delta-winlogbeat/test/integration/default/serverspec/*_spec.rb
       Running command: rspec -c -f documentation --default-path  /home/gitlab-runner/builds/4efd7d24/0/delta_roles/delta-winlogbeat  -P /home/gitlab-runner/builds/4efd7d24/0/delta_roles/delta-winlogbeat/test/integration/default/serverspec/*_spec.rb
/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- spec_helper (LoadError)
    from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
    from /home/gitlab-runner/builds/4efd7d24/0/delta_roles/delta-winlogbeat/test/integration/default/serverspec/default_spec.rb:1:in `<top (required)>'
    from /usr/local/share/gems/gems/rspec-core-3.5.4/lib/rspec/core/configuration.rb:1435:in `load'
    from /usr/local/share/gems/gems/rspec-core-3.5.4/lib/rspec/core/configuration.rb:1435:in `block in load_spec_files'
    from /usr/local/share/gems/gems/rspec-core-3.5.4/lib/rspec/core/configuration.rb:1433:in `each'
    from /usr/local/share/gems/gems/rspec-core-3.5.4/lib/rspec/core/configuration.rb:1433:in `load_spec_files'
    from /usr/local/share/gems/gems/rspec-core-3.5.4/lib/rspec/core/runner.rb:100:in `setup'
    from /usr/local/share/gems/gems/rspec-core-3.5.4/lib/rspec/core/runner.rb:86:in `run'
    from /usr/local/share/gems/gems/rspec-core-3.5.4/lib/rspec/core/runner.rb:71:in `run'
    from /usr/local/share/gems/gems/rspec-core-3.5.4/lib/rspec/core/runner.rb:45:in `invoke'
    from /usr/local/share/gems/gems/rspec-core-3.5.4/exe/rspec:4:in `<top (required)>'
    from /usr/local/bin/rspec:23:in `load'
    from /usr/local/bin/rspec:23:in `<main>'
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
mikef-nl commented 7 years ago

Meant to add the .kitchen.yml:

---
driver:
  name: vagrant
  gui: false

platforms:
  - name: ansibleserver
    driver_plugin: vagrant
    driver_config:
      box: centos7
      network:
      - [ 'private_network', { ip: '172.28.128.10' } ]
    transport:
      max_ssh_sessions: 1
    provisioner:
      name: ansible_playbook
      roles_path: ./
      role_name: delta-winlogbeat
      ansible_connection: winrm
      ansible_inventory: test/integration/inventory/hosts
      require_windows_support: true
      require_chef_for_busser: false
      require_pip: true
      ansible_host_key_checking: false
      ansible_verbose: false
      ansible_verbosity: 4
      ansible_version: 2.2.1
      playbook: default.yml
    verifier:
      name: shell
      command: echo "$(ansible --version)"

  - name: w2k12
    transport:
      name: winrm
    driver_plugin: vagrant
    driver_config:
      vm_hostname: false
      gui: false
      box: w2k12
      communicator: winrm
      network:
      - [ 'private_network', { ip: '172.28.128.11' } ]
      provision: true
      vagrantfiles:
        - test/integration/vagrant_windows_target.rb
    provisioner:
      name: shell
    verifier:
      name: serverspec
      remote_exec: false
      default_pattern: true

suites:
  - name: default
mikef-nl commented 7 years ago

Further thought, in order to create the tests do you need to use serverspec-init ?

neillturner commented 7 years ago

I would change require 'spec_helper' to the full path require 'fullpathoffile/spec_helper' then when it works you experiment with using a relative path like ../..

mikef-nl commented 7 years ago

Hi Neil, when you say relative path...relative to the .kitchen.yml ? ├───defaults ├───files ├───handlers ├───meta ├───tasks ├───templates ├───test │ └───integration │ ├───default │ │ └───serverspec │ │ └───spec_helper.rb │ └───inventory │ └───groups_vars ├───tests └───vars └───.kitchen.yml

neillturner commented 7 years ago

yes the root path is the directory with the .kitchen.yml file

mikef-nl commented 7 years ago

ok so having looked at examples, do I need a spec folder in the root path with the spec_helper in it?

neillturner commented 7 years ago

see examples https://github.com/neillturner/ansible_vagrant_repo https://github.com/neillturner/puppet_vagrant_repo

mikef-nl commented 7 years ago

Ok those I see but neither uses remote_exec: false. I can see both use the spec folder for the spec_helper and seem to be included be full path. However in the case of the non-remote exec my default_spec.rb in test>integration>default would then call spec_helper with the relative path of ../../../spec/spec_helper.

neillturner commented 7 years ago

i haven't really figured out a good way to do this.

mikef-nl commented 7 years ago

Finally got further. You were right about using the full path for the spec_helper require. I'm running it with Gitlab CI so luckily my project directory is an environmental variable I can call to get the full path.

mikef-nl commented 7 years ago

Awesome finally got it working with spec file similar to this: https://github.com/josephkumarmichael/centos-serverspec-windows-testbed/blob/master/tests/spec/spec_helper.rb in the spec folder. Then full path to the spec_helper in the default_spec test.

neillturner commented 7 years ago

thanks for that. i updated the readme with the links you found. closing.