search

neillturner / kitchen-verifier-serverspec

A Test Kitchen Serverspec Verifer without having to transit the Busser layer
Other
39 stars 19 forks source link

spec_helper.rb - confused with search locations. #24

Closed AutomationD closed 7 years ago

AutomationD commented 7 years ago

I'm trying to get remote_exec: false mode to get working. I am using username/empty password login, so I had to modify spec_helper.rb. But I don't think spec files are picking it up correctly.

I'm probably doing something wrong. I would appreciate any kind of help. I'm usingMacOS X.

I'm getting test failures for each test. like that:

File "/test-file1"
  should be file (FAILED - 1)

Failures:

  1) File "/test-file1" should be file
     On host `127.0.0.1'
     Failure/Error: it { should be_file }
       expected `File "/test-file1".file?` to return true, got false
       /bin/sh -c test\ -f\ /test-file1

     # ./test/integration/default/serverspec/touchtest_spec.rb:7:in `block (2 levels) in <top (required)>'
tree .
.
└── integration
    └── default
        └── serverspec
            ├── spec_helper.rb
            ├── ssh_spec.rb
            └── touchtest_spec.rb
# touchtest_spec.rb
require_relative 'spec_helper'
require 'serverspec'
set :backend, :exec
set :os, :family => 'linux'

describe file('/test-file1') do
  it { should be_file }
end
# spec_helper.rb
require 'rubygems'
require 'bundler/setup'

require 'serverspec'
require 'pathname'
require 'net/ssh'

RSpec.configure do |config|
  set :host, '127.0.0.1'
  # ssh options at http://net-ssh.github.io/net-ssh/Net/SSH.html#method-c-start
  # ssh via ssh key (only)
  set :ssh_options,
      :user => 'root',
      :password => '',
      :port => '2222',
      # :auth_methods => [ 'publickey', 'password' ],
      :paranoid => false,
      :verbose => :debug
  set :backend, :ssh
  set :request_pty, true
end
# ../kitchen.yml
---
driver:
  name: vagrant
  ssh:
    insert_key: false
    shell: '"/bin/sh"'

  # Ensure no synced directores are needed for caching
  disable_cache: true
  cache_directory:

  customize:
    cableconnected1: 'on'

verifier:
  name: serverspec
  remote_exec: false
  bundler_path: '/usr/local/bin'
  rspec_path: '/usr/local/bin'
  gemfile: Gemfile
  default_pattern: true

suites:
  - name: default
neillturner commented 7 years ago

have a look in my sample repo https://github.com/neillturner/ansible_repo in https://github.com/neillturner/ansible_repo/blob/master/test/integration/ansible/serverspec/apache_spec.rb i include require '/tmp/kitchen/spec/spec_helper.rb'

AutomationD commented 7 years ago

Thank you for your response. Now I can confirm that my spec_helper is loaded correctly.

I'm still seeing issues with actual verification. Is there any way to troubleshoot it? Feels like ssh layer or something like that has issues.

FYI, busser verifier works and I can get test results from my vagrant box.

neillturner commented 7 years ago

well you need to ssh to your vagrant box and see if /test-file1 on host 127.0.0.1 is there and has right permissions. also make sure your test are running on the correct box and you are picking up the spec_helper you think you are. this verifer-serverspec plugin is new so certainly not fully sorted. have not figured out all the issues to do with sudo for example.

AutomationD commented 7 years ago

@neillturner thanks for staying on top of this, and no worries, I'm willing to do testing to make it work :)

so, I just checked, I am able to do kitchen provision, then kitchen login (also ssh root@127.0.0.1 -p 2222 works). I ensured that the test file is in there.

I'm wondering if there is a way to add more debug? I'm terrible with ruby's magic, but maybe we can do some override in the spec_helper.rb?

neillturner commented 7 years ago

when in the .kitchen folder you get files with all the logs to look at -)

AutomationD commented 7 years ago

Same output as -l debug.

it breaks on the actual test running:

 ---Nested Exception---
D      Class: Kitchen::ActionFailed
D      Message: Command "/usr/local/bin/rspec -c -f documentation --default-path  /myproject  -P /myproject/test/integration/default/serverspec/*_spec.rb" failed for <default-myproject>
D      ----------------------
D      ------Backtrace-------
D      /usr/local/lib/ruby/gems/2.4.0/gems/kitchen-verifier-serverspec-0.6.3/lib/kitchen/verifier/serverspec.rb:383:in `rescue in shellout'
D      /usr/local/lib/ruby/gems/2.4.0/gems/kitchen-verifier-serverspec-0.6.3/lib/kitchen/verifier/serverspec.rb:380:in `shellout'
D      /usr/local/lib/ruby/gems/2.4.0/gems/kitchen-verifier-serverspec-0.6.3/lib/kitchen/verifier/serverspec.rb:121:in `serverspec_commands'
D      /usr/local/lib/ruby/gems/2.4.0/gems/kitchen-verifier-serverspec-0.6.3/lib/kitchen/verifier/serverspec.rb:74:in `call'
D      /usr/local/lib/ruby/gems/2.4.0/gems/test-kitchen-1.15.0/lib/kitchen/instance.rb:434:in `block in verify_action'
D      /usr/local/lib/ruby/gems/2.4.0/gems/test-kitchen-1.15.0/lib/kitchen/instance.rb:527:in `synchronize_or_call'
D      /usr/local/lib/ruby/gems/2.4.0/gems/test-kitchen-1.15.0/lib/kitchen/instance.rb:489:in `block in action'
D      /usr/local/Cellar/ruby/2.4.0/lib/ruby/2.4.0/benchmark.rb:293:in `measure'
D      /usr/local/lib/ruby/gems/2.4.0/gems/test-kitchen-1.15.0/lib/kitchen/instance.rb:488:in `action'
D      /usr/local/lib/ruby/gems/2.4.0/gems/test-kitchen-1.15.0/lib/kitchen/instance.rb:426:in `verify_action'
D      /usr/local/lib/ruby/gems/2.4.0/gems/test-kitchen-1.15.0/lib/kitchen/instance.rb:359:in `block in transition_to'
D      /usr/local/lib/ruby/gems/2.4.0/gems/test-kitchen-1.15.0/lib/kitchen/instance.rb:358:in `each'
D      /usr/local/lib/ruby/gems/2.4.0/gems/test-kitchen-1.15.0/lib/kitchen/instance.rb:358:in `transition_to'
D      /usr/local/lib/ruby/gems/2.4.0/gems/test-kitchen-1.15.0/lib/kitchen/instance.rb:157:in `verify'
D      /usr/local/lib/ruby/gems/2.4.0/gems/test-kitchen-1.15.0/lib/kitchen/instance.rb:186:in `block in test'
D      /usr/local/Cellar/ruby/2.4.0/lib/ruby/2.4.0/benchmark.rb:293:in `measure'
D      /usr/local/lib/ruby/gems/2.4.0/gems/test-kitchen-1.15.0/lib/kitchen/instance.rb:182:in `test'
D      /usr/local/lib/ruby/gems/2.4.0/gems/test-kitchen-1.15.0/lib/kitchen/command.rb:197:in `public_send'
D      /usr/local/lib/ruby/gems/2.4.0/gems/test-kitchen-1.15.0/lib/kitchen/command.rb:197:in `run_action_in_thread'
D      /usr/local/lib/ruby/gems/2.4.0/gems/test-kitchen-1.15.0/lib/kitchen/command.rb:169:in `block (2 levels) in run_action'
D      ----End Backtrace-----

If I run rspec with -w I get:

/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/command/redhat/base/package.rb:23: warning: assigned but unused variable - cmd
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/command/darwin/base/package.rb:33: warning: assigned but unused variable - cmd
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/command/darwin/base/package.rb:37: warning: assigned but unused variable - cmd
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/command/gentoo/base/package.rb:17: warning: assigned but unused variable - cmd
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/command/gentoo/base/package.rb:21: warning: assigned but unused variable - cmd
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/command/suse/base/package.rb:14: warning: assigned but unused variable - cmd
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/backend/ssh.rb:122: warning: shadowing outer local variable - ch
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/backend/ssh.rb:125: warning: ambiguous first argument; put parentheses or a space even after `/' operator
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/backend/ssh.rb:130: warning: ambiguous first argument; put parentheses or a space even after `/' operator
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/backend/ssh.rb:136: warning: shadowing outer local variable - ch
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/backend/ssh.rb:137: warning: ambiguous first argument; put parentheses or a space even after `/' operator
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/backend/ssh.rb:141: warning: ambiguous first argument; put parentheses or a space even after `/' operator
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/backend/ssh.rb:149: warning: shadowing outer local variable - ch
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/backend/ssh.rb:153: warning: shadowing outer local variable - ch
/usr/local/lib/ruby/gems/2.4.0/gems/net-ssh-3.2.0/lib/net/ssh/connection/session.rb:465: warning: mismatched indentations at 'end' with 'if' at 462
/usr/local/lib/ruby/gems/2.4.0/gems/net-scp-1.2.1/lib/net/scp.rb:365: warning: shadowing outer local variable - ch
/usr/local/lib/ruby/gems/2.4.0/gems/net-scp-1.2.1/lib/net/scp.rb:366: warning: shadowing outer local variable - ch
/usr/local/lib/ruby/gems/2.4.0/gems/net-scp-1.2.1/lib/net/scp.rb:367: warning: shadowing outer local variable - ch
/usr/local/lib/ruby/gems/2.4.0/gems/net-scp-1.2.1/lib/net/scp.rb:368: warning: shadowing outer local variable - ch
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/backend/powershell/command.rb:9: warning: `&' interpreted as argument prefix
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/backend/powershell/command.rb:31: warning: assigned but unused variable - identity
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/backend/telnet.rb:64: warning: assigned but unused variable - retry_prompt
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/processor.rb:171: warning: assigned but unused variable - match
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/processor.rb:222: warning: assigned but unused variable - match
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @backend not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @env not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @path not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @stdout not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @stderr not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @exit_status not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/serverspec-2.38.0/lib/serverspec/type/file.rb:111: warning: method redefined; discarding old content
/usr/local/lib/ruby/gems/2.4.0/gems/serverspec-2.38.0/lib/serverspec/type/service.rb:34: warning: assigned but unused variable - res
/usr/local/lib/ruby/gems/2.4.0/gems/serverspec-2.38.0/lib/serverspec/type/hadoop_config.rb:22: warning: mismatched indentations at 'end' with 'case' at 17
/usr/local/lib/ruby/gems/2.4.0/gems/serverspec-2.38.0/lib/serverspec.rb:40: warning: method redefined; discarding old failure_lines
/usr/local/lib/ruby/gems/2.4.0/gems/rspec-core-3.5.4/lib/rspec/core/formatters/exception_presenter.rb:146: warning: previous definition of failure_lines was here

File "/etc/dropbear/authorized_keys"
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @env not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @path not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @shell not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @interactive_shell not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @login_shell not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @pre_command not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @stdout not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @stderr not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @exit_status not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @sudo_path not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @disable_sudo not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @sudo_options not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @docker_container_create_options not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @docker_container_exec_options not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @docker_image not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @docker_url not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @lxc not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @ssh_without_env not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @dockerfile_finalizer not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/configuration.rb:53: warning: instance variable @telnet_options not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/backend/exec.rb:72: warning: instance variable @stdout_handler not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/backend/exec.rb:73: warning: instance variable @stderr_handler not initialized
  should be file (FAILED - 1)

Port "22"
/usr/local/lib/ruby/gems/2.4.0/gems/serverspec-2.38.0/lib/serverspec/matcher/be_listening.rb:3: warning: instance variable @with not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/serverspec-2.38.0/lib/serverspec/matcher/be_listening.rb:3: warning: instance variable @local_address not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/backend/exec.rb:72: warning: instance variable @stdout_handler not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/specinfra-2.66.6/lib/specinfra/backend/exec.rb:73: warning: instance variable @stderr_handler not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/serverspec-2.38.0/lib/serverspec/matcher/be_listening.rb:8: warning: instance variable @local_address not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/serverspec-2.38.0/lib/serverspec/matcher/be_listening.rb:9: warning: instance variable @with not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/serverspec-2.38.0/lib/serverspec/matcher/be_listening.rb:8: warning: instance variable @local_address not initialized
/usr/local/lib/ruby/gems/2.4.0/gems/serverspec-2.38.0/lib/serverspec/matcher/be_listening.rb:9: warning: instance variable @with not initialized
  should be listening (FAILED - 2)
neillturner commented 7 years ago

that's disappointing. Well normally when there are errors in the rspec you see the line of the rspec listed and then you correct. As you say it works with busser serverspec then its not suprising that the problem is not an error in your rspec. I suspect it is to do with the version of serverspec or rspec or something.
Are you testing using vagrant and a linux virtualbox image?
I have not tested using a macosx virtualbox image. If you have a test repo on github that i can use i can reproduce.

AutomationD commented 7 years ago

I am using vagrant image that runs LEDE (openwrt fork). Main shell is /bin/sh Could you show me the gem versions / Gemfile that you had working?

Also, in order to prevent environmental issues I installed ruby via rbenv, but having same issues.

So to my understanding, Rspec by itself should work, right?

/usr/local/bin/rspec -c -f documentation --default-path  /myproject  -P /myproject/test/integration/default/serverspec/*_spec.rb

So something seems to be wrong with the spec_helper as it doesn't ssh correctly into the machine. So wondering how we can debug it...

My repo is closed, but you could probably use https://vagrantcloud.com/lifeeth/boxes/openwrt-15.05-x86 which is very similar, but might lack some packages like sudo.

AutomationD commented 7 years ago

Update: I was able to port 22 test passing in this test repo: https://github.com/kireevco/serverspec-test:

(spec helper is form serverspec-init)

$ rspec

Port "80"
/Users/dmitry/.rbenv/versions/2.4.0/lib/ruby/gems/2.4.0/gems/net-ssh-2.9.4/lib/net/ssh/transport/session.rb:67:in `initialize': Object#timeout is deprecated, use Timeout.timeout instead.
/Users/dmitry/.rbenv/versions/2.4.0/lib/ruby/gems/2.4.0/gems/net-ssh-2.9.4/lib/net/ssh/transport/session.rb:84:in `initialize': Object#timeout is deprecated, use Timeout.timeout instead.
/Users/dmitry/.rbenv/versions/2.4.0/lib/ruby/gems/2.4.0/gems/net-ssh-2.9.4/lib/net/ssh/transport/cipher_factory.rb:98: warning: constant OpenSSL::Cipher::Cipher is deprecated
/Users/dmitry/.rbenv/versions/2.4.0/lib/ruby/gems/2.4.0/gems/net-ssh-2.9.4/lib/net/ssh/transport/cipher_factory.rb:72: warning: constant OpenSSL::Cipher::Cipher is deprecated
  should be listening (FAILED - 1)

Port "22"
  should be listening

Failures:

  1) Port "80" should be listening
     Failure/Error: it { should be_listening }
       expected Port "80" to be listening
       /bin/sh -c netstat\ -tunl\ \|\ grep\ --\ :80\\\

     # ./spec/127.0.0.1/sample_spec.rb:27:in `block (2 levels) in <top (required)>'

Finished in 0.04258 seconds (files took 0.34095 seconds to load)
2 examples, 1 failure

Failed examples:

rspec ./spec/127.0.0.1/sample_spec.rb:27 # Port "80" should be listening

(see port 22 should be listening is passing)

But this spec_helper dosn't work with kitchen repo

AutomationD commented 7 years ago

Ok, here's a test repo that Is also failing tests: https://github.com/kireevco/kitchen-test-test

It's using some 3rd party openwrt vagrant vm that uses a socket for serial communication, so you will need to temporarily touch a file: /Users/praneeth/work/openwrt/openwrt-in-vagrant/utils/serial. Also, I had to hard-require spec_helper.rb

Then kitchen test. Then enter root as a password (in my original test there is no password):

kitchen test
-----> Starting Kitchen (v1.15.0)
-----> Cleaning up any prior instances of <default-test-platform>
-----> Destroying <default-test-platform>...
       Finished destroying <default-test-platform> (0m0.00s).
-----> Testing <default-test-platform>
-----> Creating <default-test-platform>...
       Bringing machine 'default' up with 'virtualbox' provider...
       ==> default: Importing base box 'lifeeth/openwrt-15.05-x86'...
==> default: Matching MAC address for NAT networking...
       ==> default: Checking if box 'lifeeth/openwrt-15.05-x86' is up to date...
       ==> default: Setting the name of the VM: kitchen-kitchen-test-test-default-test-platform_default_1485878760992_77371
       ==> default: Fixed port collision for 22 => 2222. Now on port 2200.
       ==> default: Clearing any previously set network interfaces...
       ==> default: Preparing network interfaces based on configuration...
           default: Adapter 1: nat
       ==> default: Forwarding ports...
           default: 22 (guest) => 2200 (host) (adapter 1)
       ==> default: Running 'pre-boot' VM customizations...
       ==> default: Booting VM...
       ==> default: Waiting for machine to boot. This may take a few minutes...
           default: SSH address: 127.0.0.1:2200
           default: SSH username: root
           default: SSH auth method: password
       ==> default: Machine booted and ready!
       ==> default: Checking for guest additions in VM...
           default: No guest additions were detected on the base box for this VM! Guest
           default: additions are required for forwarded ports, shared folders, host only
           default: networking, and more. If SSH fails on this machine, please install
           default: the guest additions and repackage the box to continue.
           default:
           default: This is not an error message; everything may continue to work properly,
           default: in which case you may ignore this message.
       ==> default: Setting hostname...
       ==> default: Machine not provisioned because `--no-provision` is specified.
/Users/dmitry/.rbenv/versions/2.4.0/lib/ruby/gems/2.4.0/gems/net-ssh-2.9.4/lib/net/ssh/transport/session.rb:67:in `initialize': Object#timeout is deprecated, use Timeout.timeout instead.
/Users/dmitry/.rbenv/versions/2.4.0/lib/ruby/gems/2.4.0/gems/net-ssh-2.9.4/lib/net/ssh/transport/session.rb:84:in `initialize': Object#timeout is deprecated, use Timeout.timeout instead.
/Users/dmitry/.rbenv/versions/2.4.0/lib/ruby/gems/2.4.0/gems/net-ssh-2.9.4/lib/net/ssh/transport/cipher_factory.rb:98: warning: constant OpenSSL::Cipher::Cipher is deprecated
/Users/dmitry/.rbenv/versions/2.4.0/lib/ruby/gems/2.4.0/gems/net-ssh-2.9.4/lib/net/ssh/transport/cipher_factory.rb:72: warning: constant OpenSSL::Cipher::Cipher is deprecated
Text will be echoed in the clear. Please install the HighLine or Termios libraries to suppress echoed text.
root@127.0.0.1's password:root
       [SSH] Established
       Vagrant instance <default-test-platform> created.
       Finished creating <default-test-platform> (0m32.96s).
-----> Converging <default-test-platform>...
       Preparing files for transfer
       Preparing script
       Transferring files to <default-test-platform>
       test OK
       Finished converging <default-test-platform> (0m0.01s).
-----> Setting up <default-test-platform>...
       Finished setting up <default-test-platform> (0m0.00s).
-----> Verifying <default-test-platform>...
       [Serverspec] Verify on instance=#<Kitchen::Instance:0x007fd788bad4e0> with state={:hostname=>"127.0.0.1", :port=>"2200", :username=>"root", :last_action=>"setup", :last_error=>nil}
       Installing bundler and serverspec locally on workstation
       Running Serverspec
       Environment variable KITCHEN_HOST value 127.0.0.1
       Environment variable KITCHEN_USERNAME value root
       Environment variable KITCHEN_PASSWORD value root
       Environment variable KITCHEN_PORT value 2222
       Running command:  /Users/dmitry/.rbenv/shims/rspec -c -f documentation --default-path  /Users/dmitry/dev/kitchen-test-test  -P test/integration/default/serverspec/ssh_spec.rb

File "/etc/dropbear/authorized_keys"
  should be file (FAILED - 1)

Port "22"
  should be listening (FAILED - 2)

Failures:

  1) File "/etc/dropbear/authorized_keys" should be file
     On host `127.0.0.1'
     Failure/Error: it { should be_file }
       expected `File "/etc/dropbear/authorized_keys".file?` to return true, got false
       /bin/sh -c test\ -f\ /etc/dropbear/authorized_keys

     # ./test/integration/default/serverspec/ssh_spec.rb:8:in `block (2 levels) in <top (required)>'

  2) Port "22" should be listening
     On host `127.0.0.1'
     Failure/Error: it { should be_listening }
       expected Port "22" to be listening
       /bin/sh -c netstat\ -tunl\ \|\ grep\ --\ :22\\\

     # ./test/integration/default/serverspec/ssh_spec.rb:13:in `block (2 levels) in <top (required)>'

Finished in 0.05844 seconds (files took 0.5087 seconds to load)
2 examples, 2 failures

Failed examples:

rspec ./test/integration/default/serverspec/ssh_spec.rb:8 # File "/etc/dropbear/authorized_keys" should be file
rspec ./test/integration/default/serverspec/ssh_spec.rb:13 # Port "22" should be listening

       Finished verifying <default-test-platform> (0m1.07s).
-----> Destroying <default-test-platform>...
       ==> default: Forcing shutdown of VM...
       ==> default: Destroying VM and associated drives...
       Vagrant instance <default-test-platform> destroyed.
       Finished destroying <default-test-platform> (0m4.01s).
       Finished testing <default-test-platform> (0m38.06s).
-----> Kitchen is finished. (0m38.15s)
neillturner commented 7 years ago

I edited the .kitchen.yml to the minimum:

driver: name: vagrant platforms:

unfortunately i cannot get this to create get the following error. i don't think this virtual box image is any good. if you can the problem with a standard ubuntu or centos image that would be much better

I, [2017-02-12T15:08:06.079185 #4424] INFO -- Kitchen: -----> Starting Kitchen (v1.7.0) D, [2017-02-12T15:08:06.110436 #4424] DEBUG -- Kitchen: [Vagrant command] BEGIN (vagrant --version) D, [2017-02-12T15:08:06.607452 #4424] DEBUG -- Kitchen: [Vagrant command] END (0m0.00s) I, [2017-02-12T15:08:06.638699 #4424] INFO -- Kitchen: -----> Creating ... D, [2017-02-12T15:08:16.727431 #4424] DEBUG -- Kitchen: ------Exception------- D, [2017-02-12T15:08:16.728407 #4424] DEBUG -- Kitchen: Class: Kitchen::ActionFailed D, [2017-02-12T15:08:16.728407 #4424] DEBUG -- Kitchen: Message: Failed to complete #create action: [Expected process to exit with [0], but received '1' ---- Begin output of vagrant up --no-provision --provider virtualbox ---- STDOUT: Bringing machine 'default' up with 'virtualbox' provider... ==> default: Importing base box 'lifeeth/openwrt-15.05-x86'...

==> default: Matching MAC address for NAT networking... ==> default: Checking if box 'lifeeth/openwrt-15.05-x86' is up to date... ==> default: Setting the name of the VM: kitchen-kitchen-test-test-default-test-platform_default_1486912091452_25710 ==> default: Clearing any previously set network interfaces... ==> default: Preparing network interfaces based on configuration... default: Adapter 1: nat ==> default: Forwarding ports... default: 22 (guest) => 2222 (host) (adapter 1) ==> default: Booting VM... STDERR: There was an error while executing VBoxManage, a CLI used by Vagrant for controlling VirtualBox. The command and stderr is shown below.

Command: ["startvm", "d93db0f3-28e6-4c2c-98d1-2979fa54eaf6", "--type", "headless"]

Stderr: VBoxManage.exe: error: NamedPipe#0 failed to create named pipe /Users/praneeth/work/openwrt/openwrt-in-vagrant/utils/serial (VERR_INVALID_NAME)

VBoxManage.exe: error: Details: code E_FAIL (0x80004005), component ConsoleWrap, interface IConsole ---- End output of vagrant up --no-provision --provider virtualbox ---- Ran vagrant up --no-provision --provider virtualbox returned 1] D, [2017-02-12T15:08:16.731340 #4424] DEBUG -- Kitchen: ---Nested Exception--- D, [2017-02-12T15:08:16.731340 #4424] DEBUG -- Kitchen: Class: Kitchen::ShellOut::ShellCommandFailed D, [2017-02-12T15:08:16.732318 #4424] DEBUG -- Kitchen: Message: Expected process to exit with [0], but received '1' ---- Begin output of vagrant up --no-provision --provider virtualbox ---- STDOUT: Bringing machine 'default' up with 'virtualbox' provider... ==> default: Importing base box 'lifeeth/openwrt-15.05-x86'...

==> default: Matching MAC address for NAT networking... ==> default: Checking if box 'lifeeth/openwrt-15.05-x86' is up to date... ==> default: Setting the name of the VM: kitchen-kitchen-test-test-default-test-platform_default_1486912091452_25710 ==> default: Clearing any previously set network interfaces... ==> default: Preparing network interfaces based on configuration... default: Adapter 1: nat ==> default: Forwarding ports... default: 22 (guest) => 2222 (host) (adapter 1) ==> default: Booting VM... STDERR: There was an error while executing VBoxManage, a CLI used by Vagrant for controlling VirtualBox. The command and stderr is shown below.

Command: ["startvm", "d93db0f3-28e6-4c2c-98d1-2979fa54eaf6", "--type", "headless"]

Stderr: VBoxManage.exe: error: NamedPipe#0 failed to create named pipe /Users/praneeth/work/openwrt/openwrt-in-vagrant/utils/serial (VERR_INVALID_NAME)

VBoxManage.exe: error: Details: code E_FAIL (0x80004005), component ConsoleWrap, interface IConsole ---- End output of vagrant up --no-provision --provider virtualbox ---- Ran vagrant up --no-provision --provider virtualbox returned 1 D, [2017-02-12T15:08:16.736228 #4424] DEBUG -- Kitchen: ------Backtrace------- D, [2017-02-12T15:08:16.761645 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.0/lib/kitchen/shell_out.rb:73:in rescue in run_command' D, [2017-02-12T15:08:16.761645 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.0/lib/kitchen/shell_out.rb:61:inrun_command' D, [2017-02-12T15:08:16.762622 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/gems/2.1.0/gems/kitchen-vagrant-0.19.0/lib/kitchen/driver/vagrant.rb:344:in run_command' D, [2017-02-12T15:08:16.763600 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/gems/2.1.0/gems/kitchen-vagrant-0.19.0/lib/kitchen/driver/vagrant.rb:303:inrun' D, [2017-02-12T15:08:16.763600 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/gems/2.1.0/gems/kitchen-vagrant-0.19.0/lib/kitchen/driver/vagrant.rb:375:in run_vagrant_up' D, [2017-02-12T15:08:16.764579 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/gems/2.1.0/gems/kitchen-vagrant-0.19.0/lib/kitchen/driver/vagrant.rb:88:increate' D, [2017-02-12T15:08:16.765556 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.0/lib/kitchen/instance.rb:449:in public_send' D, [2017-02-12T15:08:16.765556 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.0/lib/kitchen/instance.rb:449:inblock in perform_action' D, [2017-02-12T15:08:16.766534 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.0/lib/kitchen/instance.rb:510:in call' D, [2017-02-12T15:08:16.767514 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.0/lib/kitchen/instance.rb:510:inblock in synchronize_or_call' D, [2017-02-12T15:08:16.768488 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.0/lib/kitchen/instance.rb:508:in synchronize' D, [2017-02-12T15:08:16.768488 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.0/lib/kitchen/instance.rb:508:insynchronize_or_call' D, [2017-02-12T15:08:16.769465 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.0/lib/kitchen/instance.rb:478:in block in action' D, [2017-02-12T15:08:16.769465 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/2.1.0/benchmark.rb:279:inmeasure' D, [2017-02-12T15:08:16.770448 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.0/lib/kitchen/instance.rb:477:in action' D, [2017-02-12T15:08:16.771421 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.0/lib/kitchen/instance.rb:449:inperform_action' D, [2017-02-12T15:08:16.771421 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.0/lib/kitchen/instance.rb:359:in create_action' D, [2017-02-12T15:08:16.772400 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.0/lib/kitchen/instance.rb:348:inblock in transition_to' D, [2017-02-12T15:08:16.772400 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.0/lib/kitchen/instance.rb:347:in each' D, [2017-02-12T15:08:16.773376 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.0/lib/kitchen/instance.rb:347:intransition_to' D, [2017-02-12T15:08:16.774354 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.0/lib/kitchen/instance.rb:127:in create' D, [2017-02-12T15:08:16.774354 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.0/lib/kitchen/command.rb:176:inpublic_send' D, [2017-02-12T15:08:16.775331 #4424] DEBUG -- Kitchen: C:/Puppet/sys/ruby/lib/ruby/gems/2.1.0/gems/test-kitchen-1.7.0/lib/kitchen/command.rb:176:in `block (2 levels) in run_action' D, [2017-02-12T15:08:16.775331 #4424] DEBUG -- Kitchen: ----------------------

AutomationD commented 7 years ago

Neil, thank you very much for looking into this. Really appreciate it 🙂

I updated my test-test case:

It feels to me like serverspec is running locally against my mac machine, and not ssh-ing into my vagrant box.

Here is the log (see the /etc/hosts and /etc/init.d tests):

kitchen test
-----> Starting Kitchen (v1.15.0)
-----> Cleaning up any prior instances of <default-ubuntu-1604>
-----> Destroying <default-ubuntu-1604>...
       ==> default: Forcing shutdown of VM...
       ==> default: Destroying VM and associated drives...
       Vagrant instance <default-ubuntu-1604> destroyed.
       Finished destroying <default-ubuntu-1604> (0m4.73s).
-----> Testing <default-ubuntu-1604>
-----> Creating <default-ubuntu-1604>...
       Bringing machine 'default' up with 'virtualbox' provider...
       ==> default: Importing base box 'bento/ubuntu-16.04'...
==> default: Matching MAC address for NAT networking...
       ==> default: Checking if box 'bento/ubuntu-16.04' is up to date...
       ==> default: Setting the name of the VM: kitchen-kitchen-test-test-default-ubuntu-1604_default_1487808071884_17674
       ==> default: Fixed port collision for 22 => 2222. Now on port 2201.
       ==> default: Clearing any previously set network interfaces...
       ==> default: Preparing network interfaces based on configuration...
           default: Adapter 1: nat
       ==> default: Forwarding ports...
           default: 22 (guest) => 2201 (host) (adapter 1)
       ==> default: Booting VM...
       ==> default: Waiting for machine to boot. This may take a few minutes...
           default: SSH address: 127.0.0.1:2201
           default: SSH username: vagrant
           default: SSH auth method: private key
       ==> default: Machine booted and ready!
       ==> default: Checking for guest additions in VM...
       ==> default: Setting hostname...
       ==> default: Machine not provisioned because `--no-provision` is specified.
       [SSH] Established
       Vagrant instance <default-ubuntu-1604> created.
       Finished creating <default-ubuntu-1604> (0m32.70s).
-----> Converging <default-ubuntu-1604>...
       Preparing files for transfer
       Preparing script
       Transferring files to <default-ubuntu-1604>
       test OK
       Finished converging <default-ubuntu-1604> (0m0.03s).
-----> Setting up <default-ubuntu-1604>...
       Finished setting up <default-ubuntu-1604> (0m0.00s).
-----> Verifying <default-ubuntu-1604>...
       [Serverspec] Verify on instance=#<Kitchen::Instance:0x007fbb0808f848> with state={:hostname=>"127.0.0.1", :port=>"2201", :username=>"vagrant", :ssh_key=>"/Users/dmitry/.vagrant.d/insecure_private_key", :last_action=>"setup", :last_error=>nil}
       Environment variable KITCHEN_HOSTNAME value 127.0.0.1
       Environment variable KITCHEN_PORT value 2201
       Environment variable KITCHEN_USERNAME value vagrant
       Environment variable KITCHEN_SSH_KEY value /Users/dmitry/.vagrant.d/insecure_private_key
       Environment variable KITCHEN_LAST_ACTION value setup
       Environment variable KITCHEN_LAST_ERROR value
       Transport Environment variable KITCHEN_USERNAME value root
       Transport Environment variable KITCHEN_PORT value 22
       Installing bundler and serverspec locally on workstation
       Running Serverspec
       Running command: /Users/dmitry/.rbenv/shims/rspec -c -f documentation --default-path  /Users/dmitry/dev/kitchen-test-test  -P test/integration/default/serverspec/ssh_spec.rb

File "/etc/dropbear/authorized_keys"
  should be file (FAILED - 1)

File "/etc/hosts"
  should be file

File "/etc/init.d"
  should be directory (FAILED - 2)

Port "22"
  should be listening (FAILED - 3)

Failures:

  1) File "/etc/dropbear/authorized_keys" should be file
     On host `127.0.0.1'
     Failure/Error: it { should be_file }
       expected `File "/etc/dropbear/authorized_keys".file?` to return true, got false
       /bin/sh -c test\ -f\ /etc/dropbear/authorized_keys

     # ./test/integration/default/serverspec/ssh_spec.rb:8:in `block (2 levels) in <top (required)>'

  2) File "/etc/init.d" should be directory
     On host `127.0.0.1'
     Failure/Error: it { should be_directory }
       expected `File "/etc/init.d".directory?` to return true, got false
       /bin/sh -c test\ -d\ /etc/init.d

     # ./test/integration/default/serverspec/ssh_spec.rb:17:in `block (2 levels) in <top (required)>'

  3) Port "22" should be listening
     On host `127.0.0.1'
     Failure/Error: it { should be_listening }
       expected Port "22" to be listening
       /bin/sh -c netstat\ -tunl\ \|\ grep\ --\ :22\\\

     # ./test/integration/default/serverspec/ssh_spec.rb:22:in `block (2 levels) in <top (required)>'

Finished in 0.08501 seconds (files took 0.58834 seconds to load)
4 examples, 3 failures

Failed examples:

rspec ./test/integration/default/serverspec/ssh_spec.rb:8 # File "/etc/dropbear/authorized_keys" should be file
rspec ./test/integration/default/serverspec/ssh_spec.rb:17 # File "/etc/init.d" should be directory
rspec ./test/integration/default/serverspec/ssh_spec.rb:22 # Port "22" should be listening

>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>>     Verify failed on instance <default-ubuntu-1604>.  Please see .kitchen/logs/default-ubuntu-1604.log for more details
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration
neillturner commented 7 years ago

I put an extra debug message to show if running remotely or locally. upgrade to 0.6.8 and run test again and this will show where running remotely or not.

neillturner commented 7 years ago

closing. no further activity