Rspec output should be configurable

[tolleiv]

Hi,

especially when using test-kitchen / busser within a CI system it's quite odd that the test result output is just part of the stdout stream. It would be great if you could allow to pass other formatter options to rspec / serverspec. So that HTML or junit reports could be generated from the testruns.

Cheers

[lsimons]

+1

My own 'fix' right now is along the lines of https://github.com/lsimons/busser-serverspec/commit/01ca79520e87c14afea7f2f070ec232a9182ff34, patching things up by hard coding in junit report outputs.

[d-higuchi]

i think this feature is not only in busser-serverspec but also in test-kitchen / busser. would you please open issue at test-kitchen for a starter?

[jsirex]

@lsimons, how do you get report from test machine on host? I'm using kitchen-docker and has the same problem. I have report, but it inside vm.

[buettner123]

I needed to format my serverspec results in JUnit xml format (I used the yarjuf gem for doing this). After some research I found out, how to do this with the default busser-serverspec. I added the following lines to the spec_helper.rb of the serverspec tests:

require 'yarjuf'

RSpec.configure do |c|
  [...]
  c.output_stream = File.open('serverspec-result.xml', 'w')
  c.formatter = 'JUnit'
end

Further I had the problem that the busser environment did not install the yarjuf gem on its own. I found a workaround, described here: https://gist.github.com/juanje/9603938.

After this my spec_helper.rb looked like this:

$LOAD_PATH << '../lib' # add ../lib to the $LOAD_PATH so busser-serverspec can find the yarjuf lib
require 'busser/rubygems'
Busser::RubyGems.install_gem('yarjuf', '~> 1.0.6') # version 1.0.6 because busser-serverspec seems to be using rspec 2.x instead of 3.x

require 'serverspec'
require 'yarjuf'

include SpecInfra::Helper::Exec
include SpecInfra::Helper::DetectOS

RSpec.configure do |c|
  if ENV['ASK_SUDO_PASSWORD']
    require 'highline/import'
    c.sudo_password = ask('Enter sudo password: ') { |q| q.echo = false }
  else
    c.sudo_password = ENV['SUDO_PASSWORD']
  end

  c.output_stream = File.open('serverspec-result.xml', 'w')
  c.formatter = 'JUnit'
end

this did the job for me.

[d-higuchi]

FYI, busser-serverspec 0.2.7 now can handle Gemfile. you can install gem you need using Gemfile.

[cstewart87]

@buettner123 Thanks for the tip! My test results are output, but they're on the test-kitchen instance. Is there a way to output the results to the host running test-kitchen?

[buettner123]

@cstewart87 I used this in a kitchen/docker enviroment where I could copy the file from the docker container to the host. If you are using vagrant a synced folder will propably work for you.

[ytsarev]

https://github.com/test-kitchen/test-kitchen/pull/783

Here I implemented generic way to get a file from an instance to host.

[atward]

+1 for this. Though the solution should really be provided upstream in test-kitchen/busser, as it stand the format options are implemented in each busser plugin.

For anyone using TeamCity, CI integration works using rspec-teamcity.

# test/integration/helpers/serverspec/Gemfile
gem 'rspec-teamcity'

# test/integration/helpers/serverspec/spec_helper.rb
require 'rspec/teamcity'
RSpec.configure do |config|
  config.add_formatter Spec::Runner::Formatter::TeamcityFormatter
end

Output isn't pretty as it still includes color + documentation formatter passed in by busser-serverspec, but results are extracted as expected.

[d-higuchi]

you can use shell verifer instead of busser, it lets serverspec run with any options.

• https://github.com/test-kitchen/test-kitchen/pull/741 (it has been merged TK 1.5)
• https://rubygems.org/gems/kitchen-verifier-shell (if you use TK 1.4, install gem)

here is my shell verifer and serverspec example: https://github.com/cl-lab-k/apache2-take/tree/shell_verifier

I have no plan to make any drastic changes to busser-serverspec. And I think shell-verifier + serverspec is better choice for testing cookbooks than busser verifier + busser-serverspec. So I am preparing the document for translation frombusser verifier + busser-serverspec to shell-verifier + serverspec. If the document is prepared, I will put link to it in README.

[d-higuchi]

https://github.com/test-kitchen/busser-serverspec#-notice