Issue with v2.0.0 (data bag related)

[rmoriz]

With v2.0.0 we're getting the following error when we run our ChefSpec specs for a cookbook that uses ssh_known_hosts:

TypeError
---------
no implicit conversion of Array into String

Relevant File Content:
----------------------
/var/folders/48/v3jpvqg155d2qfktth99_8t40000gp/T/d20141205-96021-7o92di/cookbooks/ssh_known_hosts/recipes/default.rb:

 55:                              }
 56:      end
 57:    end
 58:  end
 59:  
 60:  # Add the data from the data_bag to the list of nodes.
 61:  # We need to rescue in case the data_bag doesn't exist.
 62>> if Chef::DataBag.list.key?('ssh_known_hosts')
 63:    begin
 64:      hosts += data_bag('ssh_known_hosts').collect do |item|
 65:        entry = data_bag_item('ssh_known_hosts', item)
 66:        {
 67:          'fqdn' => entry['fqdn'] || entry['ipaddress'] || entry['hostname'],
 68:          'key'  => entry['rsa'] || entry['dsa']
 69:        }
 70:      end
 71:    rescue

our usage in a recipe:

include_recipe 'ssh_known_hosts'
ssh_known_hosts_entry 'github.com'

no data bags.

[cwebberOps]

@rmoriz think you could get me a failing spec in a gist?

[cwebberOps]

Also, it would be interesting to know what version of chefspec you are running.

[rmoriz]

example-cookbook: https://github.com/rmoriz/ssh_known_hosts_issues_37

output

➜  /tmp  git clone https://github.com/rmoriz/ssh_known_hosts_issues_37
Cloning into 'ssh_known_hosts_issues_37'...
remote: Counting objects: 18, done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 18 (delta 0), reused 18 (delta 0)
Unpacking objects: 100% (18/18), done.
Checking connectivity... done.
➜  /tmp  cd ssh_known_hosts_issues_37 
➜  ssh_known_hosts_issues_37 git:(master) bundle install
Fetching gem metadata from https://rubygems.org/.......
Fetching additional metadata from https://rubygems.org/..
Resolving dependencies...
Using i18n 0.6.11
Using rake 10.4.2
Using json 1.8.1
Using minitest 5.4.3
Using thread_safe 0.3.4
Using addressable 2.3.6
Using ice_nine 0.11.1
Using multipart-post 2.0.0
Using ast 2.0.0
Using buff-extensions 1.0.0
Using hashie 2.1.2
Using buff-ruby_engine 0.1.0
Using slop 3.6.0
Using hitimes 1.2.2
Using nio4r 1.0.1
Using cleanroom 1.0.0
Using minitar 0.5.4
Using retryable 1.3.6
Using buff-ignore 1.1.1
Using erubis 2.7.0
Using mixlib-log 1.6.0
Using net-http-persistent 2.9.4
Using semverse 1.2.1
Using ffi 1.9.6
Using dep-selector-libgecode 1.0.2
Using thor 0.19.1
Using libyajl2 1.2.0
Using rack 1.5.2
Using diff-lcs 1.2.5
Using highline 1.6.21
Using mixlib-cli 1.5.0
Using mime-types 1.25.1
Using mixlib-config 2.1.0
Using mixlib-shellout 1.6.1
Using net-ssh 2.9.1
Using ipaddress 0.8.0
Using systemu 2.6.4
Using wmi-lite 1.0.0
Using plist 3.1.0
Using coderay 1.1.0
Using method_source 0.8.2
Using chef-rewind 0.0.8
Using rspec-support 3.1.2
Using kartograph 0.2.0
Using equalizer 0.0.9
Using multi_json 1.10.1
Using mini_portile 0.6.1
Using rufus-lru 1.0.5
Using yajl-ruby 1.2.1
Using polyglot 0.3.5
Using formatador 0.2.5
Using growl 1.0.3
Using rb-fsevent 0.9.4
Using lumberjack 1.0.9
Using powerpack 0.0.9
Using rainbow 1.99.2
Using safe_yaml 1.0.4
Using knife-solo_data_bag 1.1.0
Using ruby-progressbar 1.7.0
Using ruby_gntp 0.3.4
Using bundler 1.7.6
Using dogapi 1.12.0
Using tzinfo 1.2.2
Using faraday 0.9.0
Using descendants_tracker 0.0.4
Using buff-shell_out 0.2.0
Using varia_model 0.4.0
Using timers 4.0.1
Using parser 2.2.0.pre.8
Using mixlib-authentication 1.3.0
Using rb-inotify 0.9.5
Using dep_selector 1.0.3
Using rest-client 1.6.7
Using ffi-yajl 1.3.1
Using net-ssh-gateway 1.2.0
Using pry 0.10.1
Using net-scp 1.2.1
Using rspec-core 3.1.7
Using rspec-expectations 3.1.2
Using rspec-mocks 3.1.3
Using nokogiri 1.6.5
Using gherkin 2.12.2
Using treetop 1.5.3
Using activesupport 4.1.8
Using chef-handler-datadog 0.6.0
Using sawyer 0.6.0
Using berkshelf-api-client 1.2.0
Using axiom-types 0.1.1
Using resource_kit 0.1.1
Using celluloid 0.16.0
Using solve 1.2.1
Using coercible 1.0.0
Using astrolabe 1.3.0
Using buff-config 1.0.1
Using chef-zero 2.2.1
Using ohai 7.4.0
Using test-kitchen 1.2.1
Using net-ssh-multi 1.2.0
Using rspec 3.1.0
Using foodcritic 4.0.0
Using octokit 3.7.0
Using celluloid-io 0.16.1
Using virtus 1.0.3
Using listen 2.8.3
Using rubocop 0.27.1
Using kitchen-vagrant 0.15.0
Using chef 11.16.4
Using ridley 4.1.0
Using fauxhai 2.2.0
Using droplet_kit 1.1.2
Using guard 2.10.1
Using berkshelf 3.2.1
Using chefspec 4.1.1
Using guard-foodcritic 1.0.3
Using guard-kitchen 0.0.2
Using guard-rubocop 1.2.0
Using guard-rspec 4.3.1
Using kitchen-digitalocean 0.8.2
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.
bundle install  3,26s user 0,32s system 43% cpu 8,185 total
➜  ssh_known_hosts_issues_37 git:(master) rspec
ffi-yajl/json_gem is deprecated, these monkeypatches will be dropped shortly

================================================================================
Recipe Compile Error in /var/folders/48/v3jpvqg155d2qfktth99_8t40000gp/T/d20141205-314-1tvqs7c/cookbooks/test/recipes/default.rb
================================================================================

TypeError
---------
no implicit conversion of Array into String

Cookbook Trace:
---------------
  /var/folders/48/v3jpvqg155d2qfktth99_8t40000gp/T/d20141205-314-1tvqs7c/cookbooks/ssh_known_hosts/recipes/default.rb:62:in `from_file'
  /var/folders/48/v3jpvqg155d2qfktth99_8t40000gp/T/d20141205-314-1tvqs7c/cookbooks/test/recipes/default.rb:9:in `from_file'

Relevant File Content:
----------------------
/var/folders/48/v3jpvqg155d2qfktth99_8t40000gp/T/d20141205-314-1tvqs7c/cookbooks/ssh_known_hosts/recipes/default.rb:

 55:                              }
 56:      end
 57:    end
 58:  end
 59:  
 60:  # Add the data from the data_bag to the list of nodes.
 61:  # We need to rescue in case the data_bag doesn't exist.
 62>> if Chef::DataBag.list.key?('ssh_known_hosts')
 63:    begin
 64:      hosts += data_bag('ssh_known_hosts').collect do |item|
 65:        entry = data_bag_item('ssh_known_hosts', item)
 66:        {
 67:          'fqdn' => entry['fqdn'] || entry['ipaddress'] || entry['hostname'],
 68:          'key'  => entry['rsa'] || entry['dsa']
 69:        }
 70:      end
 71:    rescue

F

Failures:

  1) test::default includes ssh_known_hosts
     Failure/Error: ChefSpec::SoloRunner.new.converge(described_recipe)
     TypeError:
       no implicit conversion of Array into String
     # /var/folders/48/v3jpvqg155d2qfktth99_8t40000gp/T/d20141205-314-1tvqs7c/cookbooks/ssh_known_hosts/recipes/default.rb:62:in `from_file'
     # /var/folders/48/v3jpvqg155d2qfktth99_8t40000gp/T/d20141205-314-1tvqs7c/cookbooks/test/recipes/default.rb:9:in `from_file'
     # ./spec/recipes/default_spec.rb:5:in `block (2 levels) in <top (required)>'
     # ./spec/recipes/default_spec.rb:9:in `block (2 levels) in <top (required)>'

Finished in 4.13 seconds (files took 1.11 seconds to load)
1 example, 1 failure

Failed examples:

rspec ./spec/recipes/default_spec.rb:8 # test::default includes ssh_known_hosts

  No Chef resources found, skipping coverage calculation...
  3,14s user 0,56s system 63% cpu 5,876 total

[rmoriz]

Suspicion: Error only occurs if you don't have any data bag, which is (only) the case in e.g. cookbook chefspec and kitchen runs.

It seems to work with ChefSpec:ServerRunner

[jhx]

I had the same issue as @rmoriz and resolved it by changing to ChefSpec::ServerRunner. I happen to be using ChefSpec HEAD a940bcc because I need the feature where one can specify cache directory.

[stevemartin]

+1 (Chef Solo + Vagrant + Ubuntu 14.04 - not ChefSpec )

[tas50]

You'll need to use the ServerRunner to avoid getting data bag errors or you'll need to mock out the data bag in solo.

[rmoriz]

It's IMHO a kind of a bad default to rely on ServerRunner and not catch the error in the recipe caused by false assumptions.

[mikelococo]

I'm also somewhat stuck by this error.

1. I'm also -1 on the ServerRunner recommendation. It's less well documented, less widely used, and seems to have a fairly regular stream of bugs associated with api-breaking changes in chef-zero occurring in minor releases.
2. Stubbing the data-bags for chef-solo sounds like a reasonable workaround, but dropping "stub_data_bag('ssh_known_hosts').and_return([])" into a before block doesn't seem to be sufficient to complete a run.

Neither of these techniques is demoed in the the chefspec for this cookbook. I'm not able to identify what it does differently from my own spec.

It's also worth bearing in mind that in addition to the data-bag based work this cookbook does, it's also a library cookbook that provides an LWRP for managing host-keys. That's the only part of it that I wish to use and for my use, it would be best if the default recipe was empty. Having to stub out data bag calls in order to get access to an LWRP feels out of proportion.

Edit: If you just want the LWRP's, you don't need to "include" the default recipe at all. Referring to it in metadata.rb is sufficient. In that scenario, the default recipe is never run and ChefSpec is happy. This is what the spec in this repository tests and why it passes with no stubbing. Chefspec tests that work for the default recipe would probably help some folks by way of example, but the aren't needed my LWRP-only use-case.