undefined method `raw_data` for {"...

[dgoradia]

When I load a regular (unencrypted) data bag chefspec is throwing an error.

recipes/default.rb

include_recipe 'chef-vault'
bag = data_bag_item('devices', 'base')
item = chef_vault_item('foo', 'bar')

spec_helper.rb

require 'chefspec'
require 'chef-vault'
require 'chef-vault/test_fixtures'
require 'chefspec/berkshelf'

def parse_data_bag (path)
  data_bags_path = File.expand_path(File.join(File.dirname(__FILE__), "../test/integration/data_bags"))
  return JSON.parse(File.read("#{data_bags_path}/#{path}"))
end

default_spec.rb

require_relative '../spec_helper'

describe 'test1::default' do

  include ChefVault::TestFixtures.rspec_shared_context

  let(:chef_run) do
    ChefSpec::ServerRunner.new() do |node, server|

      server.create_data_bag('devices', {
        'insight-base' => parse_data_bag('devices/base/base.json')
      })

    end.converge(described_recipe)
  end

  it 'should converge' do
    expect(chef_run).to include_recipe(described_recipe)
  end

end

Error:

ERROR: Failed to load data bag item: "devices" {"public_keys"=>{"encrypted_data"=>"..."}}

Relevant File Content:
----------------------
/var/folders/tk/7s_vzqz50k19df8msmp7ct3c0000gn/T/d20150505-55843-1g0qxsb/cookbooks/test1/recipes/default.rb:

  2:  # Cookbook Name:: test1
  3:  # Recipe:: default
  4:  #
  5:  # Copyright 2015, Atlas Digital LLC
  6:  #
  7:  include_recipe 'chef-vault'
  8:  
  9>> bag = data_bag_item('devices', 'insight-base')
 10:  
 11:  item = chef_vault_item('foo', 'bar')
 12:  

F

Failure/Error: end.converge(described_recipe)
     NoMethodError:
       undefined method `raw_data' for {"public_keys"=>{"encrypted_data"=>"..."}}:Hash

[jf647]

There's a number of approaches to take here, but I'm not sure which makes sense.

The problem is that the gem assumes that anything in test/integration/data_bags is a vault. In the last release, I added stubbing of Chef::DataBagItem.load to return a bogus hash with just a single key of encrypted_data. This was to satisfy the data bag probing logic in cookbooks like sensu:

https://github.com/sensu/sensu-chef/blob/master/libraries/sensu_helpers.rb#L41-45

The side effect is that if you're mixing normal data bags and vault data bags in test/integration, the unencrypted data bags are no longer usable because they appear to just contain the bogus hash. This doesn't affect the chef_vault_item helper because it doesn't stub anything - it just catches an exception from ChefVault::Item.load and falls back to Chef::DataBagItem.load instead.

I'm reluctant to put a hint into the JSON file itself, because then you've got some indicator key hanging around that wouldn't be there in a production configuration. I could provide a mechanism to set an inclusion/exclusion list for vaults (defaulting to everything included, nothing exckuded). You could then do something like this in your spec_helper.rb:

ChefVault::TestFixtures.exclude('devices/insight-base')

Thoughts?

[jf647]

I think I found a seamless fix. Check out the issue_11 branch and see if that does what you need.

[dgoradia]

That works but it adds an extra top level key (raw_data) to the data bag item hash.

{"raw_data"=>{"id"=>"insight-base", ...

[dgoradia]

Would it be possible to only stub chef_vault_item and we can stub any unencrypted data bags ourselves? I think that chef-vault-testfixtures should only try to stub vault items and leave everything else alone. Is that a fair assumption?

For example

  let(:chef_run) do
    ChefSpec::ServerRunner.new() do |node, server|

      server.create_data_bag('devices', {
        'insight-base' => parse_data_bag('devices/base/base.json')
      })

    end.converge(described_recipe)
  end

I'm stubbing the unencrypted data bag myself here.

[jf647]

That's how it worked up to v0.2.0, but then Chef released v1.3.0 of the chef-vault cookbook and their helper changed from doing this:

https://github.com/opscode-cookbooks/chef-vault/blob/v1.2.5/libraries/chef_vault_item.rb#L39-43

to this:

https://github.com/opscode-cookbooks/chef-vault/blob/v1.3.0/libraries/chef_vault_item.rb#L33-49

Which is when I added the additional stubs so that it thinks the data bag containing the vault item also has a side-along item suffixed with _keys.

The stubbing of Chef::DataBagItem.load was added for an issue raised internally because the sensu cookbook does a similar but more extensive probe, where it first checks that the normal data bag item has a key for encrypted_data, then checks for the side-along keys item.

What if I put in a mechanism to turn off that second extra stub? The one for the _keys item probably isn't going to screw anyone up, it's faking out the normal data bag access routines that seem to be the problem here. If I made them off by default and added a class method to turn them on, then we could continue to use them in cookbooks that consume sensu by flipping it on, but it wouldn't break your attempted usage.

[dgoradia]

@jf647 That's sounds like a better approach.

[dgoradia]

Would it be possible to just remove the raw_data key from the hash for the regular data bag item?

[jf647]

I did try removing that, but the data bag item loader blows up without it.

I think I'm close to having something working. One last failing test.

[jf647]

I'm cautiously optimistic about 9d702f8

Take a look and see if that works in your use case. If it does, I just have to clean up some rdoc and README stuff and I can release it as 0.5.0

[dgoradia]

Yes, that works great! It stubs the encrypted data bag only and lets me stub the unencrypted ones myself. :+1:

Thank you @jf647

[mhenrixon]

1. Why do I have to use test/integration/data_bags/foo/bar.json? Errno::ENOENT: No such file or directory @ dir_initialize - test/integration/data_bags is this for integration testing or unit testing?
2. Why do I have to use files at all? Why can't I just stub in the spec setup anymore? Failure/Error: specify { expect(chef_run).to include_recipe('app-server::users') } <Chef::DataBag (class)> received :load with unexpected arguments expected: (:foo) got: ("certificates")

[jf647]

The point of this gem is to use the same JSON data bags used by the chef_vault_item helper for Test-Kitchen from ChefSpec. One file for your vault test data that can be used in both unit and integration testing.

You can look at what the gem did in v0.2.0, where the fake data bags were provided via plugins, but I tired of keeping both the plugin and the TK integration files up to date.

If you don't want to use files and aren't doing integration testing with Test-Kitchen, then you probably don't need this gem - just stub ChefVault::Item.load to return whatever hash you want.