Cookbook @recipe_files not found fatal while runing with chef 12.3

[szymonpk]

I'm running simple provisioning

require 'chef/provisioning/fog_driver/driver'

with_chef_server 'https://api.opscode.com/organizations/someorg',
                 client_name: Chef::Config[:node_name],
                 signing_key_filename: Chef::Config[:client_key]

machine 'web1' do
  recipe 'web_server'
  converge true
end

via CHEF_PROFILE=shellyinten chef-client -c ~/.chef/knife.rb provision.rb. Server is booting up properly, chef is running, but the end I'm getting fatal error:

[2015-05-01T17:04:28+02:00] FATAL: Stacktrace dumped to /Users/szymon/.chef/cache/chef-stacktrace.out
Chef Client failed. 1 resources updated in 57.613821 seconds
[2015-05-01T17:04:28+02:00] ERROR: Cookbook @recipe_files not found. If you're loading @recipe_files from another cookbook, make sure you configure the dependency in your metadata
[2015-05-01T17:04:28+02:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

[vinayakbhadage]

Yes I am also facing the same issue with AWS as provider.
For reference , using Chef Client, version 12.3.0

# it is windows.rb
require 'chef/provisioning/aws_driver'

with_driver 'aws::us-east-1'

with_machine_options({
  bootstrap_options: {
    image_id: "ami-f70cdd9c", # default for us-west-1
    instance_type: "t2.small"
  },
  convergence_options: {
    # The following are only available for Windows machines
    install_msi_url: "https://opscode-omnibus-packages.s3.amazonaws.com/windows/2008r2/x86_64/chef-client-12.4.1-1.msi"
  },
  aws_tags: { :key1 => "value", "key2" => "value"},
  is_windows: true # set to true if using a Windows AMI
})

machine 'my-windows-web-7' do
  run_list ['hello-world'] 
end

The command used for is

chef-client.bat -c D:\project\source\chef-repo\.chef\client.rb .\windows.rb

Any solution for this ?

Regards, -Vinayak

[chrisduong]

+1

[szymonpk]

In matter of fact I don't have such issue anymore. I'm not sure how it was solved. Where do you store cookbooks? In same directory from you are running chef-client?

[chrisduong]

No, I'm provisioning with Chef Server. all my cookbooks are in Chef Server

[szymonpk]

And how does your client.rb look like?

[chrisduong]

log_location             STDOUT
node_name                "my-provisioner"
client_key               "/etc/chef/my-provisioner.pem"
chef_server_url          'https://chef-srv02.ireland.internal/organizations/my'
trusted_certs_dir        '/etc/chef/trusted_certs'

[szymonpk]

I've added cookbook_path to mine (I'm always running from chef repo directory so paths are relative)

cookbook_path ['./cookbooks', './site-cookbooks']

Check it please.

[chrisduong]

I added as you suggested, it is still the same error. I pointed to my Chef Repo cookbooks.

[szymonpk]

Ok I see what helped :/ downgrade to 12.2.x

[chrisduong]

kk, thanks. But I stay with 12.4.1. It is still minor. Hope they would fix soon

[szymonpk]

I think the best solution is to use ChefDK, these warnings may be caused by some mixup of chef-provisioning and chef gems versions.

[chrisduong]

I'm actually using ChefDK 0.7.0 for my provisioner.

[tyler-ball]

@chrisduong Can you grab a full stacktrace from a failing run and put it in a gist?

[chrisduong]

Hi @tyler-ball, here is the gist

[tyler-ball]

Okay, it looks like the recipe name is set by https://github.com/chef/chef/blob/1ef5f566480ba84091a12f2b6c5e85f44af54a5b/lib/chef/run_context.rb#L306-L315

I know there were some issues with Chef 12.3.0 - would you mind using 12.4.1 or the latest ChefDK and see if this is still an issue?

[chrisduong]

I'm using the latest one.

$ chef --version
Chef Development Kit Version: 0.7.0
chef-client version: 12.4.1
berks version: 3.2.4

[tyler-ball]

I don't think this has anything to do with provisioning except that users are far more likely to run into this using chef-client -z my_provisioning_recipe.rb. But Its still a bug and I would like to investigate to create a repro (hopefully one without provisioning).

[chrisduong]

Hi,

I've just found out. No matter what I set in client.rb for cookbook_path, when provisioning, the return aws_object always come with the same cookbook_path to "/home/abc/.chef/cache/cookbook"

aws_object("some-name") do
  action [:create]
  retries 0
  retry_delay 2
  default_guard_interpreter :default
  driver #<Chef::Provisioning::AWSDriver::Driver:0x00000008e62708 @driver_url="aws::eu-west-1",@config={:log_location=>#<IO:<STDOUT>>, :node_name=>"my-provisioner", :client_key=>"/home/abc/my/repo/chef-repo/.chef/my-provisioner.pem", :chef_server_url=>"https://chef-srv02.ireland.my.internal/organizations/my", 
:cookbook_path=>**"/home/abcd/.chef/cache/cookbooks"**, 
:trusted_certs_dir=>"/home/abcd/my/repo/chef-repo/.chef/trusted_certs", :force_logger=>false, :force_formatter=>false, :color=>true, :named_run_list=>nil, :config_file=>".chef/client.rb", :log_level=>:info, :specific_recipes=>["/home/abcd/Dropbox/my/repo/chef-repo/cookbooks/s_aws_resources/test/scripts/aws_security_group/dest_is_all_port.rb"], :formatters=>[], :event_handlers=>[], :authentication_protocol_version=>"1.0", :start_handlers=>[], :chef_provisioning=>{}},

[ghost]

Any workaround on this issue? It blocks production deployment process

[nascimento]

+1

[chrisduong]

Hi,

I've just kinda sorted it out, at least for my case. It meant that chef-client cannot found the recipe even though it still execute the the ruby file(provision.rb) as the last argument.

IMHO, Chef client means to run with recipe by looking in cookbook_path, not to execute a ruby file - chef-apply is for doing that job. So the proper way is to pass a recipe, not a ruby file. For e.g.

chef-client -o 'recipe[abd::provisioning_sth]'

Your job is to make sure that you set the correct cookbook_path so chef-client can find it.

[szymonpk]

It worked fine with previous chef iterations, there were no deprecation warnings between versions. It it was unintended feature it should be restored ;-)

[pdehlke]

bump

What @szymonpk said. This should either work as previously or be explicitly documented.

[rainkinz]

Same issue here

[szymonpk]

I don't see this issue on 12.6.0 anymore.

[ramereth]

I still see the issue in 12.7.2.

[ramereth]

Aha! I see what's still not working. If you give chef-client two ruby files, it still breaks but if you give it one file it works again. Easy workaround for me at least.

[stuszynski]

I'm seeing this issue on v12.9.38. We also use multiple files that we pass to chef-client with desired order. Packing everything to one file make it work but it's breaking our current workflow. :/

[equick]

I think I recreated this issue too whilst trying to provision a vagrant host - https://gist.github.com/equick/915010979aefc2f80aef9260a28d8fa1 . It seems to be caused by the two files provided as @ramereth mentioned above although in my case I need both.

[tyler-ball]

I'm not sure chef-client -z was ever supposed to accept multiple recipes to run. You could make a single recipe that includes the others, either using include_recipe, Kernel.load or instance_eval. I'm not sure which will work off the top of my head

[stuszynski]

@tyler-ball Tried Kernel.load with combination of instance_eval but can't make this happen. I guess the included dependencies (configuration) files are still evaluating in some other namespace or maybe I'm doing something wrong.

[2016-05-05T18:11:49+02:00] ERROR: undefined method `with_chef_server' for main:Object

Anyway. Is there any working example for splitting provisioning recipes with the recent versions of chef and provisioning-gem?

[jeevandongre]

Using Chef client 12.10.24 with vagrant provisioner. Issue persists. I separate file for vagrant configuration and another file for recipe. It still says, Cookbook not found.

Cookbook @recipe_files not found. If you're loading @recipe_files from another cookbook, make sure you configure the dependency in your metadata

[Blatwurst]

+1 with client 12.11.18 and vagrant provisioner

[lehn-etracker]

It seems to be related to https://github.com/chef-cookbooks/compat_resource/issues/70

[lamont-granquist]

^ this has nothing to do with compat_resource

[lamont-granquist]

its also very likely that this is due to trying to use resources that require a cookbook structure when running a recipe file on the command line or using chef-apply.

fail.rb:

template "/tmp/foo.xyz" do
  source "foo.xyz.erb"
end

# chef-client -z ./fail.rb
[2016-08-29T09:07:34-07:00] INFO: Started chef-zero at chefzero://localhost:8889 with repository at /Users/lamont/chef
  One version per cookbook

[2016-08-29T09:07:34-07:00] INFO: Forking chef instance to converge...
Starting Chef Client, version 12.13.37
[2016-08-29T09:07:34-07:00] INFO: *** Chef 12.13.37 ***
[2016-08-29T09:07:34-07:00] INFO: Platform: x86_64-linux
[2016-08-29T09:07:34-07:00] INFO: Chef-client pid: 9357
[2016-08-29T09:07:36-07:00] INFO: Run List is []
[2016-08-29T09:07:36-07:00] INFO: Run List expands to []
[2016-08-29T09:07:36-07:00] INFO: Starting Chef Run for lamont
[2016-08-29T09:07:36-07:00] INFO: Running start handlers
[2016-08-29T09:07:36-07:00] INFO: Start handlers complete.
[2016-08-29T09:07:36-07:00] INFO: HTTP Request Returned 404 Not Found: Object not found:
resolving cookbooks for run list: []
[2016-08-29T09:07:36-07:00] INFO: Loading cookbooks []
Synchronizing Cookbooks:
Installing Cookbook Gems:
Compiling Cookbooks...
[2016-08-29T09:07:36-07:00] WARN: Node lamont has an empty run list.
Converging 1 resources
Recipe: @recipe_files::/home/lamont/fail.rb
  * template[/tmp/foo.xyz] action create[2016-08-29T09:07:36-07:00] INFO: Processing template[/tmp/foo.xyz] action create (@recipe_files::/home/lamont/fail.rb line 1)

    ================================================================================
    Error executing action `create` on resource 'template[/tmp/foo.xyz]'
    ================================================================================

    Chef::Exceptions::CookbookNotFound
    ----------------------------------
    Cookbook @recipe_files not found. If you're loading @recipe_files from another cookbook, make sure you configure the dependency in your metadata

    Resource Declaration:
    ---------------------
    # In /home/lamont/fail.rb

      1: template "/tmp/foo.xyz" do
      2:   source "foo.xyz.erb"
      3: end

    Compiled Resource:
    ------------------
    # Declared in /home/lamont/fail.rb:1:in `from_file'

    template("/tmp/foo.xyz") do
      action [:create]
      retries 0
      retry_delay 2
      default_guard_interpreter :default
      source "foo.xyz.erb"
      declared_type :template
      cookbook_name "@recipe_files"
      recipe_name "/home/lamont/fail.rb"
      atomic_update true
      path "/tmp/foo.xyz"
    end

    Platform:
    ---------
    x86_64-linux

[2016-08-29T09:07:36-07:00] INFO: Running queued delayed notifications before re-raising exception

Running handlers:
[2016-08-29T09:07:36-07:00] ERROR: Running exception handlers
Running handlers complete
[2016-08-29T09:07:36-07:00] ERROR: Exception handlers complete
Chef Client failed. 0 resources updated in 01 seconds
[2016-08-29T09:07:36-07:00] FATAL: Stacktrace dumped to /home/lamont/.chef/local-mode-cache/cache/chef-stacktrace.out
[2016-08-29T09:07:36-07:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2016-08-29T09:07:36-07:00] ERROR: template[/tmp/foo.xyz] (@recipe_files::/home/lamont/fail.rb line 1) had an error: Chef::Exceptions::CookbookNotFound: Cookbook @recipe_files not found. If you're loading @recipe_files from another cookbook, make sure you configure the dependency in your metadata
[2016-08-29T09:07:36-07:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

at that point you have to abandon the just-whip-up-a-single-ruby-file approach like chef-apply and create a real cookbook structure with, say, ./cookbooks/test/recipes/default.rb and ./cookbooks/test/templates/foo.xyz.erb and then invoke with chef-client -r -z 'test::default'

unsure what is going on with chef-provisioning to tickle this, but afaik its never worked.

[szymonpk]

@lamont-granquist It worked for us for some time, don't know why ;-) We've moved to full-cookbook convention and it works even better now. It may look like a little bit hassle for smaller environments, but it should be only supported 'way' of running chef-provisioning.

I'm not sure if this ticket should be continued at all.

[lamont-granquist]

It is certainly a really crappy error message at the very least...