search

matschaffer / knife-solo

DEPRECATED: Please consider using https://knife-zero.github.io/, ansible, or visit https://www.chef.io/ for other ideas
MIT License
787 stars 213 forks source link

Community cookbooks fail to compile after patching them by knife solo #397

Open m1ndful opened 10 years ago

m1ndful commented 10 years ago

Hi,

I have faced some issues during patching my cookbook-set by knife-solo. It prevents to very strange errors with community cookbooks. But the same set of cookbooks is working if we will use it separate from knife solo.

Some examples can be found below:

Below configuration works find on vagrant+berkshelf+chef-solo

  config.vm.provision "chef_solo" do |chef|
  #   chef.cookbooks_path = "../my-recipes/cookbooks"
  #   chef.roles_path = "../my-recipes/roles"
  #   chef.data_bags_path = "../my-recipes/data_bags"
  #   chef.add_recipe "mysql"
  #   chef.add_role "web"
    chef.add_recipe "mysql::client"
    chef.add_recipe "database::mysql"
    chef.add_recipe "mysql::server"
    chef.add_recipe "zabbix"
    chef.add_recipe "zabbix::database"
    chef.add_recipe "zabbix::server"
    chef.add_recipe "zabbix::web"
  #
  #   # You may also specify custom JSON attributes:
  #   chef.json = { mysql_password: "foo" }
    chef.json = {
      "zabbix" => {
        "database" => {
          "dbpassword" => "password"
        },
        "agent" => {
          #"source_url" => "http://heanet.dl.sourceforge.net/project/zabbix/ZABBIX%20Latest%20Stable/2.2.5/zabbix-2.2.5.tar.gz",
          "servers" => ['localhost']
        },
        "server" => {
          #"source_url" => "http://heanet.dl.sourceforge.net/project/zabbix/ZABBIX%20Latest%20Stable/2.2.5/zabbix-2.2.5.tar.gz",
          "version" => "2.2.5",
          "name" => "localhost"
        }
      }
    }
  end

Berksfile:

source 'https://api.berkshelf.com'

cookbook "zabbix", "= 0.8.0", git: "https://github.com/laradji/zabbix"

But with knife solo node configuration and same berksfile it throw different errors: _Node.json_

{
    "run_list": [
        "recipe[mysql::client]",
        "recipe[database::mysql]",
        "recipe[mysql::server]",
        "recipe[zabbix]",
        "recipe[zabbix::database]",
        "recipe[zabbix::server]",
        "recipe[zabbix::web]"
    ],
    "zabbix": {
        "database" : {
          "dbpassword" : "password"
        },
        "agent" : {
          "servers" : [
            "localhost"
          ]
        },
        "server" : {
          "version" : "2.2.5",
          "name" : "localhost"
        }
  },
  "mysql": {
    "server_debian_password": "ilikerandompasswords",
    "server_root_password": "ilikerandompasswords",
    "server_repl_password": "ilikerandompasswords"
  }
}

_chef-solo output while bootstrapping with knife solo_

Running Chef...

Starting Chef Client, version 11.14.0.rc.2
[2014-08-18T10:23:21+01:00] WARN: found a directory phpmyadmin in the cookbook path, but it contains no cookbook files. skipping.
[2014-08-18T10:23:21+01:00] WARN: found a directory php in the cookbook path, but it contains no cookbook files. skipping.
Compiling Cookbooks...

================================================================================
Recipe Compile Error in /home/chef/chef-solo/cookbooks-2/mysql/libraries/provider_mysql_service_freebsd.rb
================================================================================

NameError
---------
uninitialized constant Opscode::Mysql

Cookbook Trace:
---------------
  /home/chef/chef-solo/cookbooks-2/mysql/libraries/provider_mysql_service_freebsd.rb:4:in `<top (required)>'

Relevant File Content:
----------------------
/home/chef/chef-solo/cookbooks-2/mysql/libraries/provider_mysql_service_freebsd.rb:

  1:  require 'chef/provider/lwrp_base'
  2:  require 'shellwords'
  3:
  4>> include Opscode::Mysql::Helpers
  5:
  6:  class Chef
  7:    class Provider
  8:      class MysqlService
  9:        class FreeBSD < Chef::Provider::MysqlService
 10:          use_inline_resources if defined?(use_inline_resources)
 11:
 12:          def whyrun_supported?
 13:            true

Running handlers:
[2014-08-18T10:23:21+01:00] ERROR: Running exception handlers
Running handlers complete
[2014-08-18T10:23:21+01:00] ERROR: Exception handlers complete
[2014-08-18T10:23:21+01:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
Chef Client failed. 0 resources updated in 2.150157662 seconds
[2014-08-18T10:23:21+01:00] ERROR: uninitialized constant Opscode::Mysql
[2014-08-18T10:23:21+01:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
ERROR: RuntimeError: chef-solo failed. See output above.

This can be fixed with previous version of MySQL cookbook in Berksfile(magic bug...), but then it will exit with another error:

Running Chef...

Starting Chef Client, version 11.14.0.rc.2
[2014-08-18T10:39:29+01:00] WARN: found a directory phpmyadmin in the cookbook path, but it contains no cookbook files. skipping.
[2014-08-18T10:39:29+01:00] WARN: found a directory php in the cookbook path, but it contains no cookbook files. skipping.
Compiling Cookbooks...
/home/chef/chef-solo/cookbooks-3/windows/libraries/helper.rb:24: warning: already initialized constant AUTO_RUN_KEY
/home/chef/chef-solo/cookbooks-3/windows/libraries/helper.rb:25: warning: already initialized constant ENV_KEY
Recipe: mysql::ruby
  * execute[apt-get update] action run
    - execute apt-get update
Recipe: build-essential::default
  * execute[apt-get-update-build-essentials] action run (skipped due to not_if)
  * package[build-essential] action install (up to date)
  * package[binutils-doc] action install (up to date)
  * package[autoconf] action install (up to date)
  * package[flex] action install (up to date)
  * package[bison] action install (up to date)
Recipe: mysql::client
  * package[mysql-client] action install (up to date)
  * package[libmysqlclient-dev] action install (up to date)
Recipe: mysql::ruby
  * chef_gem[mysql] action install (up to date)
[2014-08-18T10:39:39+01:00] WARN: Cloning resource attributes for directory[/etc/mysql/conf.d] from prior resource (CHEF-3694)
[2014-08-18T10:39:39+01:00] WARN: Previous directory[/etc/mysql/conf.d]: /home/chef/chef-solo/cookbooks-3/mysql/recipes/server.rb:108:in `block in from_file'
[2014-08-18T10:39:39+01:00] WARN: Current  directory[/etc/mysql/conf.d]: /home/chef/chef-solo/cookbooks-3/mysql/recipes/server.rb:108:in `block in from_file'
[2014-08-18T10:39:39+01:00] WARN: Cloning resource attributes for directory[/var/lib/mysql] from prior resource (CHEF-3694)
[2014-08-18T10:39:39+01:00] WARN: Previous directory[/var/lib/mysql]: /home/chef/chef-solo/cookbooks-3/mysql/recipes/server.rb:108:in `block in from_file'
[2014-08-18T10:39:39+01:00] WARN: Current  directory[/var/lib/mysql]: /home/chef/chef-solo/cookbooks-3/mysql/recipes/server.rb:108:in `block in from_file'
[2014-08-18T10:39:39+01:00] WARN: Cloning resource attributes for service[mysql] from prior resource (CHEF-3694)
[2014-08-18T10:39:39+01:00] WARN: Previous service[mysql]: /home/chef/chef-solo/cookbooks-3/mysql/recipes/server.rb:154:in `from_file'
[2014-08-18T10:39:39+01:00] WARN: Current  service[mysql]: /home/chef/chef-solo/cookbooks-3/mysql/recipes/server.rb:218:in `from_file'

  ================================================================================
  Recipe Compile Error in /home/chef/chef-solo/cookbooks-2/zabbix/recipes/default.rb
  ================================================================================

  NoMethodError
  -------------
  undefined method `strip_components' for Chef::Resource::Ark

  Cookbook Trace:
  ---------------
    /home/chef/chef-solo/cookbooks-2/zabbix/recipes/agent_prebuild.rb:22:in `block in from_file'
    /home/chef/chef-solo/cookbooks-2/zabbix/recipes/agent_prebuild.rb:15:in `from_file'
    /home/chef/chef-solo/cookbooks-2/zabbix/recipes/agent.rb:1:in `from_file'
    /home/chef/chef-solo/cookbooks-2/zabbix/recipes/default.rb:10:in `from_file'

  Relevant File Content:
  ----------------------
  /home/chef/chef-solo/cookbooks-2/zabbix/recipes/agent_prebuild.rb:

   15:  ark 'zabbix_agent' do
   16:    name 'zabbix'
   17:    url node['zabbix']['agent']['prebuild']['url']
   18:    owner node['zabbix']['agent']['user']
   19:    group node['zabbix']['agent']['group']
   20:    action :put
   21:    path '/opt'
   22>>   strip_components 0
   23:    has_binaries ['bin/zabbix_sender', 'bin/zabbix_get', 'sbin/zabbix_agent', 'sbin/zabbix_agentd']
   24:    notifies :restart, 'service[zabbix_agentd]'
   25:    checksum node['zabbix']['agent']['checksum']
   26:  end
   27:

  Running handlers:
[2014-08-18T10:39:39+01:00] ERROR: Running exception handlers
  Running handlers complete
[2014-08-18T10:39:39+01:00] ERROR: Exception handlers complete
[2014-08-18T10:39:39+01:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
  Chef Client failed. 1 resources updated in 13.332625405 seconds
[2014-08-18T10:39:39+01:00] ERROR: undefined method `strip_components' for Chef::Resource::Ark
[2014-08-18T10:39:39+01:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
ERROR: RuntimeError: chef-solo failed. See output above.
m1ndful commented 10 years ago

All of these can be fixed by commenting one line in solo.rb and running chef-solo locally $ sudo chef-solo -c solo.rb -j dna.json

node_name "10.1.1.1"

base = File.expand_path('..', __FILE__)

nodes_path                File.join(base, 'nodes')
role_path                 File.join(base, 'roles')
data_bag_path             File.join(base, 'data_bags')
encrypted_data_bag_secret File.join(base, 'data_bag_key')
environment_path          File.join(base, 'environments')
environment               "_default"
ssl_verify_mode           :verify_none

cookbook_path []
cookbook_path << File.join(base, 'cookbooks-1') # /Users/myusername/.chefdk/gem/ruby/2.1.0/gems/knife-solo-0.4.2/lib/knife-solo/resources/patch_cookbooks
cookbook_path << File.join(base, 'cookbooks-2') # /Users/myusername/.berkshelf/knife-solo/2cbcbb32a9140c335af087b9b2f505f5458944a2
#cookbook_path << File.join(base, 'cookbooks-3') # /Users/myusername/zabbix.host/cookbooks
matschaffer commented 10 years ago

Interesting. Would you be able to post your kitchen to a git repo (or enough of it to cause the problem)? Would probably help diagnosis to be able to reproduce it myself.

matschaffer commented 10 years ago

Pretty sure cookbooks-3 should be the patch cookbooks. So wondering if we have some sort of interaction between the chef-solo-search and something else you having going on.

m1ndful commented 10 years ago

Published my kitchen to: https://github.com/m1ndful/knife-solo-kitchen

matschaffer commented 10 years ago

So reading through the code that sets up cookbook paths and the implementation of add_cookbook_path which uses unshift, the mapping should be:

cookbooks-1 => knife-solo patches
cookbooks-2 => librarian cookbooks
cookbooks-3 => berkshelf cookbooks

Which your repo agrees with. Do you have a Berksfile in your local kitchen? Removing that or running the command with --no-berkshelf should have the same effect as commenting cookbooks-3 in the solo.rb.

m1ndful commented 10 years ago

Yes, I do have Berksfile. And it works normally with --no-berkshelf option. Is it normal behaviour?

matschaffer commented 10 years ago

So you also have a Cheffile? If so, yes. But use of both files is (I believe) quite uncommon.

On Wednesday, August 20, 2014, Mind notifications@github.com wrote:

Yes, I do have Berksfile. And it works normally with --no-berkshelf option. Is it normal behaviour?

— Reply to this email directly or view it on GitHub https://github.com/matschaffer/knife-solo/issues/397#issuecomment-52759645 .

-Mat

about.me/matschaffer

matschaffer commented 10 years ago

Gonna close this out since I suspect the interplay is between librarian-chef & berkshelf rather than knife-solo. Feel free to re-open if you find something to the contrary. Thanks!

m1ndful commented 10 years ago

Sorry for delay, but I do not have Cheffile. Only Berksfile. It's quite strange.

matschaffer commented 10 years ago

I'm unclear on what that extra cookbook dir is then. Do you have a local cookbooks folder referenced from knife.rb?

vglocus commented 9 years ago

I am encountering the same error. 

================================================================================
Recipe Compile Error in /tmp/vagrant-chef/147dc1df76820b45ed8922af85ceb3f9/cookbooks/mysql/libraries/provider_mysql_service_freebsd.rb
================================================================================

NameError
---------
uninitialized constant Opscode::Mysql

Cookbook Trace:
---------------
  /tmp/vagrant-chef/147dc1df76820b45ed8922af85ceb3f9/cookbooks/mysql/libraries/provider_mysql_service_freebsd.rb:4:in `<top (required)>'

Relevant File Content:
----------------------
/tmp/vagrant-chef/147dc1df76820b45ed8922af85ceb3f9/cookbooks/mysql/libraries/provider_mysql_service_freebsd.rb:

  1:  require 'chef/provider/lwrp_base'
  2:  require 'shellwords'
  3:
  4>> include Opscode::Mysql::Helpers
  5:
  6:  class Chef
  7:    class Provider
  8:      class MysqlService
  9:        class FreeBSD < Chef::Provider::MysqlService
 10:          use_inline_resources if defined?(use_inline_resources)
 11:
 12:          def whyrun_supported?
 13:            true

Though I am rather novice to chef. What I find curious is why it tries to run (and fail) with the free bsd service where your variables suggest you are running Debian. I am running this on a cloudstack VM with CentOS and I get the same error (when compiling provider_mysql_service_freebsd.rb).

matschaffer commented 9 years ago

Can you post your kitchen somewhere so I can give it a try? This sorta feels like a bug in the mysql cookbook but I can poke at it a bit and see what I can find. You might also want to show your error to the maintainers of that project.