search

matschaffer / knife-solo

DEPRECATED: Please consider using https://knife-zero.github.io/, ansible, or visit https://www.chef.io/ for other ideas
MIT License
786 stars 213 forks source link

knife google server create => uninitialized constant KnifeSolo::Bootstraps #534

Open osimmasgard opened 6 years ago

osimmasgard commented 6 years ago

Version:

Chef Development Kit Version: 3.1.0 chef-client version: 14.2.0 delivery version: master (6862f27aba89109a9630f0b6c6798efec56b4efe) berks version: ERROR kitchen version: 1.22.0 inspec version: 2.1.72

knife-google version: 3.2.0 knife-solo version: 0.7.0.pre3, 0.6.0

Arch Linux box with kernel: 4.17.4-1-ARCH and ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]

Scenario:

knife google server create little-waterfall --config .chef/knife-google.rb --gce-use-private-ip --gce-public-ip none --gce-image _obfuscated_ -m f1-micro --gce-network _obfuscated_ --gce-subnet eu-west1b-private --ssh-user _obfuscated_ --identity-file _obfuscated_ --run-list "role[base]" --gce-tags base

Content of knife-google.rb

knife[:berkshelf_path] = "cookbooks"
knife[:solo] = true
knife[:ssh_user] = "_obfuscated_"
Chef::Config[:ssl_verify_mode] = :verify_peer if defined? ::Chef

# GCP
knife[:gce_project] = '_obfuscated_'
knife[:gce_zone]    = 'europe-west1-b'
knife[:machine_type] = 'f1-micro'

Result:

After running above command, this is the output:

Creating instance...

Current status: PENDING.
Current status: RUNNING.

Instance created!
Instance Name: little-waterfall
Status: RUNNING
Machine Type: f1-micro
Project: _obfuscated_
Zone: europe-west1-b
Network: _obfuscated_
Private IP: _obfuscated_
Public IP: unknown
Bootstrapping the server by using bootstrap_protocol: ssh and image_os_type: linux

Waiting for sshd to host (_obfuscated_).done
WARNING: `--identity-file` is deprecated, please use `--ssh-identity-file`.
WARNING: `--identity-file` is deprecated, please use `--ssh-identity-file`.
Bootstrapping Chef...
FATAL: Check if --bootstrap-protocol and --image-os-type is correct. uninitialized constant KnifeSolo::Bootstraps
Traceback (most recent call last):
    10: from /opt/chefdk/embedded/bin/knife:23:in `<main>'
     9: from /opt/chefdk/embedded/bin/knife:23:in `load'
     8: from /opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.2.0/bin/knife:25:in `<top (required)>'
     7: from /opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.2.0/lib/chef/application/knife.rb:160:in `run'
     6: from /opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.2.0/lib/chef/knife.rb:220:in `run'
     5: from /opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.2.0/lib/chef/knife.rb:441:in `run_with_pretty_exceptions'
     4: from /opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.2.0/lib/chef/local_mode.rb:44:in `with_server_connectivity'
     3: from /opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/chef-14.2.0/lib/chef/knife.rb:442:in `block in run_with_pretty_exceptions'
     2: from /opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/knife-cloud-1.2.1/lib/chef/knife/cloud/command.rb:54:in `run'
     1: from /opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/knife-cloud-1.2.1/lib/chef/knife/cloud/server/create_command.rb:79:in `after_exec_command'
/opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/knife-cloud-1.2.1/lib/chef/knife/cloud/server/create_command.rb:91:in `rescue in after_exec_command': Check if --bootstrap-protocol and --image-os-type is correct. uninitialized constant KnifeSolo::Bootstraps (NameError)

I can't find a way past the "uninitialized constant KnifeSolo::Bootstraps" error, what could be wrong? Let me know if you need more info about my environment etc

Thanks!

matschaffer commented 6 years ago

Ah interesting. We did testing ages ago for knife bootstrap support with ec2, but never knife-google or knife-cloud. Normally bootstraps would be defined in https://github.com/matschaffer/knife-solo/blob/master/lib/knife-solo/bootstraps.rb - we'll need to find out how it's being wrapped though.

Looks like knife-cloud has it's own bootstrap components - https://github.com/chef/knife-cloud/blob/master/lib/chef/knife/cloud/chefbootstrap/bootstrapper.rb

Have you tried uninstalling knife-solo and working with the knife-cloud bootstrappers instead?

osimmasgard commented 6 years ago

Hi,

thanks for the quick reply! We are using knife ec2 a lot and recently started testing with google cloud platform. I will do as you propose and test with the knife-cloud bootstrappers. Will get back with the results..

osimmasgard commented 6 years ago

Hi again,

I have uninstalled knife-solo and tried again. I get the following behaviour:

DEBUG: Bootstrapping the server...
Bootstrapping the server by using bootstrap_protocol: ssh and image_os_type: linux
....
ERROR: Connection refused connecting to https://localhost/nodes/nameless-fire, retry 1/5

I guess this is because it expects a chef-server (in my case: localhost) to serve the node-config for the new machine?

matschaffer commented 6 years ago

Would seem that way. Can't say for sure if I'll get the time to look into this, but if you were motivated to dig in, I'd get it set up like you did for the original error and hook a debugger up to see what's happening at /opt/chefdk/embedded/lib/ruby/gems/2.5.0/gems/knife-cloud-1.2.1/lib/chef/knife/cloud/command.rb:54 and maybe get some sense as to why KnifeSolo::Bootstraps is undefined.

My best guess is that knife-cloud is shelling out to knife-solo in some odd way such that it can't find it's own dependencies.

osimmasgard commented 6 years ago

Hi, an update on some progress I've made. I have added some requirements to lib/chef/knife/solo_prepare.rb (see below) and now I get further when using knife google server create. It is "uploading the kitchen" when I get this error:

uninitialized constant Chef::Cookbook::Chefignore
/home/os/.rvm/gems/ruby-2.2.2/gems/knife-cloud-1.2.1/lib/chef/knife/cloud/server/create_command.rb:91:in `rescue in after_exec_command': Check if --bootstrap-protocol and --image-os-type is correct. uninitialized constant Chef::Cookbook::Chefignore (NameError)
    from /home/os/.rvm/gems/ruby-2.2.2/gems/knife-cloud-1.2.1/lib/chef/knife/cloud/server/create_command.rb:80:in `after_exec_command'
    from /home/os/.rvm/gems/ruby-2.2.2/gems/knife-cloud-1.2.1/lib/chef/knife/cloud/command.rb:54:in `run'
    from /home/os/.rvm/gems/ruby-2.2.2/gems/chef-12.16.42/lib/chef/knife.rb:443:in `block in run_with_pretty_exceptions'
    from /home/os/.rvm/gems/ruby-2.2.2/gems/chef-12.16.42/lib/chef/local_mode.rb:44:in `with_server_connectivity'
    from /home/os/.rvm/gems/ruby-2.2.2/gems/chef-12.16.42/lib/chef/knife.rb:442:in `run_with_pretty_exceptions'
    from /home/os/.rvm/gems/ruby-2.2.2/gems/chef-12.16.42/lib/chef/knife.rb:219:in `run'
    from /home/os/.rvm/gems/ruby-2.2.2/gems/chef-12.16.42/lib/chef/application/knife.rb:156:in `run'
    from /home/os/.rvm/gems/ruby-2.2.2/gems/chef-12.16.42/bin/knife:25:in `<top (required)>'
    from /home/os/.rvm/gems/ruby-2.2.2/bin/knife:23:in `load'
    from /home/os/.rvm/gems/ruby-2.2.2/bin/knife:23:in `<main>'
    from /home/os/.rvm/gems/ruby-2.2.2/bin/ruby_executable_hooks:24:in `eval'
    from /home/os/.rvm/gems/ruby-2.2.2/bin/ruby_executable_hooks:24:in `<main>'

These are the requirements I added... 

require 'chef/knife'
require 'knife-solo'
require 'knife-solo/ssh_command'
require 'knife-solo/node_config_command'
require 'knife-solo/ssh_connection'
require 'knife-solo/bootstraps'
require 'knife-solo/berkshelf'
require 'knife-solo/librarian'
require 'knife-solo/librarian'

Any thoughts/comments?

matschaffer commented 6 years ago

Odd, that's a core chef class - https://github.com/chef/chef/blob/master/lib/chef/cookbook/chefignore.rb

The use of rvm has me wondering if possibly knife-cloud is doing something that's un-rvm-ing the environment before invoking knife-solo.

Have you tried doing this with rbenv or chefdk?

osimmasgard commented 6 years ago

Hi,

actually using knife ec2 server create with chef-dk have stopped working for me (have no idea why). It hangs on "chef_gem[aws-sdk] action install" so I am forced now to use RVM. I have no experience using rbenv.

All in all I am getting worried we can't get knife-solo to work with knife google. Are we going to be forced to install a chef-server, or is it maybe chef-zero we should try out - we have managed to avoid this for so long..

Thanks for your help so far..

matschaffer commented 6 years ago

Hrm... normally I'd expect you can remove re-install chefdk to get normal behavior back again. It's pretty isolated.

Might be worth rooting that out first since if you're having trouble with gem/ruby environments that could be contributing to the problems resolving constants that should be there.