search

neillturner / kitchen-verifier-serverspec

A Test Kitchen Serverspec Verifer without having to transit the Busser layer
Other
39 stars 19 forks source link

Action fails on Amazon Linux #13

Closed fiunchinho closed 7 years ago

fiunchinho commented 7 years ago

Provisioning with kitchen-ansible and trying to execute serverspec on ec2 using Amazon Linux platform, results in the following error

kitchen verify ansible-amazon
-----> Starting Kitchen (v1.10.2)
-----> Verifying <ansible-amazon>...
       [Serverspec] Verify on instance=#<Kitchen::Instance:0x007f9bab37cac8> with state={:server_id=>"i-8aa66001", :hostname=>"ec2-52-209-171-33.eu-west-1.compute.amazonaws.com", :last_action=>"setup"}
       Environment variable KITCHEN_SERVER_ID value i-8aa66001
       Environment variable KITCHEN_HOSTNAME value ec2-52-209-171-33.eu-west-1.compute.amazonaws.com
       Environment variable KITCHEN_LAST_ACTION value setup
       Installing ruby, bundler and serverspec remotely on server
Fetching: bundler-1.12.5.gem (100%)
       Successfully installed bundler-1.12.5
       1 gem installed
       /usr/local/bin/bundle
       /usr/share/ruby/vendor_ruby/2.0/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- io/console (LoadError)
        from /usr/share/ruby/vendor_ruby/2.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/vendor/thor/lib/thor/shell/basic.rb:2:in `<top (required)>'
        from /usr/share/ruby/vendor_ruby/2.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /usr/share/ruby/vendor_ruby/2.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/vendor/thor/lib/thor/shell/color.rb:1:in `<top (required)>'
        from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/vendor/thor/lib/thor/shell.rb:17:in `shell'
        from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/ui/shell.rb:15:in `initialize'
        from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/cli.rb:13:in `new'
        from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/cli.rb:13:in `rescue in start'
        from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/cli.rb:11:in `start'
        from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/exe/bundle:27:in `block in <top (required)>'
        from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/friendly_errors.rb:98:in `with_friendly_errors'
        from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/exe/bundle:19:in `<top (required)>'
        from /usr/local/bin/bundle:23:in `load'
        from /usr/local/bin/bundle:23:in `<main>'
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>>     Failed to complete #verify action: [SSH exited (1) for command: [
            if [ ! $(which ruby) ]; then
              echo '-----> Installing ruby, will try to determine platform os'
              if [ -f /etc/centos-release ] || [ -f /etc/redhat-release ] || [ -f /etc/oracle-release ]; then
                sudo -E -H yum -y install ruby
              else
                if [ -f /etc/system-release ] || [ grep -q 'Amazon Linux' /etc/system-release ]; then
                  sudo -E -H yum -y install ruby
                else
                  sudo -E -H apt-get -y install ruby
                fi
              fi
            fi
                        if [ $(sudo -E -H gem list bundler -i) == 'false' ]; then
              sudo -E -H gem install  --no-ri --no-rdoc bundler
            fi

            if [ -d /tmp/kitchen ]; then
                            if [ $(sudo -E -H gem list serverspec -i) == 'false' ]; then
                        sudo -E -H rm -f /tmp/kitchen/Gemfile
sudo -E -H echo "source 'https://rubygems.org'
" >> /tmp/kitchen/Gemfile
sudo -E -H echo "
" >> /tmp/kitchen/Gemfile
sudo -E -H echo "gem 'net-ssh','~> 2.9'
" >> /tmp/kitchen/Gemfile
sudo -E -H echo "gem 'io-console', '~> 0.4.2'
" >> /tmp/kitchen/Gemfile
sudo -E -H echo "gem "json"
" >> /tmp/kitchen/Gemfile
sudo -E -H echo "gem "rake"
" >> /tmp/kitchen/Gemfile
sudo -E -H echo "gem "serverspec"" >> /tmp/kitchen/Gemfile

              BUNDLE_CMD=$(which bundle)
              echo $BUNDLE_CMD
              sudo -E -H  $BUNDLE_CMD install --gemfile=/tmp/kitchen/Gemfile
            fi

            else
              echo "ERROR: Default path '/tmp/kitchen' does not exist"
              exit 1
            fi
]] on ansible-amazon
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

Any ideas?

neillturner commented 7 years ago

basically bundler is trying to install the following: gem 'net-ssh','~> 2.9' gem 'io-console', '~> 0.4.2' gem "json" gem "rake" gem "serverspec

and if failing on install the io-console i suppose. you could try doing a gem install io-console on the ec2 box to see what problem is. you can create you own gemfiles to override the default. the default is gem 'net-ssh','~> 2.9 gem 'serverspec'" >> #{config[:default_path]}/Gemfile

so i guess you are creating a custom gemfile.

fiunchinho commented 7 years ago

That's my custom Gemfile

source 'https://rubygems.org'

gem 'net-ssh','~> 2.9'
gem 'io-console', '~> 0.4.2'
gem "json"
gem "rake"
gem "serverspec"

I added io-console trying to fix the problem, but it didn't help. I was getting the same error even before adding my custom Gemfile.

fiunchinho commented 7 years ago

Running bundle on the ec2 fails with the same error

$ bundle help
/usr/share/ruby/vendor_ruby/2.0/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- io/console (LoadError)
    from /usr/share/ruby/vendor_ruby/2.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/vendor/thor/lib/thor/shell/basic.rb:2:in `<top (required)>'
    from /usr/share/ruby/vendor_ruby/2.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from /usr/share/ruby/vendor_ruby/2.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/vendor/thor/lib/thor/shell/color.rb:1:in `<top (required)>'
    from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/vendor/thor/lib/thor/shell.rb:17:in `shell'
    from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/ui/shell.rb:15:in `initialize'
    from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/cli.rb:13:in `new'
    from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/cli.rb:13:in `rescue in start'
    from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/cli.rb:11:in `start'
    from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/exe/bundle:27:in `block in <top (required)>'
    from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/friendly_errors.rb:98:in `with_friendly_errors'
    from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/exe/bundle:19:in `<top (required)>'
    from /usr/local/bin/bundle:23:in `load'
    from /usr/local/bin/bundle:23:in `<main>'

I'm using Amazon Linux

neillturner commented 7 years ago

the problem is that io-console has some c code in it so need the ruby devkit. why do you want io-console?

neillturner commented 7 years ago

we would need an extra option install_ruby_devkit = true/false and have the code to install the devkit. its doable but an enhancement

fiunchinho commented 7 years ago

I don't want it. I only added it after getting the error. If I try the same but without any custom Gemfile I get the same error

Finished converging <ansible-amazon> (1m11.82s).
-----> Setting up <ansible-amazon>...
       Finished setting up <ansible-amazon> (0m0.00s).
-----> Verifying <ansible-amazon>...
       [Serverspec] Verify on instance=#<Kitchen::Instance:0x007fc7d4b3cb78> with state={:server_id=>"i-a2d11729", :hostname=>"ec2-52-209-213-239.eu-west-1.compute.amazonaws.com", :last_action=>"setup"}
       Environment variable KITCHEN_SERVER_ID value i-a2d11729
       Environment variable KITCHEN_HOSTNAME value ec2-52-209-213-239.eu-west-1.compute.amazonaws.com
       Environment variable KITCHEN_LAST_ACTION value setup
       Installing ruby, bundler and serverspec remotely on server
Fetching: bundler-1.12.5.gem (100%)
       Successfully installed bundler-1.12.5
       1 gem installed
       /usr/local/bin/bundle
       /usr/share/ruby/vendor_ruby/2.0/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- io/console (LoadError)
        from /usr/share/ruby/vendor_ruby/2.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/vendor/thor/lib/thor/shell/basic.rb:2:in `<top (required)>'
        from /usr/share/ruby/vendor_ruby/2.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /usr/share/ruby/vendor_ruby/2.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/vendor/thor/lib/thor/shell/color.rb:1:in `<top (required)>'
        from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/vendor/thor/lib/thor/shell.rb:17:in `shell'
        from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/ui/shell.rb:15:in `initialize'
        from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/cli.rb:13:in `new'
        from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/cli.rb:13:in `rescue in start'
        from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/cli.rb:11:in `start'
        from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/exe/bundle:27:in `block in <top (required)>'
        from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/lib/bundler/friendly_errors.rb:98:in `with_friendly_errors'
        from /usr/local/share/ruby/gems/2.0/gems/bundler-1.12.5/exe/bundle:19:in `<top (required)>'
        from /usr/local/bin/bundle:23:in `load'
        from /usr/local/bin/bundle:23:in `<main>'
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>>     Failed to complete #verify action: [SSH exited (1) for command: [
            if [ ! $(which ruby) ]; then
              echo '-----> Installing ruby, will try to determine platform os'
              if [ -f /etc/centos-release ] || [ -f /etc/redhat-release ] || [ -f /etc/oracle-release ]; then
                sudo -E -H yum -y install ruby
              else
                if [ -f /etc/system-release ] || [ grep -q 'Amazon Linux' /etc/system-release ]; then
                  sudo -E -H yum -y install ruby
                else
                  sudo -E -H apt-get -y install ruby
                fi
              fi
            fi
                        if [ $(sudo -E -H gem list bundler -i) == 'false' ]; then
              sudo -E -H gem install  --no-ri --no-rdoc bundler
            fi

            if [ -d /tmp/kitchen ]; then
                            if [ $(sudo -E -H gem list serverspec -i) == 'false' ]; then
                        sudo -E -H rm -f /tmp/kitchen/Gemfile
          sudo -E -H echo "source 'https://rubygems.org'" >> /tmp/kitchen/Gemfile
          sudo -E -H echo "gem 'net-ssh','~> 2.9'"  >> /tmp/kitchen/Gemfile
          sudo -E -H echo "gem 'serverspec'" >> /tmp/kitchen/Gemfile

              BUNDLE_CMD=$(which bundle)
              echo $BUNDLE_CMD
              sudo -E -H  $BUNDLE_CMD install --gemfile=/tmp/kitchen/Gemfile
            fi

            else
              echo "ERROR: Default path '/tmp/kitchen' does not exist"
              exit 1
            fi
]] on ansible-amazon
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration
fiunchinho commented 7 years ago

Meanwhile, could I use additional_install_command somehow to fix these? Any alternatives?

neillturner commented 7 years ago

yes you can. I think all you need to do is install: yum -y install ruby-devel and it will install the dependencies like gcc etc,. if this works i can add to documentation. generally the ruby installation needs work as i just have a very simple ruby install at the moment. it would be nice to support rvm and different versions of ruby

neillturner commented 7 years ago

I probably need to create an omnibus_ruby script something like: On RedHat/Fedora sudo yum install ruby rubygems ruby-devel gcc sudo gem update --system On Ubuntu you have to install several packages: sudo apt-get install ruby-full ruby1.8-dev libopenssl-ruby build-essential and get it to handle this different versions of the linux operating systems. then folks can modify it and just point to a differnt url.

fiunchinho commented 7 years ago

I confirm that it works with the following code

verifier:
  name: serverspec
  additional_install_command: 'sudo -E -H yum -y install ruby-devel; sudo gem install io-console'
neillturner commented 7 years ago

excellent that why we have these parameters to provide flexibility I'm going to create an extra install options to provide more flexibility called ruby_omnibus_install: true/false ruby_omnibus_url: https://raw.githubusercontent.com /neillturner/omnibus-ruby /master/ruby_install.sh that will point to a shell that will work for all the platforms like puppet_omnibus at https://github.com/neillturner/kitchen-puppet/blob/master/provisioner_options.md

fiunchinho commented 7 years ago

Nice!