Error executing action `install` on resource 'chef_gem[pg]'

[gayan]

Has anyone had this happen to them using tabula_rasa?

Error executing action install on resource 'chef_gem[pg]'

Bundler::GemfileNotFound

Could not locate Gemfile

Cookbook Trace:

/home/tabula-rasa/cache/cookbooks/postgresql/recipes/ruby.rb:55:in rescue in from_file' /home/tabula-rasa/cache/cookbooks/postgresql/recipes/ruby.rb:21:infrom_file' /home/tabula-rasa/cache/cookbooks/postgres-install/recipes/default.rb:8:in `from_file'

Resource Declaration:

In /home/tabula-rasa/cache/cookbooks/postgresql/recipes/ruby.rb

55: chef_gem "pg" 56: rescue Gem::Installer::ExtensionBuildError, Mixlib::ShellOut::ShellCommandFailed => e

Compiled Resource:

Declared in /home/tabula-rasa/cache/cookbooks/postgresql/recipes/ruby.rb:55:in `rescue in from_file'

chef_gem("pg") do provider Chef::Provider::Package::Rubygems action :install retries 0 retry_delay 2 package_name "pg" version "0.17.1" cookbook_name "postgresql" recipe_name "ruby" end

[shlomoswidler]

That's a new one for me. All I can say is "Ugh. Bundler."

I will need help from the community to track this down and fix it.

[shlomoswidler]

@gayan What does your tabula_rasa JSON look like? How and from where are you importing the postgres-install cookbook? Can you share your Berksfile?

[gayan]

hey thanks for the quick response. postgres-install is a cookbook we wrote ourselves to setup, and configure the DB. It includes the following community cookbooks: include_recipe 'postgis' include_recipe 'postgresql::ruby' <= I think this is the include that causes the problem.

JSON:

{ 
  "tabula_rasa": {
    "scm": {
      "type":       "s3",
      "repository": "https://s3-us-west-2.amazonaws.com/.../cookbooks-src.zip",
      "user":       "XXX",
      "password":   "XXX"
    },
    "recipes": {
      "deploy":      [ "postgres-install", "geolocationapi" ]
    }
  }
}

The berksfile:

source 'http://api.berkshelf.com'

cookbook 'golang', '~> 1.4.0'
cookbook 'mercurial' # required for some golang packages
cookbook 'postgis'
cookbook 'postgresql'
cookbook 'ssh_known_hosts',  '~> 1.3.2'
cookbook 'postgres-install', path: 'cookbooks-src/postgres-install'
cookbook 'geolocationapi', path: 'cookbooks-src/geolocationapi'
cookbook 'logrotate'

[gayan]

On a side note: I am trying to remove our dependency on the database, postgresql cookbooks altogether, to get around the OpsWorks cookbook (mysql) name collision issue. Great workaround you have here though. I'm sure a lot of people will need this.

[shlomoswidler]

I can reproduce this. I created a minimal cookbook tarball based on Berkshelfing only the postgres community cookbook and put it in S3. It's public, so anyone can try to reproduce this too. Here is my Stack JSON:

{ 
  "tabula_rasa" : {
    "scm": {
      "type": "archive",
      "repository": "https://s3.amazonaws.com/shlomo-test/tabula-rasa-issue-6-test.tar.gz"
    },
    "recipes" : { 
      "setup": [ "postgresql::ruby" ] 
    }
  }
}

My Layer has the tabula_rasa recipe in the Setup phase.

When an instance is launched, the Setup phase produces the following error trace:

[2014-12-03T10:45:15+00:00] INFO: Processing package[autoconf] action install (build-essential::_debian line 108)
[2014-12-03T10:45:23+00:00] INFO: Processing package[binutils-doc] action install (build-essential::_debian line 108)
[2014-12-03T10:45:29+00:00] INFO: Processing package[bison] action install (build-essential::_debian line 108)
[2014-12-03T10:45:37+00:00] INFO: Processing package[build-essential] action install (build-essential::_debian line 108)
[2014-12-03T10:45:37+00:00] INFO: Processing package[flex] action install (build-essential::_debian line 108)
[2014-12-03T10:45:43+00:00] INFO: Processing package[gettext] action install (build-essential::_debian line 108)
[2014-12-03T10:45:51+00:00] INFO: Processing package[ncurses-dev] action install (build-essential::_debian line 108)
[2014-12-03T10:45:51+00:00] INFO: package[ncurses-dev] is a virtual package, actually acting on package[libncurses5-dev]
[2014-12-03T10:45:56+00:00] INFO: Processing package[postgresql-client-9.1] action install (postgresql::client line 31)
[2014-12-03T10:46:06+00:00] INFO: Processing package[libpq-dev] action install (postgresql::client line 31)
[2014-12-03T10:46:24+00:00] INFO: Processing chef_gem[pg] action install (postgresql::ruby line 55)

================================================================================
Error executing action `install` on resource 'chef_gem[pg]'
================================================================================

Bundler::GemfileNotFound
------------------------
Could not locate Gemfile

Cookbook Trace:
---------------
/home/tabula-rasa/cache/cookbooks/postgresql/recipes/ruby.rb:55:in `rescue in from_file'
/home/tabula-rasa/cache/cookbooks/postgresql/recipes/ruby.rb:21:in `from_file'

Resource Declaration:
---------------------
# In /home/tabula-rasa/cache/cookbooks/postgresql/recipes/ruby.rb

55:     chef_gem "pg"
56:   rescue Gem::Installer::ExtensionBuildError, Mixlib::ShellOut::ShellCommandFailed => e

Compiled Resource:
------------------
# Declared in /home/tabula-rasa/cache/cookbooks/postgresql/recipes/ruby.rb:55:in `rescue in from_file'

chef_gem("pg") do
provider Chef::Provider::Package::Rubygems
action :install
retries 0
retry_delay 2
package_name "pg"
version "0.17.1"
cookbook_name "postgresql"
recipe_name "ruby"
end

================================================================================
Recipe Compile Error in /home/tabula-rasa/cache/cookbooks/postgresql/recipes/ruby.rb
================================================================================

Bundler::GemfileNotFound
------------------------
chef_gem[pg] (postgresql::ruby line 55) had an error: Bundler::GemfileNotFound: Could not locate Gemfile

Cookbook Trace:
---------------
/home/tabula-rasa/cache/cookbooks/postgresql/recipes/ruby.rb:55:in `rescue in from_file'
/home/tabula-rasa/cache/cookbooks/postgresql/recipes/ruby.rb:21:in `from_file'

Relevant File Content:
----------------------
/home/tabula-rasa/cache/cookbooks/postgresql/recipes/ruby.rb:

48:    end
49:  
50:    node['postgresql']['client']['packages'].each do |pg_pack|
51:      resources("package[#{pg_pack}]").run_action(:install)
52:    end
53:  
54:    begin
55>>     chef_gem "pg"
56:    rescue Gem::Installer::ExtensionBuildError, Mixlib::ShellOut::ShellCommandFailed => e
57:      # Are we an omnibus install?
58:      raise if RbConfig.ruby.scan(%r{(chef|opscode)}).empty?
59:      # Still here, must be omnibus. Lets make this thing install!
60:      Chef::Log.warn 'Failed to properly build pg gem. Forcing properly linking and retrying (omnibus fix)'
61:      gem_dir = e.message.scan(%r{will remain installed in ([^ ]+)}).flatten.first
62:      raise unless gem_dir
63:      gem_name = File.basename(gem_dir)
64:      ext_dir = File.join(gem_dir, 'ext')

[2014-12-03T10:47:30+00:00] ERROR: Running exception handlers
[2014-12-03T10:47:30+00:00] ERROR: Exception handlers complete
[2014-12-03T10:47:30+00:00] FATAL: Stacktrace dumped to /home/tabula-rasa/cache/chef-stacktrace.out
[2014-12-03T10:47:30+00:00] ERROR: chef_gem[pg] (postgresql::ruby line 55) had an error: Bundler::GemfileNotFound: Could not locate Gemfile
[2014-12-03T10:47:30+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

[gayan]

So is it a problem with a missing gem on amazon? I don't use a lot of ruby, so this could be a dumb question.

[shlomoswidler]

No, that's not the issue because directly using the postgresql::ruby recipe in the OpsWorks Layer works just fine. It's something about the tabula_rasa environment that is different.

Unfortunately, that's as far as I can take this on my own. :-(

[gayan]

Cool. I'll try looking around your repo, but I doubt I'll be any better at solving this, than you.

[tarky]

I have the same problem.

[mfdii]

What OS (platform/version) @gayan and @tarky? Are you able to install the gem via the command line?

/opt/chef/embedded/bin/gem install pg would be the command on a normal chef install, not sure about opsworks.

[shlomoswidler]

The recipe works when run as a normal OpsWorks recipe (from the Layer setup), just not when run in the tabula_rasa sandbox. Maybe that answers your question, @mfdii ? On Dec 9, 2014 6:25 AM, "mfdii" notifications@github.com wrote:

What OS (platform/version) @gayan https://github.com/gayan and @tarky https://github.com/tarky? Are you able to install the gem via the command line?

/opt/chef/embedded/bin/gem install pg

— Reply to this email directly or view it on GitHub https://github.com/shlomoswidler/tabula_rasa/issues/6#issuecomment-66289493 .

[gayan]

I was provisioning a ubuntu 14.04 instance on aws.

[gayan]

Just trying this on a VirtualBox VM, it seems that the problem might be the chef version. Chef 12.0.0 seems to result in this: Error executing action install on resource 'chef_gem[pg]'

[gayan]

This ticket on their bug tracking system might be relevant: https://tickets.opscode.com/browse/COOK-1406

[shlomoswidler]

I've been looking into this on and off for the past month, and I am sorry to say that I am no closer to understanding the cause of this.

[nickjwebb]

I've hit this bug, but with chef_gem[ruby-augeas]

[2015-05-31T21:01:32+00:00] INFO: Processing chef_gem[ruby-augeas] action install (augeas::geminstall line 10)

================================================================================
Error executing action `install` on resource 'chef_gem[ruby-augeas]'
================================================================================

Bundler::GemfileNotFound
------------------------
Could not locate Gemfile

Cookbook Trace:
---------------
/home/tabula-rasa/cache/cookbooks/augeas/recipes/geminstall.rb:10:in `from_file'
/home/tabula-rasa/cache/cookbooks/bamboo-agent/recipes/default.rb:32:in `from_file'
/home/tabula-rasa/cache/cookbooks/tile-web/recipes/default.rb:12:in `from_file'

Resource Declaration:
---------------------
# In /home/tabula-rasa/cache/cookbooks/augeas/recipes/geminstall.rb

10: chef_gem 'ruby-augeas' do
11:   action :install
12: end

Compiled Resource:
------------------
# Declared in /home/tabula-rasa/cache/cookbooks/augeas/recipes/geminstall.rb:10:in `from_file'

chef_gem("ruby-augeas") do
provider Chef::Provider::Package::Rubygems
action [:install]
retries 0
retry_delay 2
package_name "ruby-augeas"
version "0.5.0"
cookbook_name "augeas"
recipe_name "geminstall"
end

================================================================================
Recipe Compile Error in /home/tabula-rasa/cache/cookbooks/tile-web/recipes/default.rb
================================================================================

Bundler::GemfileNotFound
------------------------
chef_gem[ruby-augeas] (augeas::geminstall line 10) had an error: Bundler::GemfileNotFound: Could not locate Gemfile

I'm not super hopeful of my ability to fix this, but I will look.

[nickjwebb]

Hi @shlomoswidler ,

Could you allow access to https://s3.amazonaws.com/shlomo-test/tabula-rasa-issue-6-test.tar.gz once more, it's giving me a 403 Forbidden right now.

Nick

[shlomoswidler]

@nickjwebb OK, it is now public readable.

[nickjwebb]

Thanks @shlomoswidler we are actively trying to fix this issue... unfortunately nothing yet, but I do have a chef export working with me.

[nickjwebb]

A little bit of progress to report here. Reproducing this in a controlled environment without a lot of distracting variables has been an exercise in patience for me.

I do not have a solution, but I can refine the problem statement.

It appears that using chef_gem within tabula rasa installs the gems in a place not accessible to OpsWorks chef. If you use the same gem (including specific version) in tabula_rasa that has already been installed by OpsWorks, it works.

In my test case I needed to use the s3_file cookbook, which depends on the rest-client gem. By default it uses the latest version (1.7.x), but OpsWorks already has version 1.6.7 installed and working. If I simply override the s3_file "version" attribute, it functions properly. Here is my custom JSON (note that the repo is public):

{ 
  "tabula_rasa" : {
    "scm": {
      "type": "git",
      "repository": "git@github.com:nickjwebb/tabula-rasa-test.git"
    },
    "recipes" : { 
      "setup": [ "tabula-rasa-test::breaks" ] 
    }
  },
  "s3_file" : {
    "rest-client" : {
       "version" : "1.6.7"
    }
  }
}

Here is the very simple tabula-rasa-test::breaks recipe (subject to change):

# Grab custom postgres package & install
s3_file "/tmp/phpunit-4.6.9.phar" do
  remote_path "/phpunit-4.6.9.phar"
  bucket "packages.rwsnw.com"
  s3_url "https://s3-us-west-2.amazonaws.com/packages.rwsnw.com"
  owner "root"
  group "root"
  mode "0644"
end

Probably impractical, but one could in theory include a recipe to install all the gems needed (via chef_gem) in a recipe "outside" of tabula_rasa as a workaround.

[hashwrapi]

@gayan @tarky @shlomoswidler @mfdii @nickjwebb Guys, can you tell me how can we lock pg version? chef_gem[pg] as per above error it is trying to compile version "0.17.1" but if we check this link so in which file we have to edit to lock pg version? Link is here as above https://s3.amazonaws.com/shlomo-test/tabula-rasa-issue-6-test.tar.gz