search

flexera-public / rightscale_cookbooks

Other
76 stars 102 forks source link

Can't install private gems fr/ BitBucket (app_passenger::install_required_app_gems) #147

Closed fuJiin closed 11 years ago

fuJiin commented 12 years ago

Hey guys,

Having some issues installing a gem from a private repo. The error I get is attached below. Any help here is appreciated.

Thanks!

Notes:

• Using a custom clone of Apache-Rails-Passenger ServerTemplate

• SSH-key has been added to the machine prior to this step

• I can SSH into the machine, and run bundle install, and git clone <repo-path>, so it doesn't seem to be a SSH issue

• Not sure where the cannot run /tmp/ssh.key.sh comes from

Error:


02:21:32: ********************************************************************************
02:21:32: *RS>  Running recipe app_passenger::install_required_app_gems   ****
02:21:32: ruby_block[log marker] called
02:21:32: Processing gem_package[bundler] action install (app_passenger::install_required_app_gems line 11)
02:21:40: gem_package[bundler] installed version 1.1.5
02:21:40: Processing log[  Bundler will install gems from Gemfile] action write (app_passenger::install_required_app_gems line 20)
02:21:40:   Bundler will install gems from Gemfile
02:21:40: Processing execute[Install apache passenger module] action run (app_passenger::install_required_app_gems line 22)
02:21:40: execute[Install apache passenger module] sh(/usr/bin/bundle install --gemfile=*******/Gemfile)
02:21:41: Fetching gem metadata from https://rubygems.org/
02:21:41: .
02:21:44: .
02:21:45: .
02:21:45: .
02:21:46: .
02:21:46: .
02:21:54: Fetching gem metadata from https://rubygems.org/
02:21:54: .
02:21:54: .
02:21:55: Fetching git@bitbucket.org:********/********.git
02:21:55: error: cannot run /tmp/ssh.key.sh: No such file or directory
02:21:55: fatal: unable to fork
02:21:55: Git error: command `git clone 'git@bitbucket.org:*******/*******.git' "/usr/lib64/ruby/gems/1.8/cache/bundler/git/*******-173cf1b19a6d0d6240606f5110e3d19111ec62fb" --bare --no-hardlinks` in directory /tmp has failed.
02:21:55: Script duration: 15.248403
*ERROR> 02:21:55: execute[Install apache passenger module] (app_passenger::install_required_app_gems line 22) has had an error

*ERROR> 02:21:55: execute[Install apache passenger module] (/var/cache/rightscale/cookbooks/default/64a12fd037d3ca5b9eadb03f05abb3b841ccf13e/cookbooks/app_passenger/recipes/install_required_app_gems.rb:22:in `from_file') had an error:
execute[Install apache passenger module] (app_passenger::install_required_app_gems line 22) had an error: RightScale::Exceptions::Exec: "/usr/bin/bundle install --gemfile=*******/Gemfile" exited with 11, expected 0.
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/mixlib-shellout-1.0.0.1/lib/mixlib/shellout/legacy_command.rb:183:in `handle_command_failures'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/mixlib-shellout-1.0.0.1/lib/mixlib/shellout.rb:236:in `error!'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/chef-0.10.10.2/lib/chef/mixin/shell_out.rb:36:in `shell_out!'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/chef-0.10.10.2/lib/chef/provider/execute.rb:58:in `action_run'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/chef-0.10.10.2/lib/chef/resource.rb:514:in `send'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/chef-0.10.10.2/lib/chef/resource.rb:514:in `run_action'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/chef-0.10.10.2/lib/chef/runner.rb:49:in `run_action'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/chef-0.10.10.2/lib/chef/runner.rb:85:in `converge'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/chef-0.10.10.2/lib/chef/runner.rb:85:in `each'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/chef-0.10.10.2/lib/chef/runner.rb:85:in `converge'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/chef-0.10.10.2/lib/chef/resource_collection.rb:94:in `execute_each_resource'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/chef-0.10.10.2/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/chef-0.10.10.2/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/chef-0.10.10.2/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/chef-0.10.10.2/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/chef-0.10.10.2/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/chef-0.10.10.2/lib/chef/resource_collection.rb:92:in `execute_each_resource'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/chef-0.10.10.2/lib/chef/runner.rb:80:in `converge'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/chef-0.10.10.2/lib/chef/client.rb:330:in `converge'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/chef-0.10.10.2/lib/chef/client.rb:163:in `run'
/opt/rightscale/right_link/lib/instance/cook/executable_sequence.rb:567:in `converge'
/opt/rightscale/right_link/lib/instance/cook/executable_sequence.rb:763:in `without_bundler_env'
/opt/rightscale/right_link/lib/instance/cook/executable_sequence.rb:566:in `converge'
/opt/rightscale/right_link/lib/instance/cook/executable_sequence.rb:752:in `audit_time'
/opt/rightscale/right_link/lib/instance/cook/executable_sequence.rb:563:in `converge'
/opt/rightscale/right_link/lib/instance/cook/executable_sequence.rb:163:in `run'
/opt/rightscale/right_link/lib/instance/cook/executable_sequence.rb:515:in `call'
/opt/rightscale/right_link/lib/instance/cook/executable_sequence.rb:515:in `check_ohai'
/opt/rightscale/right_link/lib/instance/cook/executable_sequence.rb:163:in `run'
/opt/rightscale/right_link/lib/instance/cook/cook.rb:98:in `run'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/eventmachine-0.12.11.6/lib/eventmachine.rb:1020:in `call'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/eventmachine-0.12.11.6/lib/eventmachine.rb:1020:in `spawn_threadpool'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/eventmachine-0.12.11.6/lib/eventmachine.rb:1016:in `initialize'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/eventmachine-0.12.11.6/lib/eventmachine.rb:1016:in `new'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/eventmachine-0.12.11.6/lib/eventmachine.rb:1016:in `spawn_threadpool'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/eventmachine-0.12.11.6/lib/eventmachine.rb:1008:in `defer'
/opt/rightscale/right_link/lib/instance/cook/cook.rb:103:in `run'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/eventmachine-0.12.11.6/lib/eventmachine.rb:194:in `call'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/eventmachine-0.12.11.6/lib/eventmachine.rb:194:in `run_machine'
/opt/rightscale/sandbox/lib/ruby/gems/1.8/gems/eventmachine-0.12.11.6/lib/eventmachine.rb:194:in `run'
/opt/rightscale/right_link/lib/instance/cook/cook.rb:95:in `run'
/opt/rightscale/right_link/bin/cook_runner.rb:22

*ERROR> Execution failed
*ERROR> External command error: "/usr/bin/bundle install --gemfile=*******/Gemfile" exited with 11, expected 0.
The command was run from "/tmp"
*ERROR> Subprocess exited with 1
arangamani commented 12 years ago

Is the ssh key added to the root user or regular user?

mrusan commented 12 years ago

About the cannot run /tmp/ssh.key.sh: /tmp/ssh.key.sh is created when the repo_git provider is called with a :pull action: it calls the create definition in the repogit helper and that creates the .sh file and sets "GITSSH" environmental variable to it's path. Check that the file actually gets created.

fuJiin commented 12 years ago

@arangamani: The key is added to ~root/.ssh/ in a custom RightScript. There's also a line in the script that adds IdentifyFile ~root/.ssh/<key-name> to ~root/.ssh/config. This works if the git clone command is run independently.

@mrusan: I see the create definition being called, but it also looks like the key is removed after repo_git:pull (see code). The error occurs in _app_passenger:install_required_appgems, by which time it looks like the key file has been deleted.

I think we have 2 options:

  1. Wrap app_passenger:install_required_app_gems with the create and destroy definitions in repo_git helper.
  2. Stop app_passenger:install_required_app_gems from looking for /tmp/ssh.key.sh. Not sure where it's calling for this in the first place, and bundler should be able to use SSH settings from ~root/.ssh/config

Any thoughts?

mrusan commented 12 years ago

Not sure what your custom RightScript is used for but this shouldn't be the case: ensure that the ssh key is provided via input or the repo_git provider is been called from another location with the right key set up or make sure node[:repo][:default][:credential] is set to the ssh key. You can also comment out the code you linked to (the one that removes the ssh.key.sh) and see that the file actually gets created, has the right permissions and contains the key.

caryp commented 12 years ago

@fuJiin: It looks like this problem is due to an ENV variable left around from the app::do_update_code recipe, which calls the repo LWRP. This line is the culprit:

https://github.com/rightscale/rightscale_cookbooks/blob/master/cookbooks/repo_git/libraries/helper.rb#L31

...which sets up the GIT_SSH environment variable that prompts git to run /tmp/ssh.key.sh. This is a bug as the git_repo provider should clear this value when it is done with the "pull" action.

You should be able to work around this issue by clearing GIT_SSH before running your command. For example:

sh(export GIT_SSH=; /usr/bin/bundle install --gemfile=***/Gemfile)

hope that helps! Thanks for reporting the issue!!

fuJiin commented 12 years ago

@caryp - that works thanks!

arangamani commented 11 years ago

Since the solution given by @caryp worked, closing this issue now.

caryp commented 11 years ago

fixed in https://github.com/rightscale/rightscale_cookbooks/commit/f7243bfeffd0c4937c79b85aba7ea090ad7cb979