search

chef-cookbooks / supermarket-omnibus-cookbook

A cookbook for installing supermarket from Omnibus (deb, rpm) packages
Apache License 2.0
17 stars 29 forks source link

Can't install onto RDS #46

Open boydj opened 8 years ago

boydj commented 8 years ago

Error executing action run on resource 'execute[create postgresql plpgsql extension

With the following config in my wrapper cookbook:

default['supermarket_omnibus']['chef_server_url'] = 'xxx'
default['supermarket_omnibus']['chef_oauth2_app_id'] = 'xxx'
default['supermarket_omnibus']['chef_oauth2_secret'] = 'xxx'
default['supermarket_omnibus']['chef_oauth2_verify_ssl'] = false
default['supermarket_omnibus']['package_url'] = 'xxx'
default['supermarket_omnibus']['config']['features'] = 'tools, gravatar, collaborator_groups, github, announcement'

default['supermarket_omnibus']['config']['postgresql']['enable'] = false
default['supermarket_omnibus']['config']['database']['user']     = 'chef'
default['supermarket_omnibus']['config']['database']['name']     = 'supermarket'
default['supermarket_omnibus']['config']['database']['host']     = 'xxx.us-east-1.rds.amazonaws.com'
default['supermarket_omnibus']['config']['database']['port']     = '5432'
default['supermarket_omnibus']['config']['database']['password'] = ''

default['supermarket_omnibus']['config']['s3_bucket'] = ''

Looks like this was fixed previously here: https://github.com/chef-boneyard/omnibus-supermarket/pull/20/commits/108f144d6262d57d79903ba518d0b4f4b40f6d38v

[0mMixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '1'
---- Begin output of echo 'CREATE EXTENSION IF NOT EXISTS plpgsql' | psql ----
STDOUT: 
STDERR: /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout.rb:206:in `getpwnam': can't find user for chef (ArgumentError)
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout.rb:206:in `uid'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout/unix.rb:153:in `set_user'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout/unix.rb:332:in `block in fork_subprocess'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout/unix.rb:316:in `fork'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout/unix.rb:316:in `fork_subprocess'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout/unix.rb:93:in `run_command'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/mixlib-shellout-2.2.7/lib/mixlib/shellout.rb:259:in `run_command'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/mixin/shell_out.rb:86:in `shell_out_command'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/mixin/shell_out.rb:39:in `shell_out'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/mixin/shell_out.rb:44:in `shell_out!'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/provider/execute.rb:62:in `block in action_run'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/mixin/why_run.rb:52:in `add_action'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/provider.rb:176:in `converge_by'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/provider/execute.rb:60:in `action_run'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/provider.rb:145:in `run_action'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/resource.rb:603:in `run_action'
    from /var/opt/supermarket/cache/cache/cookbooks/compat_resource/files/lib/chef_compat/monkeypatches/chef/runner.rb:78:in `run_action'
    from /var/opt/supermarket/cache/cache/cookbooks/compat_resource/files/lib/chef_compat/monkeypatches/chef/runner.rb:106:in `block (2 levels) in converge'
    from /var/opt/supermarket/cache/cache/cookbooks/compat_resource/files/lib/chef_compat/monkeypatches/chef/runner.rb:106:in `each'
    from /var/opt/supermarket/cache/cache/cookbooks/compat_resource/files/lib/chef_compat/monkeypatches/chef/runner.rb:106:in `block in converge'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/resource_collection/resource_list.rb:94:in `block in execute_each_resource'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/resource_collection/resource_list.rb:92:in `execute_each_resource'
    from /opt/supermarket/embedded/lib/ruby/2.3.0/forwardable.rb:184:in `execute_each_resource'
    from /var/opt/supermarket/cache/cache/cookbooks/compat_resource/files/lib/chef_compat/monkeypatches/chef/runner.rb:105:in `converge'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/client.rb:669:in `block in converge'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/client.rb:664:in `catch'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/client.rb:664:in `converge'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/client.rb:703:in `converge_and_save'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/client.rb:283:in `run'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/application.rb:302:in `block in fork_chef_client'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/application.rb:290:in `fork'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/application.rb:290:in `fork_chef_client'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/application.rb:255:in `block in run_chef_client'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/local_mode.rb:44:in `with_server_connectivity'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/application.rb:243:in `run_chef_client'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/application/client.rb:464:in `sleep_then_run_chef_client'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/application/client.rb:451:in `block in interval_run_chef_client'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/application/client.rb:450:in `loop'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/application/client.rb:450:in `interval_run_chef_client'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/application/client.rb:434:in `run_application'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/lib/chef/application.rb:60:in `run'
    from /opt/supermarket/embedded/lib/ruby/gems/2.3.0/gems/chef-12.13.37/bin/chef-client:26:in `<top (required)>'
    from /opt/supermarket/embedded/bin/chef-client:23:in `load'
    from /opt/supermarket/embedded/bin/chef-client:23:in `<main>'
---- End output of echo 'CREATE EXTENSION IF NOT EXISTS plpgsql' | psql ----
Ran echo 'CREATE EXTENSION IF NOT EXISTS plpgsql' | psql returned 1

Cookbook Trace:
---------------
/var/opt/supermarket/cache/cache/cookbooks/compat_resource/files/lib/chef_compat/monkeypatches/chef/runner.rb:78:in `run_action'
/var/opt/supermarket/cache/cache/cookbooks/compat_resource/files/lib/chef_compat/monkeypatches/chef/runner.rb:106:in `block (2 levels) in converge'
/var/opt/supermarket/cache/cache/cookbooks/compat_resource/files/lib/chef_compat/monkeypatches/chef/runner.rb:106:in `each'
/var/opt/supermarket/cache/cache/cookbooks/compat_resource/files/lib/chef_compat/monkeypatches/chef/runner.rb:106:in `block in converge'
/var/opt/supermarket/cache/cache/cookbooks/compat_resource/files/lib/chef_compat/monkeypatches/chef/runner.rb:105:in `converge'

Resource Declaration:
---------------------
# In /var/opt/supermarket/cache/cache/cookbooks/omnibus-supermarket/recipes/database.rb

 46:   execute "create postgresql #{ext} extension" do
 47:     user node['supermarket']['database']['user']
 48:     command "echo 'CREATE EXTENSION IF NOT EXISTS #{ext}' | psql"
 49:     not_if "echo '\\dx' | psql #{node['supermarket']['database']['name']} | grep #{ext}"
 50:   end
 51: end

Compiled Resource:
------------------
# Declared in /var/opt/supermarket/cache/cache/cookbooks/omnibus-supermarket/recipes/database.rb:46:in `block in from_file'

execute("create postgresql plpgsql extension") do
  action [:run]
  retries 0
  retry_delay 2
  default_guard_interpreter :execute
  command "echo 'CREATE EXTENSION IF NOT EXISTS plpgsql' | psql"
  backup 5
  returns 0
  user "chef"
  declared_type :execute
  cookbook_name "omnibus-supermarket"
  recipe_name "database"
  not_if "echo '\dx' | psql supermarket | grep plpgsql"
end

Platform:
---------
x86_64-linux
irvingpop commented 8 years ago

Thanks for reporting that @boydj ! Would you mind submitting a PR to re-add those lines?

robbkidd commented 8 years ago

I think if these lines were to go somewhere, they would be in the omnibus-supermarket recipe that is internal to the omnibus package. I propose that this issue move to the chef/supermarket repo under the guise of "handle PG database ownership and enabling PG extensions when PG is run external to the omnibus install." The solution might be something other than flipping these resources off if omnibus'd-postgresql is disabled. For example, I can picture adding to the database recipe a .pgpass file for psql commands to pick up and use to authenticate to wherever PG is running during these configuration steps.

robbkidd commented 8 years ago

Looking a bit closer at the error, the specific error has to do with the expectation that the execute resource for the psql command will be run with an OS user that matches the name of a user within the PostgreSQL service who has the privileges to install extensions. This is true for the default install—supermarket OS service account and supermarket PG database user—but not necessarily true for a PG database configured elsewhere. In @boydj's case, there is a chef user in the RDS database, but that an OS user of the same name does not exist on the Supermarket host (and also maybe shouldn't).

I sense that the bug here is assuming that an OS user for the command to install extensions will match an admin DB user. We should spend a little time unpacking that.

boydj commented 8 years ago

I can confirm that changing the DB user to match a system user, in this case, supermarket, allows installation to complete successfully.

robbkidd commented 8 years ago

Thanks for the confirmation, @boydj. That helps shape our options. I see them as:

Are those reasonable? Are there others?