search

express42 / postgresql_lwrp

Express 42 postgresql cookbook
MIT License
55 stars 16 forks source link

Problems after DeepMerge under PG 9.1 using ssl_cert/ssl_key attributes #23

Closed serjs closed 9 years ago

serjs commented 9 years ago

Hi!

Looks like after Deep Merge some resource/attributes are lost during provider execution, which broke recipe execution at all. In my case it's ssl_key_file and ssl_cert_file attributes.

I'm using postgresql 9.1 and chef-client v11.18.12, and main problem is nil param during symlink resource:

[2015-09-23T15:49:37+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2015-09-23T15:49:37+00:00] ERROR: postgresql[main] (paymantix::postgresql-slave line 84) had an error: TypeError: link[/var/lib/postgresql/9.1/main/server.key] (/var/chef/cache/cookbooks/postgresql_lwrp/providers/default.rb line 208) had an error: TypeError: can't convert nil into String

Stack trace:

Generated at 2015-09-23 15:49:37 +0000
TypeError: postgresql[main] (paymantix::postgresql-slave line 84) had an error: TypeError: link[/var/lib/postgresql/9.1/main/server.key] (/var/chef/cache/cookbooks/postgresql_lwrp/providers/default.rb line 208) had an error: TypeError: can't convert nil into String
/opt/chef/embedded/apps/chef/lib/chef/provider/link.rb:101:in `symlink'
/opt/chef/embedded/apps/chef/lib/chef/provider/link.rb:101:in `block in action_create'
/opt/chef/embedded/apps/chef/lib/chef/mixin/why_run.rb:52:in `call'
/opt/chef/embedded/apps/chef/lib/chef/mixin/why_run.rb:52:in `add_action'
/opt/chef/embedded/apps/chef/lib/chef/provider.rb:156:in `converge_by'
/opt/chef/embedded/apps/chef/lib/chef/provider/link.rb:100:in `action_create'
/opt/chef/embedded/apps/chef/lib/chef/provider.rb:121:in `run_action'
/opt/chef/embedded/apps/chef/lib/chef/resource.rb:648:in `run_action'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:49:in `run_action'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:81:in `block (2 levels) in converge'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:81:in `each'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:81:in `block in converge'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection.rb:98:in `block in execute_each_resource'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection.rb:96:in `execute_each_resource'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:80:in `converge'
/opt/chef/embedded/apps/chef/lib/chef/provider/lwrp_base.rb:61:in `recipe_eval_with_update_check'
/opt/chef/embedded/apps/chef/lib/chef/provider/lwrp_base.rb:45:in `block in action'
/opt/chef/embedded/apps/chef/lib/chef/provider.rb:125:in `run_action'
/opt/chef/embedded/apps/chef/lib/chef/resource.rb:648:in `run_action'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:49:in `run_action'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:81:in `block (2 levels) in converge'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:81:in `each'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:81:in `block in converge'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection.rb:98:in `block in execute_each_resource'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection.rb:96:in `execute_each_resource'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:80:in `converge'
/opt/chef/embedded/apps/chef/lib/chef/client.rb:345:in `converge'
/opt/chef/embedded/apps/chef/lib/chef/client.rb:431:in `do_run'
/opt/chef/embedded/apps/chef/lib/chef/client.rb:213:in `block in run'
/opt/chef/embedded/apps/chef/lib/chef/client.rb:207:in `fork'
/opt/chef/embedded/apps/chef/lib/chef/client.rb:207:in `run'
/opt/chef/embedded/apps/chef/lib/chef/application.rb:236:in `run_chef_client'
/opt/chef/embedded/apps/chef/lib/chef/application/client.rb:338:in `block in run_application'
/opt/chef/embedded/apps/chef/lib/chef/application/client.rb:327:in `loop'
/opt/chef/embedded/apps/chef/lib/chef/application/client.rb:327:in `run_application'
/opt/chef/embedded/apps/chef/lib/chef/application.rb:55:in `run'
/opt/chef/embedded/apps/chef/bin/chef-client:26:in `<top (required)>'
/usr/bin/chef-client:38:in `load'
/usr/bin/chef-client:38:in `<main>'

I add some logs output for debugging default provider near link, and i notice that ssl attributes lost after Deep Merge, here's code which i added before link resources:

...
        not_if { ::File.exist?("/var/lib/postgresql/#{cluster_version}/#{cluster_name}/base") }
      end
    end

    log("#{configuration['ssl_key_file']}")
    log("#{configuration['ssl_cert_file']}")
    log(node['postgresql']['defaults']['server']['configuration'])
    log(new_resource.configuration)
    log(configuration)

    link "/var/lib/postgresql/#{cluster_version}/#{cluster_name}/server.key" do
      to configuration['ssl_key_file']
      not_if { cluster_version.to_f > 9.1 && ::File.exist?("/var/lib/postgresql/#{cluster_version}/#{cluster_name}/server.key") }
    end

    link "/var/lib/postgresql/#{cluster_version}/#{cluster_name}/server.crt" do
...

Here is output of each log execution

[2015-09-23T16:20:02+00:00] INFO: Processing log[] action write (/var/chef/cache/cookbooks/postgresql_lwrp/providers/default.rb line 210)
[2015-09-23T16:20:02+00:00] INFO: 
[2015-09-23T16:20:02+00:00] INFO: Processing log[] action write (/var/chef/cache/cookbooks/postgresql_lwrp/providers/default.rb line 211)
[2015-09-23T16:20:02+00:00] INFO: 
[2015-09-23T16:20:02+00:00] INFO: {"listen_addresses"=>"localhost", "port"=>5432, "max_connections"=>100, "shared_buffers"=>"100MB", "temp_buffers"=>"8MB", "max_prepared_tr
ansactions"=>0, "work_mem"=>"2MB", "maintenance_work_mem"=>"64MB", "max_stack_depth"=>"2MB", "max_files_per_process"=>"1000", "vacuum_cost_delay"=>0, "wal_level"=>"hot_stan
dby", "fsync"=>"on", "synchronous_commit"=>"on", "checkpoint_segments"=>"64", "wal_sync_method"=>"fsync", "checkpoint_completion_target"=>"0.9", "effective_cache_size"=>509674, "log_destination"=>"stderr", "syslog_ident"=>"postgres", "log_min_duration_statement"=>200, "log_truncate_on_rotation"=>"on", "log_rotation_age"=>"1d", "log_rotation_size"=>0, "log_line_prefix"=>"%t [%p]: [%l-1]", "track_activities"=>"on", "track_counts"=>"on", "autovacuum"=>"on", "autovacuum_naptime"=>"1min", "archive_mode"=>"off", "max_wal_senders"=>5, "wal_keep_segments"=>32, "hot_standby"=>"off", "max_standby_archive_delay"=>"30s", "max_standby_streaming_delay"=>"30s", "wal_receiver_status_interval"=>"10s", "hot_standby_feedback"=>"on", "extra_float_digits"=>0, "client_encoding"=>"UTF8", "ssl"=>true, "ssl_cert_file"=>"/etc/ssl/certs/ssl-cert-snakeoil.pem", "ssl_key_file"=>"/etc/ssl/private/ssl-cert-snakeoil.key", "ssl_renegotiation_limit"=>0, "lc_messages"=>"en_US.UTF-8", "lc_monetary"=>"en_US.UTF-8", "lc_numeric"=>"en_US.UTF-8", "lc_time"=>"en_US.UTF-8", "default_text_search_config"=>"pg_catalog.russian"}
[2015-09-23T16:20:02+00:00] INFO: {:listen_addresses=>"0.0.0.0", :max_connections=>300, :ssl_renegotiation_limit=>0, :shared_buffers=>"512MB", :maintenance_work_mem=>"64MB", :work_mem=>"8MB", :effective_cache_size=>"512MB", :checkpoint_completion_target=>"0.9", :hot_standby=>"on", :hot_standby_feedback=>"on", :log_min_duration_statement=>"200", :unix_socket_directory=>"/var/run/postgresql", :lc_monetary=>"ru_RU.UTF-8", :lc_numeric=>"ru_RU.UTF-8", :lc_time=>"ru_RU.UTF-8", :ssl_cert_file=>"blabla", :ssl_key_file=>"blabla"}
[2015-09-23T16:20:02+00:00] INFO: {"listen_addresses"=>"0.0.0.0", "port"=>5432, "max_connections"=>300, "shared_buffers"=>"512MB", "temp_buffers"=>"8MB", "max_prepared_transactions"=>0, "work_mem"=>"8MB", "maintenance_work_mem"=>"64MB", "max_stack_depth"=>"2MB", "max_files_per_process"=>"1000", "vacuum_cost_delay"=>0, "wal_level"=>"hot_standby", "fsync"=>"on", "synchronous_commit"=>"on", "checkpoint_segments"=>"64", "wal_sync_method"=>"fsync", "checkpoint_completion_target"=>"0.9", "effective_cache_size"=>"512MB", "log_destination"=>"stderr", "syslog_ident"=>"postgres", "log_min_duration_statement"=>"200", "log_truncate_on_rotation"=>"on", "log_rotation_age"=>"1d", "log_rotation_size"=>0, "log_line_prefix"=>"%t [%p]: [%l-1]", "track_activities"=>"on", "track_counts"=>"on", "autovacuum"=>"on", "autovacuum_naptime"=>"1min", "archive_mode"=>"off", "max_wal_senders"=>5, "wal_keep_segments"=>32, "hot_standby"=>"on", "max_standby_archive_delay"=>"30s", "max_standby_streaming_delay"=>"30s", "wal_receiver_status_interval"=>"10s", "hot_standby_feedback"=>"on", "extra_float_digits"=>0, "client_encoding"=>"UTF8", "ssl"=>true, "ssl_renegotiation_limit"=>0, "lc_messages"=>"en_US.UTF-8", "lc_monetary"=>"ru_RU.UTF-8", "lc_numeric"=>"ru_RU.UTF-8", "lc_time"=>"ru_RU.UTF-8", "default_text_search_config"=>"pg_catalog.russian", "unix_socket_directory"=>"/var/run/postgresql"}

You can notice that attributes node['postgresql']['defaults']['server']['configuration'] have ssl_key_file and ssl_cert_file attributes values, we don't override it on resources and after merge there are no ssl_key_file and ssl_cert_file params at all.

I even try to set key file and cert resources into lwrp, in logs they are exists as attributes (as defaults), also in my lwrp resources (overridden) and after merge there are no attributes at all.

sample commented 9 years ago

Fixed in v1.1.15