elasticsearch_keystore resource fails due to legacy instance code

johnkazmerzak commented 2 years ago

Affected Puppet, Ruby, OS and module versions/distributions

Puppet: 6.26.0
Ruby: 2.5.9p229
Distribution: CentOS 7
Module version: 8.0.1

How to reproduce (e.g Puppet code you use)

elasticsearch_keystore { $hostname:
  settings => {
    'bootstrap.password' => $bootstrap_password,
    'xpack.security.transport.ssl.keystore.secure_password' => $keystore_truststore_secure_password,
    'xpack.security.transport.ssl.truststore.secure_password' => $keystore_truststore_secure_password,
  },
}

What are you seeing

Could not evaluate: Execution of '/usr/share/elasticsearch/bin/elasticsearch-keystore create' returned 1:

What behaviour did you expect instead

Successful execution of elasticsearch_keystore resource

Output log

Notice: /Stage[main]/Sstsrdnalin_profiles::Elastic/Elasticsearch_keystore[redacted]/ensure: created
Debug: Executing with uid=elasticsearch gid=elasticsearch: '/usr/share/elasticsearch/bin/elasticsearch-keystore create'
Error: /Stage[main]/Sstsrdnalin_profiles::Elastic/Elasticsearch_keystore[dnatestlog03]: Could not evaluate: Execution of '/usr/share/elasticsearch/bin/elasticsearch-keystore create' returned 1: 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/execution.rb:297:in `execute'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/provider.rb:106:in `execute'
/opt/puppetlabs/puppet/cache/lib/puppet/provider/elasticsearch_keystore/ruby.rb:49:in `run_keystore'
/opt/puppetlabs/puppet/cache/lib/puppet/provider/elasticsearch_keystore/ruby.rb:98:in `flush'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/type.rb:1045:in `flush'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:25:in `evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:267:in `apply'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:287:in `eval_resource'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:191:in `call'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:191:in `block (2 levels) in evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:546:in `block in thinmark'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/benchmark.rb:308:in `realtime'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:545:in `thinmark'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:191:in `block in evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:122:in `traverse'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:178:in `evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:240:in `block (2 levels) in apply'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:546:in `block in thinmark'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/benchmark.rb:308:in `realtime'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:545:in `thinmark'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:239:in `block in apply'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/log.rb:165:in `with_destination'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/report.rb:152:in `as_logging_destination'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:238:in `apply'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:196:in `block (2 levels) in apply_catalog'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:546:in `block in thinmark'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/benchmark.rb:308:in `realtime'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:545:in `thinmark'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:195:in `block in apply_catalog'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:233:in `block in benchmark'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/benchmark.rb:308:in `realtime'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:232:in `benchmark'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:194:in `apply_catalog'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:412:in `run_internal'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:251:in `block in run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:62:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:310:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:221:in `run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:60:in `block (5 levels) in run'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/timeout.rb:93:in `block in timeout'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/timeout.rb:103:in `timeout'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:59:in `block (4 levels) in run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent/locker.rb:21:in `lock'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:53:in `block (3 levels) in run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:136:in `with_client'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:50:in `block (2 levels) in run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:101:in `run_in_fork'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:49:in `block in run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:179:in `controlled_run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:47:in `run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/agent.rb:437:in `onetime'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/agent.rb:397:in `block in run_command'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:62:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:310:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/agent.rb:393:in `run_command'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:382:in `block in run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:735:in `exit_on_fail'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:382:in `run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:143:in `run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:77:in `execute'
/opt/puppetlabs/puppet/bin/puppet:5:in `<main>'

Any additional information you'd like to impart

The resource fails because of legacy instance configuration in this definition, https://github.com/voxpupuli/puppet-elasticsearch/blob/7ef798331104790bda4fc900586cb29af4d86bef/lib/puppet/provider/elasticsearch_keystore/ruby.rb#L30

psychonaut commented 1 year ago

this actually made secrets management unusable with new module

psychonaut commented 1 year ago

I've created a hackish, drop-in replacement for elasticsearch_keystore provider: https://github.com/psychonaut/puppet-elasticsearch/blob/keystore/lib/puppet/provider/elasticsearch_keystore/ruby.rb - it works but to be honest, I don't like this approach: since in new module there's only one keystore there's no point of creating resource for keystore. Moreover, even in the old instance version, it didn't check the value of the secret, it only checked if a secret with this name exists.

The new approach should be built around secrets, not keystores. So instead of

elasticsearch_keystore { 'whatever': # it's no longer relevant what you put here since there's only one keystore
  ..
  settings => {secret1_key => secret1_value, secret2_key => secret2_value},
}

it should be

elasticsearch_secret { 'secret_key':
  ensure => absent|present
  value  => secret_value,
}

victorvarias commented 1 year ago

Here is another "dirty" solution similar to the one provided by @psychonaut but also solves #983 https://github.com/victorvarias/puppet-elasticsearch/commit/1b02c895459b4706468e259166ed2c8b3c6ea66f

voxpupuli / puppet-elasticsearch