V7.0.0 of this module struggling with keystore creation

nick-george commented 3 years ago

edited to add ES version number

I'm wondering if this is the same issue as #983.

Note, I'm using ES 7.4.2 (OSS RPM version on Oracle Linux 7.9)

Here's my call into the ES module:

  $es_secrets = {
      'keystore.seed' => '<redacted>',
  }

  class { '::elasticsearch':
    ensure            => 'present',
    version           => $version,
    autoupgrade       => false ,
    manage_repo       => false,
    require           => [Package['java',],File_line['block-es-package-obsoletion']],
    status            => 'enabled',
    restart_on_change => true,
    config            => $config,
    api_timeout       => 30, #increase the timeout for a slow starting ES instance.
    package_name      => $package_name,
    jvm_options       => [
      "-Xms${heapstr}",
      "-Xmx${heapstr}"
    ],
    secrets           => $es_secrets,
    datadir           => '/data/elasticsearch',
    #keystore_password => '<password_redacted>', #passphrase only appears to be used by xpack?
  }

Symptoms:

If I delete the keystore, the module runs without issue.
If I let the module create the keystore, then run puppet again, I get the following error

Debug: Prefetching elasticsearch_keystore resources for elasticsearch_keystore
Notice: /Stage[main]/Elasticsearch::Config/Elasticsearch_keystore[elasticsearch_secrets]/ensure: created (corrective)
Debug: Executing with uid=elasticsearch gid=elasticsearch: '/usr/share/elasticsearch/bin/elasticsearch-keystore create'
Error: /Stage[main]/Elasticsearch::Config/Elasticsearch_keystore[elasticsearch_secrets]: Could not evaluate: Execution of '/usr/share/elasticsearch/bin/elasticsearch-keystore create' returned 1: 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/execution.rb:297:in `execute'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/provider.rb:106:in `execute'
/opt/puppetlabs/puppet/cache/lib/puppet/provider/elasticsearch_keystore/ruby.rb:47:in `run_keystore'
/opt/puppetlabs/puppet/cache/lib/puppet/provider/elasticsearch_keystore/ruby.rb:96:in `flush'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/type.rb:1045:in `flush'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:25:in `evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:267:in `apply'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:287:in `eval_resource'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:191:in `call'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:191:in `block (2 levels) in evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:546:in `block in thinmark'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/benchmark.rb:308:in `realtime'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:545:in `thinmark'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:191:in `block in evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:122:in `traverse'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:178:in `evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:240:in `block (2 levels) in apply'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:546:in `block in thinmark'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/benchmark.rb:308:in `realtime'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:545:in `thinmark'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:239:in `block in apply'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/log.rb:161:in `with_destination'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/report.rb:148:in `as_logging_destination'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:238:in `apply'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:193:in `block (2 levels) in apply_catalog'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:546:in `block in thinmark'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/benchmark.rb:308:in `realtime'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:545:in `thinmark'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:192:in `block in apply_catalog'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:233:in `block in benchmark'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/benchmark.rb:308:in `realtime'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:232:in `benchmark'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:191:in `apply_catalog'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:423:in `run_internal'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:251:in `block in run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:62:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:314:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:218:in `run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:60:in `block (5 levels) in run'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/timeout.rb:93:in `block in timeout'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/timeout.rb:103:in `timeout'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:59:in `block (4 levels) in run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent/locker.rb:21:in `lock'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:53:in `block (3 levels) in run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:136:in `with_client'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:50:in `block (2 levels) in run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:101:in `run_in_fork'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:49:in `block in run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:179:in `controlled_run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:47:in `run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/agent.rb:425:in `onetime'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/agent.rb:385:in `block in run_command'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:62:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:314:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/agent.rb:381:in `run_command'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:382:in `block in run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:735:in `exit_on_fail'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:382:in `run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:143:in `run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:77:in `execute'
/opt/puppetlabs/puppet/bin/puppet:5:in `<main>'

Please let me know if I can provide any further information to assist with troubleshooting.

Regards, Nick George

mmack000 commented 3 years ago

Any updates on this? I'm having the same issue. Keystore can be created ONLY if the Keystore isn't present.

tmanninger commented 3 years ago

I have the same issue :(

Elbandi commented 3 years ago

i think, keystore provider is not compatible with new "one-instance" code

psychonaut commented 1 year ago

See https://github.com/voxpupuli/puppet-elasticsearch/issues/1153

voxpupuli / puppet-elasticsearch

V7.0.0 of this module struggling with keystore creation #1108

Symptoms: