undefined method `credentials' for nil:NilClass

[wesleymilan]

I'm using AMI from AWS version 2 connecting to an instance fully opened, no credentials, no IP restriction. I've tried without aws key and with blank aws keys, but it returns the same error

# /usr/share/logstash/bin/logstash --path.settings /etc/logstash OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/usr/share/logstash/logstash-core/lib/jars/jruby-complete-9.2.7.0.jar) to field java.io.FileDescriptor.fd WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties [2019-06-26T21:40:29,375][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.8.1"} /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client/pool.rb:33: warning: already initialized constant ROOT_URI_PATH /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client/pool.rb:36: warning: already initialized constant DEFAULT_OPTIONS /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client/pool.rb:160: warning: already initialized constant ES1_SNIFF_RE_URL /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client/manticore_adapter.rb:7: warning: already initialized constant DEFAULT_HEADERS /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client.rb:24: warning: already initialized constant TARGET_BULK_BYTES /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/common.rb:8: warning: already initialized constant DOC_DLQ_CODES /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/common.rb:9: warning: already initialized constant DOC_SUCCESS_CODES /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/common.rb:10: warning: already initialized constant DOC_CONFLICT_CODE /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/common.rb:16: warning: already initialized constant VERSION_TYPES_PERMITTING_CONFLICT /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/common.rb:133: warning: already initialized constant VALID_HTTP_ACTIONS /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/common.rb:247: warning: already initialized constant DEFAULT_EVENT_TYPE_ES6 /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/common.rb:248: warning: already initialized constant DEFAULT_EVENT_TYPE_ES7 [2019-06-26T21:40:35,647][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"kong", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50} url template {:scheme=>nil, :user=>nil, :password=>nil, :host=>"URLTEMPLATE", :port=>443, :path=>nil} [2019-06-26T21:40:37,207][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://search-hsg-logs-opened-vwjpqfqxmr7xyanw6xl2fxmjxu.us-east-1.es.amazonaws.com:443/]}} [2019-06-26T21:40:37,228][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>https://search-hsg-logs-opened-vwjpqfqxmr7xyanw6xl2fxmjxu.us-east-1.es.amazonaws.com:443/, :path=>"/"} [2019-06-26T21:40:37,482][ERROR][logstash.pipeline ] Error registering plugin {:pipeline_id=>"kong", :plugin=>"#<LogStash::OutputDelegator:0x163bd074>", :error=>"undefined methodcredentials' for nil:NilClass", :thread=>"#"} [2019-06-26T21:40:37,514][ERROR][logstash.pipeline ] Pipeline aborted due to error {:pipeline_id=>"kong", :exception=>#<NoMethodError: undefined method credentials' for nil:NilClass>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.291/lib/aws-sdk-core/signers/v4.rb:45:ininitialize'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client/manticore_adapter.rb:108:in perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client/pool.rb:291:inperform_request_to_url'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client/pool.rb:245:in block in healthcheck!'", "org/jruby/RubyHash.java:1419:ineach'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client/pool.rb:241:in healthcheck!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client/pool.rb:341:inupdate_urls'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client/pool.rb:71:in start'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client.rb:316:inbuild_pool'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client.rb:66:in initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client_builder.rb:108:increate_http_client'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/http_client_builder.rb:98:in build'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es.rb:253:inbuild_client'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-6.4.2-java/lib/logstash/outputs/amazon_es/common.rb:25:in register'", "org/logstash/config/ir/compiler/OutputStrategyExt.java:106:inregister'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:48:in register'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:259:inregister_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:270:in block in register_plugins'", "org/jruby/RubyArray.java:1792:ineach'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:270:in register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:611:inmaybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:280:in start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:217:inrun'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:176:in block in start'"], :thread=>"#<Thread:0x1e2a1108 run>"} [2019-06-26T21:40:37,545][ERROR][logstash.agent ] Failed to execute action {:id=>:kong, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<kong>, action_result: false", :backtrace=>nil} [2019-06-26T21:40:38,064][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600} [2019-06-26T21:40:42,822][INFO ][logstash.runner ] Logstash shut down.

[climbbike]

From what I can tell you either have to run on an ec2 instance in which case it uses the iam role or you have to pass aws_access_key_id and aws_secret_access_key

[AustinTag]

Can you provide the logstash conf files that you have tries, as well as your versions for Logstash, ES, and this plugin?

[abhisheksoni27]

I am getting the same error when I put the logstash conf file inside /etc/logstash/conf.d and start logstash as a service. It works fine, if I directly invoke logstash and pass my conf file as an argument.

TL;DR;

It works if i do this: logstash -f posts.conf

It doesn't work if I do this: sudo service logstash start where my conf file is at /etc/logstash/conf.d/posts.conf

Here's the trace:

Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<NoMethodError: undefined method `credentials' for nil:NilClass>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.372/lib/aws-sdk-core/signers/v4.rb:45:in `initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-7.0-java/lib/logstash/outputs/amazon_es/http_client/manticore_adapter.rb:108:in `perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-7.0-java/lib/logstash/outputs/amazon_es/http_client/pool.rb:291:in `perform_request_to_url'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-7.0-java/lib/logstash/outputs/amazon_es/http_client/pool.rb:245:in `block in healthcheck!'", "org/jruby/RubyHash.java:1417:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-7.0-java/lib/logstash/outputs/amazon_es/http_client/pool.rb:241:in `healthcheck!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-7.0-java/lib/logstash/outputs/amazon_es/http_client/pool.rb:341:in `update_urls'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-7.0-java/lib/logstash/outputs/amazon_es/http_client/pool.rb:71:in `start'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-7.0-java/lib/logstash/outputs/amazon_es/http_client.rb:316:in `build_pool'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-7.0-java/lib/logstash/outputs/amazon_es/http_client.rb:66:in `initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-7.0-java/lib/logstash/outputs/amazon_es/http_client_builder.rb:108:in `create_http_client'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-7.0-java/lib/logstash/outputs/amazon_es/http_client_builder.rb:98:in `build'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-7.0-java/lib/logstash/outputs/amazon_es.rb:253:in `build_client'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-amazon_es-7.0-java/lib/logstash/outputs/amazon_es/common.rb:25:in `register'", "org/logstash/config/ir/compiler/OutputStrategyExt.java:106:in `register'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:48:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:195:in `block in register_plugins'", "org/jruby/RubyArray.java:1800:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:194:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:467:in `maybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:207:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:149:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:108:in `block in start'"], :thread=>"#<Thread:0x7aa2e24d run>"}
[2019-10-27T07:13:11,058][ERROR][logstash.agent           ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}

[vaishkri23]

I had the same error (Logstash version: 7.6.2, Elasticsearch version: 7.4) and as @climbbike mentioned, creating an user with the roles as described in this article https://aws.amazon.com/elasticsearch-service/resources/articles/logstash-tutorial/ worked in getting logstash to talk to Elasticsearch.