search

lukewaite / logstash-input-cloudwatch-logs

Input plugin for Logstash to stream events from CloudWatch Logs
Other
142 stars 74 forks source link

Error on start: uninitialized constant Aws::CloudWatchLogs #16

Closed wilaszekg closed 7 years ago

wilaszekg commented 7 years ago

Hi! I get the following error when trying to start the plugin with logstash-5.2.0:

[2017-04-12T19:39:30,513][ERROR][logstash.plugins.registry] Problems loading a plugin with {:type=>"input", :name=>"cloudwatch_logs", :path=>"logstash/inputs/cloudwatch_logs", :error_message=>"uninitialized constant Aws::CloudWatchLogs", :error_class=>NameError, :error_backtrace=>["org/jruby/RubyModule.java:2719:in const_missing'", "/home/ec2-user/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/logstash-input-cloudwatch_logs-0.10.0/lib/logstash/inputs/cloudwatch/patch.rb:16:inAws'", "/home/ec2-user/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/logstash-input-cloudwatch_logs-0.10.0/lib/logstash/inputs/cloudwatch/patch.rb:15:in (root)'", "org/jruby/RubyKernel.java:1040:inrequire'", "/home/ec2-user/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/polyglot-0.3.5/lib/polyglot.rb:65:in require'", "/home/ec2-user/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/logstash-input-cloudwatch_logs-0.10.0/lib/logstash/inputs/cloudwatch_logs.rb:1:in(root)'", "org/jruby/RubyKernel.java:1040:in require'", "/home/ec2-user/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/polyglot-0.3.5/lib/polyglot.rb:65:inrequire'", "/home/ec2-user/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/logstash-input-cloudwatch_logs-0.10.0/lib/logstash/inputs/cloudwatch_logs.rb:10:in (root)'", "/home/ec2-user/logstash-5.2.0/logstash-core/lib/logstash/plugins/registry.rb:1:in(root)'", "/home/ec2-user/logstash-5.2.0/logstash-core/lib/logstash/plugins/registry.rb:151:in legacy_lookup'", "/home/ec2-user/logstash-5.2.0/logstash-core/lib/logstash/plugins/registry.rb:133:inlookup'", "/home/ec2-user/logstash-5.2.0/logstash-core/lib/logstash/plugins/registry.rb:175:in lookup_pipeline_plugin'", "/home/ec2-user/logstash-5.2.0/logstash-core/lib/logstash/plugin.rb:127:inlookup'", "org/jruby/RubyKernel.java:1079:in eval'", "/home/ec2-user/logstash-5.2.0/logstash-core/lib/logstash/pipeline.rb:452:inplugin'", "(eval):8:in initialize'", "/home/ec2-user/logstash-5.2.0/logstash-core/lib/logstash/pipeline.rb:98:ininitialize'", "/home/ec2-user/logstash-5.2.0/logstash-core/lib/logstash/agent.rb:246:in create_pipeline'", "/home/ec2-user/logstash-5.2.0/logstash-core/lib/logstash/agent.rb:95:inregister_pipeline'", "/home/ec2-user/logstash-5.2.0/logstash-core/lib/logstash/runner.rb:264:in execute'", "/home/ec2-user/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:inrun'", "/home/ec2-user/logstash-5.2.0/lib/bootstrap/environment.rb:71:in `(root)'"]}

lukewaite commented 7 years ago

@jamtur01 Could it be that the need for https://github.com/lukewaite/logstash-input-cloudwatch-logs/commit/579b05eac53ff85afca6c0af77eb20d2072a1efc is now gone?

jamtur01 commented 7 years ago

Could well be. Let me test it tonight.

jamtur01 commented 7 years ago

Looks like this is resolved now in 5.3.0.

jamtur01 commented 7 years ago

Wait no - I am wrong. This issue is still present.

jamtur01 commented 7 years ago

I've tried the plugin with the patch on 5.2.2 and 5.3.0 and without. Works fine on both releases WITH patch. Fails on both without.

lukewaite commented 7 years ago

Thanks for checking that out, @jamtur01.

@wilaszekg I'm a bit perplexed by this one. Are you able to provide any steps to reproduce?

It looks to me like the error you are hitting is essentially what the previously mentioned commit was in place to solve, though I'm just actually peeling through the trace of previous discussion around it now.

From: https://github.com/logstash-plugins/logstash-input-s3/issues/99

When a permission error is raised, the error is hidden with an internal error. The Aws-sdk rely on lazy loading of the libraries file using autoload, there is some issues concerning autoloading with ruby or jruby in general that makes theses classes to not loaded correctly when we actually need them.

Is there a chance that logstash was installed with a given user, but then the plugin was installed via sudo?

lukewaite commented 7 years ago

Hmm - on further reflection and reading, permissions in those cases probably referred to s3 access permissions, and that was explicitly trying to call the :Error class, which you're not seeming to hit.

lukewaite commented 7 years ago

@wilaszekg I've noticed that other plugins using this patch load the aws-sdk first, whereas we weren't requiring it in until during the register method.. I think that may be the problem.

I'm on a poor connection, and don't have a logstash docker image handy at the moment to test in, but I think this should probably do the trick: https://github.com/lukewaite/logstash-input-cloudwatch-logs/pull/17

If you're able, would you be able to test that branch? If not, I'll try to get it tested early next week once I have access to reliable internet again.

lukewaite commented 7 years ago

@wilaszekg This issue should be resolved now, and is available under v0.10.1 on rubygems.

I'm not super familiar with ruby loading, but I suspect this issue may not have been hit by users which had other plugins load ahead of logstash-input-cloudwatch_logs that loaded the aws-sdk.

kazizi-swe commented 5 years ago

I had


Error: Failed to open TCP connection to logs.us-west-2.amazonaws.com:443 (initialize: name or service not known)
  Exception: Seahorse::Client::NetworkingError
  Stack: org/jruby/ext/socket/RubyTCPSocket.java:138:in `initialize'
org/jruby/RubyIO.java:1155:in `open'
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:941:in `block in connect'
org/jruby/ext/timeout/Timeout.java:99:in `timeout'
org/jruby/ext/timeout/Timeout.java:75:in `timeout'
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:939:in `connect'
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:924:in `do_start'
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:919:in `start'
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/delegate.rb:83:in `method_missing'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.318/lib/seahorse/client/net_http/connection_pool.rb:285:in `start_session'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.318/lib/seahorse/client/net_http/connection_pool.rb:92:in `session_for'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.318/lib/seahorse/client/net_http/handler.rb:119:in `session'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.318/lib/seahorse/client/net_http/handler.rb:71:in `transmit'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.318/lib/seahorse/client/net_http/handler.rb:45:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.318/lib/seahorse/client/plugins/content_length.rb:12:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.318/lib/aws-sdk-core/json/error_handler.rb:8:in `call'

error regarding aws-sdk, and to my elasticsearch output plugin I added worker => 1 and it fixed the issue for me. Not sure why this would be the solution tho, if anyone has any thoughts on this, feel free to tag me.