search

patant / fluent-plugin-loggly

Fluentd plugin for output to loggly
Apache License 2.0
13 stars 24 forks source link

Error during plugin load #22

Closed janekolszak closed 7 years ago

janekolszak commented 7 years ago

Hi there! td-agent can't load your pluggin. 

$ gem -v
2.4.8

Here are the logs:

2016-10-13 01:29:02 +0300 [info]: starting fluentd-0.12.29
2016-10-13 01:29:02 +0300 [info]: gem 'fluent-mixin-config-placeholders' version '0.4.0'
2016-10-13 01:29:02 +0300 [info]: gem 'fluent-mixin-plaintextformatter' version '0.2.6'
2016-10-13 01:29:02 +0300 [info]: gem 'fluent-plugin-kafka' version '0.3.1'
2016-10-13 01:29:02 +0300 [info]: gem 'fluent-plugin-loggly' version '0.0.8'
2016-10-13 01:29:02 +0300 [info]: gem 'fluent-plugin-mongo' version '0.7.15'
2016-10-13 01:29:02 +0300 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.5.5'
2016-10-13 01:29:02 +0300 [info]: gem 'fluent-plugin-s3' version '0.7.1'
2016-10-13 01:29:02 +0300 [info]: gem 'fluent-plugin-scribe' version '0.10.14'
2016-10-13 01:29:02 +0300 [info]: gem 'fluent-plugin-td' version '0.10.29'
2016-10-13 01:29:02 +0300 [info]: gem 'fluent-plugin-td-monitoring' version '0.2.2'
2016-10-13 01:29:02 +0300 [info]: gem 'fluent-plugin-webhdfs' version '0.4.2'
2016-10-13 01:29:02 +0300 [info]: gem 'fluentd' version '0.12.29'
2016-10-13 01:29:02 +0300 [info]: adding match pattern="**" type="stdout"
2016-10-13 01:29:02 +0300 [info]: adding match pattern="**" type="loggly"
2016-10-13 01:29:02 +0300 [info]: adding source type="forward"
2016-10-13 01:29:02 +0300 [info]: using configuration file: <ROOT>
  <source>
    @type forward
    port 24224
    bind 0.0.0.0
  </source>
  <match **>
    @type stdout
  </match>
  <match **>
    type loggly
    loggly_url https://logs-01.loggly.com/inputs/mytoken
  </match>
</ROOT>
2016-10-13 01:29:02 +0300 [error]: unexpected error error_class=ArgumentError error=#<ArgumentError: wrong number of arguments (2 for 0)>
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/net-http-persistent-3.0.0/lib/net/http/persistent.rb:505:in `initialize'
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-loggly-0.0.8/lib/fluent/plugin/out_loggly.rb:37:in `new'
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-loggly-0.0.8/lib/fluent/plugin/out_loggly.rb:37:in `start'
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/agent.rb:71:in `block in start'
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/agent.rb:70:in `each'
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/agent.rb:70:in `start'
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/root_agent.rb:108:in `start'
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/engine.rb:237:in `start'
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/engine.rb:187:in `run'
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:570:in `run_engine'
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:162:in `block in start'
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:366:in `call'
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:366:in `main_process'
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:339:in `block in supervise'
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:338:in `fork'
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:338:in `supervise'
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:156:in `start'
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/command/fluentd.rb:173:in `<top (required)>'
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:54:in `require'
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:54:in `require'
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/bin/fluentd:5:in `<top (required)>'
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/bin/fluentd:23:in `load'
  2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/bin/fluentd:23:in `<top (required)>'
  2016-10-13 01:29:02 +0300 [error]: /usr/sbin/td-agent:7:in `load'
  2016-10-13 01:29:02 +0300 [error]: /usr/sbin/td-agent:7:in `<main>'
2016-10-13 01:29:02 +0300 [info]: shutting down fluentd
2016-10-13 01:29:02 +0300 [info]: shutting down output type="stdout" plugin_id="object:3ff5058c2bb8"
2016-10-13 01:29:02 +0300 [info]: process finished code=0
2016-10-13 01:29:02 +0300 [warn]: process died within 1 second. exit.
njam commented 7 years ago

Duplicate of https://github.com/patant/fluent-plugin-loggly/issues/21

patant commented 7 years ago

Fixed in version 0.0.9.

janekolszak commented 7 years ago

No, sorry it's not fixed. I updated to 0.0.9 and it's still here. Here's my configuration:

<source>
  @type forward
  port 24224
  bind 0.0.0.0
</source>

<match **>
  @type loggly
  loggly_url https://logs-01.loggly.com/inputs/mytoken
</match>
janekolszak commented 7 years ago 
2016-10-14 00:04:29 +0300 [info]: starting fluentd-0.12.29
2016-10-14 00:04:29 +0300 [info]: gem 'fluent-mixin-config-placeholders' version '0.4.0'
2016-10-14 00:04:29 +0300 [info]: gem 'fluent-mixin-plaintextformatter' version '0.2.6'
2016-10-14 00:04:29 +0300 [info]: gem 'fluent-plugin-kafka' version '0.3.1'
2016-10-14 00:04:29 +0300 [info]: gem 'fluent-plugin-loggly' version '0.0.9'
2016-10-14 00:04:29 +0300 [info]: gem 'fluent-plugin-mongo' version '0.7.15'
2016-10-14 00:04:29 +0300 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.5.5'
2016-10-14 00:04:29 +0300 [info]: gem 'fluent-plugin-s3' version '0.7.1'
2016-10-14 00:04:29 +0300 [info]: gem 'fluent-plugin-scribe' version '0.10.14'
2016-10-14 00:04:29 +0300 [info]: gem 'fluent-plugin-td' version '0.10.29'
2016-10-14 00:04:29 +0300 [info]: gem 'fluent-plugin-td-monitoring' version '0.2.2'
2016-10-14 00:04:29 +0300 [info]: gem 'fluent-plugin-webhdfs' version '0.4.2'
2016-10-14 00:04:29 +0300 [info]: gem 'fluentd' version '0.12.29'
patant commented 7 years ago

Did a quick test and no problems. ` 2016-10-13 22:34:40 +0000 [info]: fluent/supervisor.rb:471:read_config: reading config file path="/etc/td-agent/td-agent.conf" 2016-10-13 22:34:40 +0000 [info]: fluent/supervisor.rb:337:supervise: starting fluentd-0.12.29 2016-10-13 22:34:40 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluent-mixin-config-placeholders' version '0.4.0' 2016-10-13 22:34:40 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluent-mixin-plaintextformatter' version '0.2.6' 2016-10-13 22:34:40 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluent-plugin-kafka' version '0.3.1' 2016-10-13 22:34:40 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluent-plugin-loggly' version '0.0.9' 2016-10-13 22:34:40 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluent-plugin-mongo' version '0.7.15' 2016-10-13 22:34:40 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluent-plugin-rewrite-tag-filter' version '1.5.5' 2016-10-13 22:34:40 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluent-plugin-s3' version '0.7.1' 2016-10-13 22:34:40 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluent-plugin-scribe' version '0.10.14' 2016-10-13 22:34:40 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluent-plugin-td' version '0.10.29' 2016-10-13 22:34:40 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluent-plugin-td-monitoring' version '0.2.2' 2016-10-13 22:34:40 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluent-plugin-webhdfs' version '0.4.2' 2016-10-13 22:34:40 +0000 [info]: fluent/engine.rb:126:block in configure: gem 'fluentd' version '0.12.29' 2016-10-13 22:34:40 +0000 [info]: fluent/agent.rb:129:addmatch: adding match pattern="*" type="loggly" 2016-10-13 22:34:40 +0000 [debug]: plugin/out_loggly.rb:30:configure: Loggly url https://logs-01.loggly.com/inputs/mytoken 2016-10-13 22:34:40 +0000 [info]: fluent/root_agent.rb:147:add_source: adding source type="forward" 2016-10-13 22:34:40 +0000 [info]: fluent/engine.rb:133:configure: using configuration file: 

@type forward
port 24224
bind 0.0.0.0

<match *_> @type loggly loggly_url https://logs-01.loggly.com/inputs/mytoken 2016-10-13 22:34:40 +0000 [info]: plugin/in_forward.rb:90:listen: listening fluent socket on 0.0.0.0:24224

root@14d7e1cae4aa:/# nc -v 0.0.0.0 24224 0.0.0.0: inverse host lookup failed: Unknown host (UNKNOWN) [0.0.0.0] 24224 (?) open test 2016-10-13 22:34:49 +0000 [warn]: plugin/in_forward.rb:146:on_message: incoming chunk is broken: source="host: 127.0.0.1, addr: 127.0.0.1, port: 58140" msg=116 2016-10-13 22:34:49 +0000 [warn]: plugin/in_forward.rb:146:on_message: incoming chunk is broken: source="host: 127.0.0.1, addr: 127.0.0.1, port: 58140" msg=101 2016-10-13 22:34:49 +0000 [warn]: plugin/in_forward.rb:146:on_message: incoming chunk is broken: source="host: 127.0.0.1, addr: 127.0.0.1, port: 58140" msg=115 2016-10-13 22:34:49 +0000 [warn]: plugin/in_forward.rb:146:on_message: incoming chunk is broken: source="host: 127.0.0.1, addr: 127.0.0.1, port: 58140" msg=116 2016-10-13 22:34:49 +0000 [warn]: plugin/in_forward.rb:146:on_message: incoming chunk is broken: source="host: 127.0.0.1, addr: 127.0.0.1, port: 58140" msg=10 2016-10-13 22:34:49 +0000 [debug]: plugin/out_loggly.rb:49:block in emit: Record sent {"source":"host: 127.0.0.1, addr: 127.0.0.1, port: 58140","msg":116,"message":"incoming chunk is broken: source=\"host: 127.0.0.1, addr: 127.0.0.1, port: 58140\" msg=116"}

2016-10-13 22:34:55 +0000 [warn]: plugin/in_forward.rb:146:on_message: incoming chunk is broken: source="host: 127.0.0.1, addr: 127.0.0.1, port: 58140" msg=10 2016-10-13 22:34:55 +0000 [debug]: plugin/out_loggly.rb:54:block in emit: HTTP Response code 403 2016-10-13 22:34:55 +0000 [error]: plugin/out_loggly.rb:55:block in emit: Invalid API key 2016-10-13 22:34:55 +0000 [debug]: plugin/out_loggly.rb:49:block in emit: Record sent {"source":"host: 127.0.0.1, addr: 127.0.0.1, port: 58140","msg":101,"message":"incoming chunk is broken: source=\"host: 127.0.0.1, addr: 127.0.0.1, port: 58140\" msg=101"}`

njam commented 7 years ago

I can confirm that fluent-plugin-loggly 0.0.9 work fine with the latest fluentd 0.14.8. Thanks!

sagar-babar commented 7 years ago

When we are using loggly_buffered and type tail that we see this error