logstash influxdb plugin error

[rguptarg]

Hi Team,

We are trying to connect logstash with influxdb , but getting below error:-

[2019-12-06T12:47:15,695][ERROR][logstash.plugins.registry] Tried to load a plugin's code, but failed. {:exception=>#<LoadError: no such file to load -- logstash/outputs/influxdb>, :path=>"logstash/outputs/influxdb", :type=>"output", :name=>"influxdb"} [2019-12-06T12:47:15,713][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::PluginLoadingError", :message=>"Couldn't find any output plugin named 'influxdb'. Are you sure this is correct? Trying to load the influxdb output plugin resulted in this error: no such file to load -- logstash/outputs/influxdb", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/plugins/registry.rb:221:in lookup_pipeline_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/plugin.rb:143:in lookup'", "org/logstash/plugins/PluginFactoryExt.java:203:in plugin'", "org/logstash/plugins/PluginFactoryExt.java:140:in buildOutput'", "org/logstash/execution/JavaBasePipelineExt.java:50:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:24:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:36:in execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:325:in block in converge_state'"]} [2019-12-06T12:47:16,297][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600} [2019-12-06T12:47:21,259][INFO ][logstash.runner ] Logstash shut down. [2019-12-06T12:47:45,177][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.3.2"}

OS :- Red Hat Enterprise Linux Server release 7.7 (Maipo)

[root@N1VL-PA-INFL01 ~]# /usr/share/logstash/bin/logstash-plugin list|grep -i influ logstash-output-influxdb

input section input { beats { port => 5044 } }

Filter section filter { if [type] == "syslog" { grok { match => { "message" => "%{SYSLOGLINE}" } } date { match => [ "timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]

} } }

output section output { influxdb { data_points => {} host => "10.5.112.175:8086" } }

[andsel]

@rguptarg seems like LogStash is not able to findInfluxDB output plugin's gem. Two questions:

• did you installed it?
• do you see it listing the installed plugins? (https://www.elastic.co/guide/en/logstash/current/working-with-plugins.html)

[rguptarg]

[root@N1VL-PA-INFL01 logstash]# /usr/share/logstash/bin/logstash-plugin list|grep influx logstash-output-influxdb

Yes, I have installed using below command :- /usr/share/logstash/bin/logstash-plugin install logstash-output-influxdb

[rguptarg]

After re-installation logstash, it's working fine. But still Unable to add data in InfluxDB, getting below error:-

[2019-12-10T13:28:15,550][WARN ][logstash.outputs.influxdb] Non recoverable exception while writing to InfluxDB {:exception=>#<InfluxDB::Error: {"error":"unable to parse 'logstash,beats_input_codec_plain_applied=true,host={\"os\"\=\u003e{\"name\"\=\u003e\"Red\ Hat\ Enterprise\ Linux\ Server\"\,\ \"version\"\=\u003e\"7.7\ (Maipo)\"\,\ \"kernel\"\=\u003e\"3.10.0-1062.el7.x86_64\"\,\ \"codename\"\=\u003e\"Maipo\"\,\ \"platform\"\=\u003e\"rhel\"\,\ \"family\"\=\u003e\"redhat\"}\,\ \"name\"\=\u003e\"N1VL-PA-INFL01\"\,\ \"containerized\"\=\u003efalse\,\ \"hostname\"\=\u003e\"N1VL-PA-INFL01\"\,\ \"id\"\=\u003e\"f462e1d352fd42739bea16c0ed67b21e\"\,\ \"architecture\"\=\u003e\"x86_64\"} event={\"dataset\"=\u003e\"system.syslog\", \"module\"=\u003e\"system\", \"timezone\"=\u003e\"+05:30\"},service={\"type\"=\u003e\"system\"},log={\"offset\"=\u003e26591603, \"file\"=\u003e{\"path\"=\u003e\"/var/log/messages\"}},fileset={\"name\"=\u003e\"syslog\"},agent={\"version\"=\u003e\"7.3.2\", \"type\"=\u003e\"filebeat\", \"hostname\"=\u003e\"N1VL-PA-INFL01\", \"id\"=\u003e\"76066f6a-49c7-47f6-9a47-f47ff93f8cbf\", \"ephemeral_id\"=\u003e\"2203accc-0d9a-4c6b-b2e8-49684f4da12a\"},input={\"type\"=\u003e\"log\"},ecs={\"version\"=\u003e\"1.0.1\"} 1575964550267': invalid boolean"}

I am using filebeat to collect and forward to logstash, and logsstash config :-

input section

input { beats { port => 5044 } }

Filter section

filter { if [type] == "syslog" { grok { match => { "message" => "%{SYSLOGLINE}" } } date { match => [ "timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]

} } }

output section

output { stdout {codec => rubydebug} influxdb { host => "10.5.112.175" port => 8086 db => "logstash" codec => "json" use_event_fields_for_data_points => true exclude_fields => ["@timestamp", "@version", "sequence", "message", "type"] data_points => { } } }

Please help to correct OUTPUT configuration

[andsel]

This time the error seems different, InfluxDB client claims that he found a "parse error", I think sent back from InfluxDB. One line of log is too poor context, and probably you should also check the logs in InfluxDB to verify it receives malformed data