search

repeatedly / fluent-plugin-multi-format-parser

Multi format parser for Fluentd
108 stars 16 forks source link

Not working under Ubuntu xenial, with td-agent 3.2v #18

Closed eugenegwon closed 3 years ago

eugenegwon commented 3 years ago

Hello, It seems like this plugin is not working on Unbuntu Xenial.

I installed this version :

http://packages.treasuredata.com.s3.amazonaws.com/3/ubuntu/xenial/pool/contrib/t/td-agent/td-agent_3.2.0-0_amd64.deb

td-agent.log

2021-02-18 18:38:07 +0900 [info]: parsing config file is succeeded path="/etc/td-agent/td-agent.conf" 2021-02-18 18:38:07 +0900 [info]: brokers has been set: ["10.232.0.134:9092", "10.232.0.94:9092", "10.232.0.78:9092"] 2021-02-18 18:38:07 +0900 [info]: brokers has been set: ["10.232.0.134:9092", "10.232.0.94:9092", "10.232.0.78:9092"] 2021-02-18 18:38:07 +0900 [error]: config error file="/etc/td-agent/td-agent.conf" error_class=Fluent::ConfigError error="Unknown parser plugin 'multi_format'. Run 'gem search -rd fluent-plugin' to find plugins" 2021-02-18 18:38:07 +0900 [info]: parsing config file is succeeded path="/etc/td-agent/td-agent.conf" 2021-02-18 18:38:07 +0900 [info]: brokers has been set: ["10.232.0.134:9092", "10.232.0.94:9092", "10.232.0.78:9092"] 2021-02-18 18:38:07 +0900 [info]: brokers has been set: ["10.232.0.134:9092", "10.232.0.94:9092", "10.232.0.78:9092"] 2021-02-18 18:38:07 +0900 [error]: config error file="/etc/td-agent/td-agent.conf" error_class=Fluent::ConfigError error="Unknown parser plugin 'multi_format'. Run 'gem search -rd fluent-plugin' to find plugins" 2021-02-18 18:38:08 +0900 [info]: parsing config file is succeeded path="/etc/td-agent/td-agent.conf" 2021-02-18 18:38:08 +0900 [info]: brokers has been set: ["10.232.0.134:9092", "10.232.0.94:9092", "10.232.0.78:9092"] 2021-02-18 18:38:08 +0900 [info]: brokers has been set: ["10.232.0.134:9092", "10.232.0.94:9092", "10.232.0.78:9092"] 2021-02-18 18:38:08 +0900 [error]: config error file="/etc/td-agent/td-agent.conf" error_class=Fluent::ConfigError error="Unknown parser plugin 'multi_format'. Run 'gem search -rd fluent-plugin' to find plugins" 2021-02-18 18:38:08 +0900 [info]: parsing config file is succeeded path="/etc/td-agent/td-agent.conf" 2021-02-18 18:38:08 +0900 [info]: brokers has been set: ["10.232.0.134:9092", "10.232.0.94:9092", "10.232.0.78:9092"] 2021-02-18 18:38:08 +0900 [info]: brokers has been set: ["10.232.0.134:9092", "10.232.0.94:9092", "10.232.0.78:9092"] 2021-02-18 18:38:08 +0900 [error]: config error file="/etc/td-agent/td-agent.conf" error_class=Fluent::ConfigError error="Unknown parser plugin 'multi_format'. Run 'gem search -rd fluent-plugin' to find plugins" 2021-02-18 18:38:09 +0900 [info]: parsing config file is succeeded path="/etc/td-agent/td-agent.conf" 2021-02-18 18:38:09 +0900 [info]: brokers has been set: ["10.232.0.134:9092", "10.232.0.94:9092", "10.232.0.78:9092"] 2021-02-18 18:38:09 +0900 [info]: brokers has been set: ["10.232.0.134:9092", "10.232.0.94:9092", "10.232.0.78:9092"] 2021-02-18 18:38:09 +0900 [error]: config error file="/etc/td-agent/td-agent.conf" error_class=Fluent::ConfigError error="Unknown parser plugin 'multi_format'. Run 'gem search -rd fluent-plugin' to find plugins"

td-agent-gem list|grep fluent

fluent-plugin-elasticsearch (2.10.3, 1.17.1) fluent-plugin-kafka (0.16.0, 0.7.3) fluent-plugin-multi-format-parser (1.0.0) fluent-plugin-record-modifier (1.1.0) fluent-plugin-rewrite-tag-filter (2.1.0) fluent-plugin-s3 (1.1.3) fluent-plugin-td (1.0.0) fluent-plugin-td-monitoring (0.2.4) fluent-plugin-webhdfs (1.2.3)

lsb_release -a

No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 16.04.7 LTS Release: 16.04 Codename: xenial

Is there any solution for this ?

Thanks.

eugenegwon commented 3 years ago

The root cause was improper permission settings of td-agent directories. Not sure why it happens... I changed owner of these directories from root to td-agent :