2019-09-09 16:26:26 +0900 [error]: config error file="./fluent.twitter.mecab.conf" error_class=Fluent::ConfigError error="Unknown filter plugin 'mecab'. Run 'gem search -rd fluent-plugin' to find plugins"
3.Source Code
This is the source code of the config file (fluent.twitter.mecab.conf)
`
type twitter
consumer_key [MY_KEY]
consumer_secret [MY_KEY_SECRET]
access_token [MY_TOKEN]
access_token_secret [MY_TOKEN_SECRET]
tag twitter
timeline sampling
lang ja
output_format nest
@type mecab
key text
@type stdout
type stdout
Also this is the source code of the .rb file which I created at [./fluentd/fluent/plugin/]
module Fluent
class MeCabFileter < Filter
Plugin.register_filter('mecab',self)
config_param : key, : string
config_param : tag, : string, default: "mecab"
def initialize
super
require 'natto'
end
def configure(config)
super
@mecab = Natto::MeCab.new
end
def start
super
end
def shutdown
super
end
def filter(tag, time, record)
end
def filter_stream(tag, es)
result_es = MultiEventStream.new
es.each do |time, record|
begin
position = 0
@mecab.parse(pre_process(record[@key])) do |mecab|
length = mecab.surface.length
next if length == 0
new_record = record.clone
new_record["mecab"] = { "word" => mecab.surface,
"length" => length,
"pos" => mecab.feature.split(/\,/),
"position" => position}
result_es.add(time, new_record)
position += length
end
rescue => e
router.emit_error_event(tag, time, record, e)
enend
return result_es
end
def pre_process(text)
# delete URL
return text.gsub(/https?\:\/\/([\w\-]+\.)+[\w-]+(\/[\w-]+)*\/?/,'').gsub(/RT\s*:\*/,'').gsub(/@[\w]+\s*/,'')
end
end
end
`
4.Things I tried
In the error message, it said to try "gem search -rd fluent-plugin" command so I tried to install the following plugin but it didn't solve the problem.
gem search -rld fluent-plugin sudo gem install fluent-plugin-mecab
5.Environment Information
OS : CentOS Linux release
rbenv 1.1.2-2-g4e92322
ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux]
fluentd 1.7.0
mecab of 0.996
1.What I want to do
I want to get the tweet information from Fluentd and analyse using Fluentd. I have done the following steps.
(1)Install rbenv => (2)Install Ruby => (3)Install Fluentd => (4)Install MeCab
2.Problems I am facing & error messages
2019-09-09 16:26:26 +0900 [error]: config error file="./fluent.twitter.mecab.conf" error_class=Fluent::ConfigError error="Unknown filter plugin 'mecab'. Run 'gem search -rd fluent-plugin' to find plugins"
3.Source Code
This is the source code of the config file (fluent.twitter.mecab.conf) `
Also this is the source code of the .rb file which I created at [./fluentd/fluent/plugin/]
module Fluent class MeCabFileter < Filter Plugin.register_filter('mecab',self) config_param : key, : string config_param : tag, : string, default: "mecab"end `
4.Things I tried
In the error message, it said to try "gem search -rd fluent-plugin" command so I tried to install the following plugin but it didn't solve the problem.
gem search -rld fluent-plugin sudo gem install fluent-plugin-mecab
5.Environment Information
OS : CentOS Linux release rbenv 1.1.2-2-g4e92322 ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux] fluentd 1.7.0 mecab of 0.996