repeatedly / fluent-plugin-beats

Fluentd plugin for Elastic beats
Apache License 2.0
45 stars 12 forks source link

Facing a config error saying "Unknown filter plugin 'mecab' when I try to create MeCab plugin for Fluentd and execute. #22

Closed yazaki-tatsuya closed 5 years ago

yazaki-tatsuya commented 5 years ago

1.What I want to do

I want to get the tweet information from Fluentd and analyse using Fluentd. I have done the following steps.

(1)Install rbenv => (2)Install Ruby => (3)Install Fluentd => (4)Install MeCab

2.Problems I am facing & error messages

2019-09-09 16:26:26 +0900 [error]: config error file="./fluent.twitter.mecab.conf" error_class=Fluent::ConfigError error="Unknown filter plugin 'mecab'. Run 'gem search -rd fluent-plugin' to find plugins"

3.Source Code

This is the source code of the config file (fluent.twitter.mecab.conf) `

    type twitter
    consumer_key [MY_KEY]
    consumer_secret [MY_KEY_SECRET]
    access_token [MY_TOKEN]
    access_token_secret [MY_TOKEN_SECRET]
    tag twitter
    timeline sampling
    lang ja
    output_format nest

@type mecab key text @type stdout type stdout

Also this is the source code of the .rb file which I created at [./fluentd/fluent/plugin/] module Fluent class MeCabFileter < Filter Plugin.register_filter('mecab',self) config_param : key, : string config_param : tag, : string, default: "mecab"

def initialize
    super
    require 'natto'
end

def configure(config)
    super
    @mecab = Natto::MeCab.new
end

def start
    super
end

def shutdown
    super
end

def filter(tag, time, record)
end

def filter_stream(tag, es)
    result_es = MultiEventStream.new
    es.each do |time, record|
    begin
        position = 0
        @mecab.parse(pre_process(record[@key])) do |mecab|

            length = mecab.surface.length
            next if length == 0

            new_record = record.clone
            new_record["mecab"] = {    "word" => mecab.surface,
                                    "length" => length,
                                    "pos" => mecab.feature.split(/\,/),
                                    "position" => position}
            result_es.add(time, new_record)

            position += length
        end
        rescue => e
            router.emit_error_event(tag, time, record, e)
        enend
        return result_es
    end
    def pre_process(text)
        # delete URL
        return text.gsub(/https?\:\/\/([\w\-]+\.)+[\w-]+(\/[\w-]+)*\/?/,'').gsub(/RT\s*:\*/,'').gsub(/@[\w]+\s*/,'')
    end
end

end `

4.Things I tried

In the error message, it said to try "gem search -rd fluent-plugin" command so I tried to install the following plugin but it didn't solve the problem. gem search -rld fluent-plugin sudo gem install fluent-plugin-mecab

5.Environment Information

OS : CentOS Linux release rbenv 1.1.2-2-g4e92322 ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux] fluentd 1.7.0 mecab of 0.996

yazaki-tatsuya commented 5 years ago

Closing since it is raised in the wrong .