search

tagomoris / fluent-plugin-route

This is copy of frsyuki's out_route
Apache License 2.0
20 stars 6 forks source link

Not working with fluentd 0.14.21 #10

Open Globegitter opened 6 years ago

Globegitter commented 6 years ago

I have a config something like:

    <match kubernetes.**>
      @type route
      remove_tag_prefix kubernetes
      <route **>
        add_tag_prefix kubernetes_copy        
        copy # For fall-through. Without copy, routing is stopped here. 
      </route>
      <route **>
        add_tag_prefix slack        
        copy
      </route>
    </match>

    <match slack.**>
      @type slack
      webhook_url something
      channel something
      username something
      icon_emoji :ghost:
      flush_interval 60s
      message_keys log_msg
    </match>

    <match **>
      @type elasticsearch
      ....
    </match>

For some reason I am seeing everything I expect to see in elasticsearch but not in slack. I have tested the slack config itself and that works. This is supposed to work like this though, right?

Also when I tested the same thing with label it does not seem to work for me.

Edit: I got a working solution now using the relabel plugin, but it seems to me not quite as nice as if I could use this plugin - so would love to get this to work still.

tagomoris commented 6 years ago

Without copy option, routing will stop at first match of <route PATTERN> section. This plugin is to route events to various output plugins using tag pattern. Specifying <route **> twice is out of target (it's better to use copy plugin for such use case).

IMO, with Fluentd v0.14 and label feature, it's best to use both of copy and relabel plugin to route events to two output plugins. This configuration doesn't modify tags of events, so it works with higher performance than using route plugin to modify tags.

<match kubernetes.**>
  @type copy
  <store>
    @type relabel
    @label @slack
  </store>
  <store>
    @type relabel
    @label @es
  </store>
</match>

<label @slack>
  <match **>
    @type slack
    # ...
  </match>
</label>
<label @es>
  <match **>
    @type elasticsearch
    # ...
  </match>
</label>
jinuxstyle commented 6 years ago

Some issue on my test. Can anyone take a look why?

jinuxstyle commented 6 years ago

My fluentd version is 0.14.25, while the route plugin 1.0.0

jinuxstyle commented 6 years ago

I used fluentd 0.12 and route plugin 0.2.0 for comparison, it works.

jinuxstyle commented 6 years ago

It's interesting that after removing the comment following the first copy in the route section, it worked. Is there a bug in the code?

 <route **>
    add_tag_prefix metrics.event
    copy # For fall-through. Without copy, routing is stopped here. 
  </route>

==>

 <route **>
    add_tag_prefix metrics.event
    copy
  </route>
tagomoris commented 6 years ago

Hmm, that might be a kind of bug, related with boolean parameter without (explicit) value and comment...

tagomoris commented 6 years ago

I confirmed that the copy with following comment is disabled, and checking the root cause.

tagomoris commented 6 years ago

This is a bug of Fluentd configuration parser, and reported at https://github.com/fluent/fluentd/pull/1883