tagomoris / fluent-plugin-flowcounter

TODO: one-line summary of your gem
Other
53 stars 20 forks source link

count is 0 when there are input records #25

Closed chgliu2018 closed 5 years ago

chgliu2018 commented 5 years ago

hi~ I use input_forward to push records to fluentd with your plugin. The td-agent.conf is:

<source>
  @type forward
  @id input_forward
  format none
  port 5170
  bind 0.0.0.0
  tag profile
  delimiter \n
</source>

<match profile>
  @type copy
  <store>
    @type stdout
    @id output_profile
  </store>
  <store>
    @type flowcounter
    count_keys *
    unit second
    aggregate all
    tag fluentd.traffic
  </store>
</match>

<match fluentd.traffic>
    @type stdout
    @id internal_metrics
</match>

In the /var/log/td-agent/td-agent.log, I can see the output of my records and flowcounters. But the couters are all zero:

2018-09-28 12:06:57 +0000 [warn]: #0 [input_forward] incoming chunk is broken: host="127.0.0.1" msg={"srvname"=>"P10077", "cmd"=>"exchange.boot", "duration"=>6.9144003093243e-05, "ok"=>true, "tag"=>"S2P"}
2018-09-28 12:06:57 +0000 [warn]: #0 [input_forward] incoming chunk is broken: host="127.0.0.1" msg={"srvname"=>"usc", "cmd"=>"node.register", "duration"=>0.0082601030021906, "ok"=>true, "tag"=>"US"}
2018-09-28 12:06:57 +0000 [warn]: #0 [input_forward] incoming chunk is broken: host="127.0.0.1" msg={"srvname"=>"P10077", "cmd"=>"chat.create_group", "duration"=>0.00014257299900055, "ok"=>true, "tag"=>"S2P"}
2018-09-28 12:06:58.323750342 +0000 fluentd.traffic: {"count":0,"bytes":0,"count_rate":0.0,"bytes_rate":0.0}
2018-09-28 12:06:59 +0000 [warn]: #0 [input_forward] incoming chunk is broken: host="127.0.0.1" msg={"srvname"=>"P10077", "cmd"=>"client.hello", "duration"=>6.766002625227e-06, "ok"=>true, "tag"=>"C2S"}
2018-09-28 12:06:59 +0000 [warn]: #0 [input_forward] incoming chunk is broken: host="127.0.0.1" msg={"srvname"=>"P10077", "cmd"=>"hall.gen_token", "duration"=>0.0024800719991326, "ok"=>true, "tag"=>"C2S"}
2018-09-28 12:06:59.823559879 +0000 fluentd.traffic: {"count":0,"bytes":0,"count_rate":0.0,"bytes_rate":0.0}
2018-09-28 12:06:59 +0000 [warn]: #0 [input_forward] incoming chunk is broken: host="127.0.0.1" msg={"srvname"=>"usc", "cmd"=>"user.online", "duration"=>7.3185000568628e-05, "ok"=>true, "tag"=>"US"}
2018-09-28 12:07:01.324681750 +0000 fluentd.traffic: {"count":0,"bytes":0,"count_rate":0.0,"bytes_rate":0.0}
2018-09-28 12:07:02.823251973 +0000 fluentd.traffic: {"count":0,"bytes":0,"count_rate":0.0,"bytes_rate":0.0}
2018-09-28 12:07:03.823780902 +0000 fluentd.traffic: {"count":0,"bytes":0,"count_rate":0.0,"bytes_rate":0.0}
2018-09-28 12:07:05.323417988 +0000 fluentd.traffic: {"count":0,"bytes":0,"count_rate":0.0,"bytes_rate":0.0}
2018-09-28 12:07:06.823469750 +0000 fluentd.traffic: {"count":0,"bytes":0,"count_rate":0.0,"bytes_rate":0.0}
2018-09-28 12:07:07.823947683 +0000 fluentd.traffic: {"count":0,"bytes":0,"count_rate":0.0,"bytes_rate":0.0}
2018-09-28 12:07:09.323681640 +0000 fluentd.traffic: {"count":0,"bytes":0,"count_rate":0.0,"bytes_rate":0.0}
2018-09-28 12:07:10 +0000 [warn]: #0 [input_forward] incoming chunk is broken: host="127.0.0.1" msg={"srvname"=>"P10077", "cmd"=>"hall.login", "duration"=>10.467536419999, "ok"=>false, "tag"=>"C2S"}
2018-09-28 12:07:10.823403507 +0000 fluentd.traffic: {"count":0,"bytes":0,"count_rate":0.0,"bytes_rate":0.0}
2018-09-28 12:07:10 +0000 [warn]: #0 [input_forward] incoming chunk is broken: host="127.0.0.1" msg={"srvname"=>"P10077", "cmd"=>"exchange.player_login", "duration"=>10.998356090002, "ok"=>false, "tag"=>"S2P"}
2018-09-28 12:07:11.823934825 +0000 fluentd.traffic: {"count":0,"bytes":0,"count_rate":0.0,"bytes_rate":0.0}
2018-09-28 12:07:11 +0000 [warn]: #0 [input_forward] incoming chunk is broken: host="127.0.0.1" msg={"srvname"=>"P10077", "cmd"=>"chat.create_user", "duration"=>5.6488998234272e-05, "ok"=>true, "tag"=>"S2P"}
2018-09-28 12:07:11 +0000 [warn]: #0 [input_forward] incoming chunk is broken: host="127.0.0.1" msg={"srvname"=>"P10077", "cmd"=>"chat.add_user_group", "duration"=>5.3125001490116e-05, "ok"=>true, "tag"=>"S2P"}
2018-09-28 12:07:13.323169155 +0000 fluentd.traffic: {"count":0,"bytes":0,"count_rate":0.0,"bytes_rate":0.0}
2018-09-28 12:07:14.323404462 +0000 fluentd.traffic: {"count":0,"bytes":0,"count_rate":0.0,"bytes_rate":0.0}
2018-09-28 12:07:15.323949766 +0000 fluentd.traffic: {"count":0,"bytes":0,"count_rate":0.0,"bytes_rate":0.0}
2018-09-28 12:07:16.823505401 +0000 fluentd.traffic: {"count":0,"bytes":0,"count_rate":0.0,"bytes_rate":0.0}
2018-09-28 12:07:18.323696075 +0000 fluentd.traffic: {"count":0,"bytes":0,"count_rate":0.0,"bytes_rate":0.0}
2018-09-28 12:07:19.824698277 +0000 fluentd.traffic: {"count":0,"bytes":0,"count_rate":0.0,"bytes_rate":0.0}
2018-09-28 12:07:21.323694822 +0000 fluentd.traffic: {"count":0,"bytes":0,"count_rate":0.0,"bytes_rate":0.0}

Please help for checking my conf file or there is something wrong I can't catch. Thank you!

tagomoris commented 5 years ago

Your log says that incoming chunk (for forward input plugin) is broken. Check your input source data at first.