配置decorate_events: true后，按topic过滤不成功(gohangout-input-confluent-kafka-go)

[bj1116]

https://github.com/arterhuo/gohangout-input-confluent-kafka-go 配置如下： inputs:

• Ckafka: decorate_events: true topic: topic1: 1 ………………………………………………………… outputs:
• Stdout: if:
  • 'EQ(@metadata,kafka,topic,"topic1")'

没有数据输出，去掉if正常，其他kafka input正常。

[childe]

这个Input看起来并没有 decorate_events 参数。

[bj1116]

代码里有该参数的，并且不加if条件也能输出meta的相关值；

[childe]

测下来功能正常，如下。 你把 不加if条件也能输出meta的相关值 这个数据发出来看一下。

inputs:
  - Stdin:
      codec: json

outputs:
  - Stdout:
      if:
        - 'EQ(@metadata,kafka,topic,"abcd")'

# cat i |  gohangout --config a.yml -exit-when-nil
gohangout version
{"@metadata":{"kafka":{"topic":"abcd"}},"@timestamp":"2022-05-04T22:45:32.050106+08:00"}

# cat i
{"@metadata":{"kafka":{"topic":"abcd"}}}
{"@metadata":{"kafka":{"topic":"abc"}}}

[bj1116]

{"@metadata":{"kafka":{"offset":73063389,"partition":0,"topic":"kd_agent_source_topic"}},"@timestamp":"2022-05-05T11:51:14.893134709+08:00","hostname":"cyber-oradb","in":8921088,"ip_address":"192.168.100.55","kd_type":"metric","meta_class":"linux","metric_class":"swap","out":9134080,"timestamp":1651722670} 感觉是ckafka插件的问题，别的input（kafka，samara等）都可以；是不是ckafka输出的在上下文的时候字节序不一样？

[childe]

看你最上面写的配置，topic名字和这个数据不匹配。

发自我的iPhone

------------------ 原始邮件 ------------------ 发件人: Jeff.Ran @.> 发送时间: 2022年5月5日 11:53 收件人: childe/gohangout @.> 抄送: childe @.>, State change @.> 主题: Re: [childe/gohangout] 配置decorate_events: true后，按topic过滤不成功(gohangout-input-confluent-kafka-go) (Issue #175)

@.**@.":"2022-05-05T11:51:14.893134709+08:00","hostname":"cyber-oradb","in":8921088,"ip_address":"192.168.100.55","kd_type":"metric","meta_class":"linux","metric_class":"swap","out":9134080,"timestamp":1651722670}

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you modified the open/close state.Message ID: @.***>

[bj1116]

是一致的，上面的是写的一个测试topic，下面是真实的，if语句里面也是真实的 ` inputs:

• Ckafka: decorate_events: true topic: kd_agent_source_topic: 1 codec: 'json' consumer_settings: …… outputs:
• Stdout: if:
  • 'EQ(@metadata,kafka,topic,"kd_agent_source_topic")' `

[bj1116]

再次测试结果，不支持多层的，单层这样是可以的； Stdout: if:

• 'EQ(kd_type,"metric")'

[childe]

再次测试结果，不支持多层的，单层这样是可以的； Stdout: if:

• 'EQ(kd_type,"metric")'

看我上面给的测试结果，多层是可以的。

我还是怀疑你配置或者是数据没有对应起来，或者是哪里写错了。 建议你使用 stdin stdout 来测试一下。如果还不符合预期，把测试数据包括配置发出来看下。

[bj1116]

Topic 配置（换成- 'EQ(kd_type,"metric")'这个正常输出）： `inputs:

• Ckafka: decorate_events: true topic: kd_agent_source_topic: 1 codec: 'json' consumer_settings: bootstrap.servers: '192.168.100.219:28083' group.id: 'kdadapter-1' client.id: 'reZbnTSg' auto.commit.interval.ms: '1000' enable.auto.commit: true auto.offset.reset: 'latest' # 'smallest, earliest, beginning, largest, latest, end, error' security.protocol: 'SASL_PLAINTEXT' # 'PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL' sasl.mechanism: 'PLAIN' # 'GSSAPI, PLAIN, SCRAM-SHA-256, SCRAM-SHA-512, OAUTHBEARER' sasl.username: 'admin' sasl.password: 'admin-secret' outputs:
• Stdout: if:
  • 'EQ(@metadata,kafka,topic,"kd_agent_source_topic")'

    - 'EQ(kd_type,"metric")'

    消息数据： {"@metadata":{"kafka":{"offset":72074895,"partition":2,"topic":"kd_agent_source_topic"}},"@timestamp":"2022-05-16T09:09:55.122004989+08:00","current_disk_queue_length":0,"free_megabytes":69,"hostname":"cyber-poc1","instance":"C:","ip_address":"192.168.100.221","kd_type":"metric","meta_class":"windows","metric_class":"win_logicaldisk","objectname":"LogicalDisk","percent_disk_read_time":0,"percent_disk_time":2.190469980239868,"percent_disk_write_time":2.190469980239868,"percent_free_space":0.16888171434402466,"percent_idle_time":98.0695571899414,"timestamp":1652663390} {"@metadata":{"kafka":{"offset":72074896,"partition":2,"topic":"kd_agent_source_topic"}},"@timestamp":"2022-05-16T09:09:55.122099633+08:00","current_disk_queue_length":0,"free_megabytes":71,"hostname":"cyber-poc1","instance":"HarddiskVolume1","ip_address":"192.168.100.221","kd_type":"metric","meta_class":"windows","metric_class":"win_logicaldisk","objectname":"LogicalDisk","percent_disk_read_time":0,"percent_disk_time":0,"percent_disk_write_time":0,"percent_free_space":71.71717071533203,"percent_idle_time":100.05501556396484,"timestamp":1652663390}`

[bj1116]

解决了，https://github.com/arterhuo/gohangout-input-confluent-kafka-go/pull/2 多谢@yandd