Open rljie opened 4 years ago
Framework filebeat --> kafka --> fluentd --> elasticsearch
Describe the problem filebeat comes with an array of tags, but fluentd doesn't filter out! The filter in logstash is
mutate { add_field => { "city" => "%{[tags][0]}" "ip" => "%{[tags][1]}" } }
filebeat output configuration
output.kafka: hosts: ["120.0.0.1:9092"] topic: test required_acks: 1 tags: ["xinyi", "19.18.14.168"] Fluentd Filter Configuration <filter **> @type record_transformer enable_ruby true auto_typecast true <record> city %{[tags][0]} ip %{[tags][1]} </record> </filter>
result The city and IP fields inserted into es are %{[tags][0]} and %{[tags][1} So, filtering didn't work, looking forward to answer, thanks
Framework filebeat --> kafka --> fluentd --> elasticsearch
Describe the problem filebeat comes with an array of tags, but fluentd doesn't filter out! The filter in logstash is
filebeat output configuration
result The city and IP fields inserted into es are %{[tags][0]} and %{[tags][1} So, filtering didn't work, looking forward to answer, thanks