[Cento]What's the flow format exported from Cento to Kafka

[Howard-Chang]

Hi, I refer this video(https://www.youtube.com/watch?v=You1ZGNtuC8) to connect cento with Kafka.Until exporting to Kafka is ok, but the format is composed of many numbers like this photo: I don't know what's the format. Cento command: cento -i ens2f1 --kafka "172.31.214.124:9092,172.31.214.124:9093;topicFlowsCento" --v9 (I want to specified netflow v9) then I want to export the data to ES through logstash and this is the config:

input {
    kafka {
            bootstrap_servers => "localhost:9092"
            topics => "topicFlowsCento"
            codec => netflow
    }
}
 output {
        elasticsearch {
            hosts => ["172.31.214.124:9200"]
        }
        stdout{codec=> rubydebug}
    }

logstash log:

[2018-03-25T14:11:16,261][WARN ][logstash.codecs.netflow  ] Ignoring Netflow version v31522
[2018-03-25T14:11:16,261][WARN ][logstash.codecs.netflow  ] Ignoring Netflow version v31522
[2018-03-25T14:11:16,261][WARN ][logstash.codecs.netflow  ] Ignoring Netflow version v31522

How could I export the flow data to Elasticsearch through logstash? thank you in advance!

[simonemainardi]

Numbers you are seeing are actually the (standardized) integer keys representative of netflow fields. See https://www.ietf.org/rfc/rfc3954.txt for the whole list of keys and associated netflow fields. For example IPV4_SRC_ADDR has key 8. For the sake of performance, cento only exports integer netflow keys and not text labels. If you need text labels you can use logstash to perform the key-to-label substitutions or nProbe directly as it supports kafka export with labels.

As regards your logstash configuration, set codec => "json" on your kafka {} and it will work. Flows are converted to JSON right before the kafka export.