Open wasifshareef opened 1 month ago
here are the schema files i am trying to use cat schemazeek.avsc { "type": "record", "name": "LogRecord", "fields": [ {"name": "_node_name", "type": "string"}, {"name": "ts", "type": "float"}, {"name": "uid", "type": "string"}, {"name": "id.orig_h", "type": "string"}, {"name": "id.orig_p", "type": "int"}, {"name": "id.resp_h", "type": "string"}, {"name": "id.resp_p", "type": "int"}, {"name": "id.vlan", "type": "float"}, {"name": "proto", "type": "string"}, {"name": "duration", "type": "float"}, {"name": "orig_bytes", "type": "float"}, {"name": "resp_bytes", "type": "float"}, {"name": "conn_state", "type": "string"}, {"name": "local_orig", "type": "boolean"}, {"name": "local_resp", "type": "boolean"}, {"name": "missed_bytes", "type": "int"}, {"name": "orig_pkts", "type": "int"}, {"name": "orig_ip_bytes", "type": "int"}, {"name": "resp_pkts", "type": "int"}, {"name": "resp_ip_bytes", "type": "int"}, {"name": "tailed_path", "type": "string"}, {"name": "beat.hostname", "type": "string"}, {"name": "tag", "type": "string"}, {"name": "clientkey", "type": "string"}, {"name": "consumer_key", "type": "string"}, {"name": "service", "type": "string"}, {"name": "history", "type": "string"}, {"name": "tunnel_parents", "type": "string"} ] }
++++++
{ "type": "record", "name": "LogRecord", "fields": [ {"name": "timestamp", "type": "string"}, {"name": "flow_id", "type": "int"}, {"name": "in_iface", "type": "string"}, {"name": "event_type", "type": "string"}, {"name": "src_ip", "type": "string"}, {"name": "src_port", "type": "float"}, {"name": "dest_ip", "type": "string"}, {"name": "dest_port", "type": "float"}, {"name": "proto", "type": "string"}, {"name": "snmp", "type": "string"}, {"name": "beat.hostname", "type": "string"}, {"name": "tag", "type": "string"}, {"name": "clientkey", "type": "string"}, {"name": "consumer_key", "type": "string"}, {"name": "alert", "type": "string"}, {"name": "flow", "type": "string"}, {"name": "payload_printable", "type": "string"}, {"name": "stream", "type": "float"}, {"name": "app_proto", "type": "string"}, {"name": "netflow", "type": "string"}, {"name": "tcp", "type": "string"}, {"name": "icmp_type", "type": "float"}, {"name": "icmp_code", "type": "float"} ] }
Thanks,
HI, I am trying to use columnify to generate output in parquet format to send data to azure using azure plugin,however i am getting error when i run the fluentd container with below error . I will appreciate if someone can help me here.