fangli / fluent-plugin-influxdb

A buffered output plugin for fluentd and InfluxDB
MIT License
111 stars 65 forks source link

auto_tag functionality #100

Open mohsin106 opened 5 years ago

mohsin106 commented 5 years ago

hi, i'm trying to ingest telemetry data from my juniper router into my influx db. i have a few fieldKeys that are of type string. i want to convert these fieldKeys to influx tags by using the auto_tag option, but this is not working.

if i use the tag_keys option then i have the same key saved as a field in influx and as a tag.

example this is what the field keys looks like in my influx db when i ingest the data:

> show field keys from interfaces
name: interfaces
fieldKey         fieldType
--------         ---------
device           string
egress_queue     integer
interface        string
interface_parent string
type             string
value            integer

if i use the tag_key option in my fluent.conf file as listed below, i then end up with a key being a fieldKey and tagKey in my influx db. this gives me problem when i use grafana. i hope i stated that clearly.

i have the following fluent.conf file configured:


#############
## INPUT ##
#############

<source>
   @type udp
   tag jnpr.jvision
   #tag interfaces
   format juniper_jti
   output_format structured
   port 50051
   bind 0.0.0.0
</source>

##############
## OUTPUT ##
##############

<match jnpr.**>
#<match interfaces>
   @type copy

   <store>
      type influxdb
      host localhost
      port 8086
      dbname testfluent
      measurement interfaces
      #user juniper
      #password juniper
      #value_keys ["value"]
      #field_keys ["value"]
      #tag_keys ["interface_parent","device","type"]
      auto_tags enable
      buffer_type memory
      # buffer_chunk_limit 524288 # 512 * 1024
      # buffer_queue_limit 1024
      flush_interval 2
      # retry_limit 17
      # retry_wait 1.0
      # num_threads 1
   </store>
</match>

## match tag=debug.** and dump to console
<match jnpr.**>
#<match interfaces>
  @type stdout
  @id stdout_output
</match>

is there a way to convert a fieldKey to a tagKey and have influx db store it as such without the same key being a field and a tag in influx? thanks!