fangli / fluent-plugin-influxdb

A buffered output plugin for fluentd and InfluxDB
MIT License
111 stars 65 forks source link

Document the format of a message #40

Closed stvnwrgs closed 8 years ago

stvnwrgs commented 8 years ago

Im trying to figure out how a line in a file must be formated to be forwarded to influx.

After 3 hours with guessing, looking at tests, previous issues and looking a little bit into the code, i think this ticket is my last hope.

Could you add a line to the config section with a example line? This would be a big help!

I think a good example would be the InfluxDB example:

cpu_load_short,host=server02 value=0.67
cpu_load_short,host=server02,region=us-west value=0.55 1422568543702900257
cpu_load_short,direction=in,host=server01,region=us-west value=2.0 1422568543702900257

What should my config and input line look like, to get the same data like the influxdb example, into my influx?:

<source>
    type tail
    path /tmp/dummy.log
    pos_file /tmp/dummy.log.pos
    tag dummy.test
    format none
  </source>
  <match dummy.*>
    type influxdb
    host 10.10.10.1
    port 8086
    dbname db
    user admin
    password xxxxxx
    use_ssl false
    tag_keys ["host"]
    time_precision s
  </match>
repeatedly commented 8 years ago

Sorry, I don't understand your problem. This plugin uses influxdb client so you can see the API on the influxdb document. I think writing format in the README is not needed because it is redundant and syncing document increases maintainance cost.

stvnwrgs commented 8 years ago

I can understand that you don't want to increase the maintenance costs, but im still not getting forward. My final goal is to get stdout from docker containers that matches a certain line. something like: my.metric tag1=asd,tag2=hostname value=3 to influx.

For testing reasons, i tried to read them from a file first. (Like the config above)

When I echoed the lines into the file I got a new measurement in influx, but without any value or tag.

That's the reason i am asking for help here. I don't get the relation of tag_keys, the value, and the message format in my file. Maybe my problem is a more general misunderstanding of chunk.msgpack_each?

repeatedly commented 8 years ago

I'm very confusing. This title is "Document the format of a message" so I assume you want to add Influxdb's REST API spec to README. But your response contains "im still not getting forward". I still can't understand what the essential of this issue.

reasn commented 8 years ago

I accidentally stumbled across this issue having the same question, if I get things right I might clarify:

As I understand from the description fluent-plugin-influxdb allows writing data from e.g. the "tail" source plugin to influxdb. If I understood this right the question is:

repeatedly commented 8 years ago

How do lines have to be formatted so that they can be parsed by your plugin?

See: http://docs.fluentd.org/articles/life-of-a-fluentd-event

Is there any live example of log output being parsed and transmitted to influxdb by the fluent-plugin-influxdb plugin?

InfluxData document writes live example.

https://docs.influxdata.com/influxdb/v0.10/guides/writing_data/

stvnwrgs commented 8 years ago

Great, was my fault. Thank you!