influxdata / nginx-influxdb-module

MIT License
39 stars 13 forks source link

Using tags instead of fields #14

Closed yugene closed 6 years ago

yugene commented 6 years ago

Hello!

Is there any reason to send all the metrics as InfluxDB fields? I guess that the following fields should be sent as tags: method, status, uri, extension, content_type.

pmoncadaisla commented 6 years ago

👍 +1 I've filled in this PR: https://github.com/influxdata/nginx-influxdb-module/pull/15/ for this exact request!

fntlnz commented 6 years ago

@yugene @pmoncadaisla you probably want this because tags are easier and more efficient to query than fields, I agree with that.

Initially we used fields because we didn't want to increase the cardinality of the final dataset, this is especially true for the uri.

So what we want to do here I think is to have method, status, extension and content_type as fields and tags while leaving uri only as field.

fntlnz commented 6 years ago

I've been thinking a bit more about this, and now I remembered the initial reason for which everything was a field here. The main reason was that using fields was less expensive on a database point of view (e.g: lower cardinality) while for example different users/configurations will want different combinations of the fields/tags in this case.

So, since this module can only be used with udp and increasing the length of the line is not a good idea (MTU and udp packets constraints) the idea we had was that if one needs something different than the default configuration she/he can achieve that using telegraf with the socket listerner input and InfluxDB output.

For example, to convert fields to tags and vice-versa you can use the Converter processor that's designed to do that and only that.

pmoncadaisla commented 6 years ago

You are right about the cardinality @fntlnz. In my case the measurament without tags is useless, but as you said it can also be achieved using Telegraf as proxy, or oven using Kapacitor (as you suggested me), which I think it is even a better approach!

I can live with the no tags implementations (as it is right now) and try a different approach to get useful data.

Thanks a lot!

yugene commented 6 years ago

@fntlnz, @pmoncadaisla, thank you! We'll try the Converter.