[Feature Request] Allow Regex on [[input]] measurement configuration

[sbengo]

Hi,

I have tested to inject some tags on all measurements grouped by host. Note that those tags are different for every host.

As I have seen, it must be done creating an input configuration for every measurement, and, in my case, on every host. It seems that there is no mechanisms to apply the same tags on all measurements, so the result is a giant configuration file which it makes quite hard to maintain.

This problem would be reduced if the Measurement clause would allow some regex expression to group inputs tags rules.

Thank, Regards!

[adejoux]

Hello,

I will need more details because I am not seeing the case where you will need to put tags on all measurements for a host.

For me, you should put all your custom tags on CPU_ALL only and use templating to use the tags in other measurements. I am talking about it in this post: https://www.djouxtech.net/posts/nmon2influxdb-tagging-partitions/

It would also avoid to put too much data by measurements.

For regexp, it's already possible to do it on the match value of the measurement like I explained in bug #42

If possible can you give an example of syntax you would like to have ? It's definitely possible to do it :)

Regards,

Alain

[sbengo]

Hi @adejoux , sorry, couldn't answer you earlier!

All our host are characterised by several tags like datacenter, enviroment (prod, devel) and more. Having tags on all measurement help us to make custom GROUP BY statements to compare in a high-level (or low-level) way our different hosts.

Example: Our host are located into different datacenters. We would like to compare the CPU usage between different datacenters so our query in InfluxDB would use the GROUP BY datacenter statement

[adejoux]

Hello,

I understand what you want to do but i think you don't need to do it on every measurement.

If you create template variables like this ones:

You can use them on other measurements:

On all measurements, you will have the host tag. The recommendation I have is to apply the tag datacenter on your measurement cpu and filter the host tag values based on your datacenter tag.

maybe a better example is what I do for linux servers:

You see I have environment and application tags but it's only used to filter the server names.

And in my charts I am only using the server tag:

Like I said i am not against adding this feature but I am not sure it's required here. let meknow what you think about it.

Thanks,

Alain

[adejoux]

I forgot to say but server can be multi value allowing you to select many systems in the same graph.

[sbengo]

Thanks for the answer and examples @adejoux!

I'm quite sorry, my example above was not the best one. I can understand that people may not need it (actually we have are doing the same as you have exposed in order to reduce where clauses in large SNMP data dashboards), but this can be an enhancement to allow the user group by other tags and improve query flexibility.

Another example (and real one): Let's say that I want to make a Dashboard with all measurements but not grouping them by host: I want to group by datacenters and see the difference between them.

Furthermore, other applications like Kapacitor or custom services which requires InfluxDB queries reduces its functionalities as they can not group by other tags.

Thanks again, Regards

[adejoux]

Hello,

Ok I understand it better now. I will implement it. Sorry for asking so many details but I didn't use kapacitor so I didn't think about it.

Regards,

Alain

[adejoux]

Hello again,

Would it be okay if it was a simple expression where comma would be used as separator ?

For example,

[input]]
  Measurement="CPU_ALL,FCADAPTER,DISKXFER"
  Name="partition"
  Match="lpm"
  [[input.tag]]
    Name="datacenter"
    Value="DC1"
  [[input.tag]]
    Name="group"
    Value="LPM"

Here it would match the measurements CPU_ALL, FCADAPTER and DISKXFER.

For me it would be better from aperformance point of view than true regular expression because I would not need to compile and match it against all measurements.

I would like to know if it would meet your needs.

Regards,

Alain

[sbengo]

Hi @adejoux !

Yes, would be fine, I understand you want to prioritise performance.

Obviously I would prefer a regex since with a simple .* would take all measurements and apply the tag for all without previously knowing the measurements.

Thanks for all, Greetings!

[adejoux]

Hello,

I implemented the feature. Soo you can specify the measurements like that:

[input]]
  Measurement="CPU_ALL,FCADAPTER,DISKXFER"
  Name="partition"
  Match="lpm"
  [[input.tag]]
    Name="datacenter"
    Value="DC1"
  [[input.tag]]
    Name="group"
    Value="LPM"

But I also added a option to match all measurements if you don't specify the measurelment parameter.

[input]]
  Name="partition"
  Match="lpm"
  [[input.tag]]
    Name="datacenter"
    Value="DC1"
  [[input.tag]]
    Name="group"
    Value="LPM"

I still need to update the documentation before releazsing the new version but you can find attached a binary for testing.

Regards,

Alain nmon2influxdb.gz

[sbengo]

Thank you so much @adejoux !!

I will try to test it in this week and give you feedback!

Greetings!

[sbengo]

Hello @adejoux !

Awesome! I tested it without setting up measurement parameter and now it is adding the tags for all measurements! EDIT: You can close the issue!

Thank you!!

[adejoux]

Great. thanks :)