Improved tagging capabilities

[krjackso]

I have a similar request to #103 where I want to whitelist tags, specifically the name tag that comes from the MBean name property.

The issue I'm having is that we have tagged our ec2 hosts with name and we can not easily get system-wide results when creating graphs with something along the lines of get jmx.domain.attribute1, from attribute_group:group1, sum by name since it splits it by both the bean name and the instance name.

To solve this, you could use something similar to what you have done aliases e.g.

include:
  attribute:
    attribute1:
      alias: jmx.$domain.$attribute
      metric_type: gauge
      tags_include:
        - "attribute_group:group1"
        - "bean_name:$name"
  bean_regex: Domain:name=group1-*

I was also thinking though that what I really want is to put the tags on all attributes of the bean, not just one. The above config becomes repetitive with multiple attributes. It would be great if the config could be simplified further to:

include:
  attribute:
    - attribute1
    - attribute2
  bean_regex: Domain:name=group1-*
  tags_include:
     attribute_group: group1
     bean_name: $name

Maybe I am missing something? Is there a way to achieve this with the current jmxfetch?

[yannmh]

Thanks for the details @krjackso,

This issue actually falls within a bigger scope: the lack of name-spacing in tags. Integrations produce various tags, with more or less generic names, which can conflicts with the host tags. We had a similar issue before, with JMXFetch producing a host tag. This constrained us to rename it to jmx_server, c.f. https://github.com/DataDog/jmxfetch/blob/master/CHANGELOG.md#080--09-17-2015

While we are working on resolving this problem at a higher level (for all integrations), we think it would make sense to implement your proposal with #103.

[krjackso]

Updating this with additional information... A use case that my organization (Tealium) is especially interested in, is being able to capture the regular expression groups in the bean name and use those in the enhanced tags.

From the jmx list_everything command:

> service datadog-agent jmx list_everything | grep ItemsProcessed
Not Matching: Bean name: Metrics:name=ItemsProcessed-account1-profile1 - Attribute name: Count  - Attribute type: long
Not Matching: Bean name: Metrics:name=ItemsProcessed-account1-profile1 - Attribute name: Mean  - Attribute type: double
Not Matching: Bean name: Metrics:name=ItemsProcessed-account1-profile2 - Attribute name: Count  - Attribute type: long
Not Matching: Bean name: Metrics:name=ItemsProcessed-account1-profile2 - Attribute name: Mean  - Attribute type: double
Not Matching: Bean name: Metrics:name=ItemsProcessed-account2-profile3 - Attribute name: Count  - Attribute type: long
Not Matching: Bean name: Metrics:name=ItemsProcessed-account2-profile3 - Attribute name: Mean  - Attribute type: double
Not Matching: Bean name: Metrics:name=ItemsProcessed-account2-profile4 - Attribute name: Count  - Attribute type: long
Not Matching: Bean name: Metrics:name=ItemsProcessed-account2-profile4 - Attribute name: Mean  - Attribute type: double

We want the following config to produce two metrics jmx.metrics.count and jmx.metrics.mean with 4 unique tag combinations for account, profile, bean_name along with the other tags we have configured with the datadog agent:

include:
  bean_regex: ^Metrics:name=ItemsProcessed-(.*)-(.*)$
  attributes
    - Count
    - Mean
  tags:
    account: $1
    profile: $2
    bean_name: items_processed

[truthbk]

@krjackso thank you for your feedback. We are going to be looking into this as it looks like a useful use case, extracting tags from metric names is nothing new and would definitely add value to JMX. I believe the implementation would not take too long. That said, this sort of implementation could be spammy in the wrong hands, so we'll have to discuss and consider in some more depth any possible downsides. We'll keep you posted. Thanks again!

[yannmh]

@krjackso can you let us now if https://github.com/DataDog/jmxfetch/pull/116 and https://github.com/DataDog/jmxfetch/pull/117 solve your problem please?

It adds the ability to blacklist some tags, and to define your own using regexp group name substitutions.

[krjackso]

@yannmh I see that #116 and #117 add the ability to exclude tags and add tags for all attributes in a bean, but I don't see in those two PRs where the regular expression grouping is done

[yannmh]

@krjackso,

You're right regular expression grouping, as described in https://github.com/DataDog/jmxfetch/issues/108#issuecomment-260725265, is not available yet. At the moment, tags can be added and named after attribute values.

[krjackso]

@yannmh Any update on this?

[yannmh]

Hey @krjackso,

We are going to work on this feature for the next JMXFetch release. I'll certainly let you know when I have more updates.

[ipleten]

It would be really great to have such "regex group" tagging. We had to workaround that in our installation by ugly scripts.

[sworisbreathing]

Hi folks, any update on this? It's tagged for 0.14.0 milestone which has been closed, but this issue is still open. I'm also keen to use this feature once it's available.

[nmuesch]

Hey! We recently released version 0.19 of JMXFetch that comes with the ability to supply tags based on regex groupings of the bean names. The yaml files for JMX checks would look like the example provided by @krjackso where its required to use bean_regex and supply the key of each grouped tag.

This version of JMXFetch was released with Agent version 6.1.0 and will also be released with version 5.23.0

[sworisbreathing]

fwiw, this was implemented in #167