Common tags not applied to every metric

[eloo]

Just tested the 0.7.0 Release and its new common tags. But it seems that not all metrics are populated with these tags.

# HELP jvm_buffer_memory_used  
# TYPE jvm_buffer_memory_used gauge
jvm_buffer_memory_used{id="direct",} 8192.0
jvm_buffer_memory_used{id="mapped",} 0.0
# HELP jvm_buffer_count  
# TYPE jvm_buffer_count gauge
jvm_buffer_count{id="direct",} 1.0
jvm_buffer_count{id="mapped",} 0.0
# HELP jvm_memory_used  
# TYPE jvm_memory_used gauge
jvm_memory_used{id="Code Cache",} 2.2007552E7
jvm_memory_used{id="PS Eden Space",} 4.88267176E8
jvm_memory_used{id="PS Old Gen",} 3.4389504E7
jvm_memory_used{id="PS Survivor Space",} 0.0
jvm_memory_used{id="Compressed Class Space",} 8292496.0
jvm_memory_used{id="Metaspace",} 6.2853696E7
# HELP logback_events  
# TYPE logback_events counter
logback_events{level="trace",region="us-east-1",stack="prod",} 4965.0
logback_events{level="warn",region="us-east-1",stack="prod",} 109.0
logback_events{level="error",region="us-east-1",stack="prod",} 0.0
logback_events{level="debug",region="us-east-1",stack="prod",} 6065.0
logback_events{level="info",region="us-east-1",stack="prod",} 98.0
# HELP jvm_buffer_total_capacity  
# TYPE jvm_buffer_total_capacity gauge
jvm_buffer_total_capacity{id="direct",} 8192.0
jvm_buffer_total_capacity{id="mapped",} 0.0
# HELP jvm_memory_max  
# TYPE jvm_memory_max gauge
jvm_memory_max{id="Code Cache",} 2.5165824E8
jvm_memory_max{id="PS Eden Space",} 1.36839168E9
jvm_memory_max{id="PS Old Gen",} 2.863661056E9
jvm_memory_max{id="PS Survivor Space",} 3.145728E7
jvm_memory_max{id="Compressed Class Space",} 1.073741824E9
jvm_memory_max{id="Metaspace",} -1.0
# HELP jvm_memory_committed  
# TYPE jvm_memory_committed gauge
jvm_memory_committed{id="Code Cache",} 2.2544384E7
jvm_memory_committed{id="PS Eden Space",} 6.88390144E8
jvm_memory_committed{id="PS Old Gen",} 2.24395264E8
jvm_memory_committed{id="PS Survivor Space",} 3.145728E7
jvm_memory_committed{id="Compressed Class Space",} 8609792.0
jvm_memory_committed{id="Metaspace",} 6.418432E7

Tags were set like this

    @Autowired
    private PrometheusMeterRegistry prometheusMeterRegistry;

    @PostConstruct
    public void setCommonTags() {
        prometheusMeterRegistry.commonTags("stack", "prod", "region", "us-east-1");
    }

Thanks

[checketts]

I think you should set up the common tags in Prometheus instead when you scrape it. That is how it is intended. The metrics that are missing the common tags are coming from the Prometheus direct instrumentation.

I think this feature is more intended for the push model backends

[jkschneider]

It's certainly a bug, or a lazy implementation on my part. The way it is right now, common tags are applied to every metric created after the common tag is appended to the registry. In this case, some of the binders were added before you get a chance to add common tags. This should certainly be improved.

Atlas also has a proprietary property-based common tag config (so it isn't really a push vs. pull thing), but I'd like for folks to at least have the option for their common tags to be portable across monitoring systems. Of course, if you prefer the server-side config, that is perfectly fine!

[jkschneider]

Thanks again @Eloo for the write-up.

[jkschneider]

I'll update the docs before the 0.8.0 release is published, but the short of it is common tags should be applied like so:

@Bean
public MeterRegistryConfigurer configurer() {
  return registry -> registry.commonTags("region", "us-east-1");
}

Any such beans are guaranteed to be applied before any meters are registered.

[eloo]

thanks @jkschneider for fixing this. looking forward to test the next release ;)

[eloo]

yep works for the micrometer metrics but seems not to work on springboot actuator metrics is this intended?

[jkschneider]

Haven't yet bridged legacy Spring Boot actuator metrics to the dimensional system (this would involve a hierarchical -> dimensional mapping in addition to 0.8.0's new support for dimensional -> hierarchical). My current thought is that there should be an equivalent MeterBinder in micrometer for everything that is currently in actuator metrics so that there should not need to be a bridge.

[shaoxt]

Thanks jkschneider.

Here is my sample how to inject "app" as common tags to all metrics.

@Component public class MicroMeterConfig implements MeterRegistryCustomizer {

@Autowired
ConfigurableApplicationContext appContext;

@Override
public void customize(MeterRegistry registry) {
    registry.config().commonTags("app", appContext.getEnvironment().getProperty("info.app.name"));
}

}

[svrdoljak]

How can i configure the common tags in Micronaut Framework. I'm kinda stuck

[shakuzen]

How can i configure the common tags in Micronaut Framework. I'm kinda stuck

That's a question for the Micronaut community. I would consult their Micronaut Micrometer documentation: https://micronaut-projects.github.io/micronaut-micrometer/latest/guide/index.html. Searching for "commonTags" you can see they show how to do it with either a MeterRegistryConfigurer or a MeterFilter.

[svrdoljak]

In case someone needs common tags for Micronaut, here is an example at how i did it:

` @Singleton public class MeterRegistryConfig implements MeterRegistryConfigurer {

@Override
public void configure(MeterRegistry meterRegistry) {
    meterRegistry.config().commonTags("your_custom_key", "your_custom_value");
}

@Override
public boolean supports(MeterRegistry meterRegistry) {
    return true;
}

} `