micrometer-metrics / micrometer

An application observability facade for the most popular observability tools. Think SLF4J, but for observability.
https://micrometer.io
Apache License 2.0
4.39k stars 966 forks source link

Dynamic MeterFilter results in wrong Meter being reused #5233

Open NiFNi opened 4 days ago

NiFNi commented 4 days ago

Describe the bug When using a MeterFilter which dynamically adjusts the tags of a meter the MeterRegistry reuses previously created Meters even though the MeterFilter would change the tags to something that does not equal the first used tags.

An example spring boot project can be found here: https://github.com/NiFNi/micrometer-dynamic-tag-example

I have defined a MeterFilter like this so that it it adds a dynamic tag containing a random uuid:

    @Bean
    public MeterFilter meterFilter() {
        return new MeterFilter() {
            @Override
            public Meter.Id map(Meter.Id id) {
                var tags = Tags.concat(id.getTagsAsIterable(), Tags.of("DYNAMIC_FILTER_TAG", UUID.randomUUID().toString()));
                return id.withTags(tags);
            }
        };
    }

Then I have a Counted Get endpoint:

    @GetMapping
    @Counted(value = "foo.bar")
    public void get() {
    }

When calling the endpoint multiple times the counter with the first created UUID is increased instead of creating a new counter for each request.

Environment

When using micrometer-core version 1.12.x the issue is not present and each request creates its own counter.

To Reproduce How to reproduce the bug: See example above. To test for yourself run the spring boot application from the repo and check http://localhost:8080/actuator/prometheus after calling GET http://localhost:8080 multiple times.

Expected behavior Each call should create its own counter as it is doing with previous micrometer versions.

Additional context When searching through things I found this pull request: https://github.com/micrometer-metrics/micrometer/pull/4857 which seems to have introduced some caching of meters which does not seem to take dynamic MeterFilters into account as the MeterFilter is executed only when the cached Meter is not valid anymore or no cached Meter is found.

We need this feature for a multi tenancy system where each request can be in a different tenant context. The tenant should be added to all metrics to allow filtering metrics by specific tenants. If MeterFilters are not intended to be used like this and there is a better way to implement this apart from using a MeterFilter which dynamically adds this tag I would also be open for that.