MicrometerObservationCapability does not add metadata to prometheus metrics

[mariusingjer]

Describe the bug

Sprint Boot: 3.2.1 Spring Cloud: 2023.0.0

pom:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
<dependency>
    <groupId>io.github.openfeign</groupId>
    <artifactId>feign-micrometer</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>

if enabled, the MicrometerObservationCapability gets added and "http.client.requests" metrics (or observations? sorry don't know the correct terminology) are recorded, but they look like this:

http_client_requests_seconds_sum{error="none",http_method="POST",http_status_code="200",http_url="/",} 0.4988093

Other collected metrics for the webflux endpoints look like this:

http_server_requests_seconds_sum{error="none",exception="none",method="POST",outcome="SUCCESS",status="200",uri="/actual-endpoint-route/{something}/and/some-more",} 11.6840865

if I disable the ObservationCapability and add the Micrometer:

    @ConditionalOnProperty(name = ["spring.cloud.openfeign.micrometer.enabled"], havingValue = "false", matchIfMissing = false)
    @Bean
    fun enableMetrics(registry: MeterRegistry): MicrometerCapability {
        return MicrometerCapability(registry)
    }

the /actuator/prometheus endpoint reports metrics with a bit more context, like this:

feign_Client_seconds_sum{client="full.path.to.feign.client.interface",host="the.actual.host",method="classify",uri="/",} 0.4843604

The uri parameter still looks weird, but this provides expected information about the client.

Example client:

@FeignClient(
    name = "categories",
    url = "\${some-config-key.url}",
    path = "\${some-config-key.base-path}",
    configuration = [SomeFeignClientConfiguration::class],
)
interface ThirdPartyApi {
    @RequestMapping(method = [RequestMethod.POST], consumes = ["application/json"])
    fun classify(request: SomeBody): SomeResponse

Expected behaviour: information about the host, url etc (things that are valuable to differentiate between metrics for different feign clients)

[jonatan-ivanov]

Both MicrometerObservationCapability and MicrometerCapability are from feign, not from Spring Could OpenFeign. SC-OpenFeign "just" auto-configures these for you so I feel this issue should have been opened in OpenFeign.

If you open such an issue, please elaborate on these:

Expected behaviour: information about the host, url etc (things that are valuable to differentiate between metrics for different feign clients)

I think I understand host and I think it is useful though I don't understand what you exactly mean by url and "etc". Fyi: this is what I get in my demo/sample app (https://github.com/jonatan-ivanov/teahouse):

http_client_requests_seconds_count{application="tea-service",error="none",http_method="GET",http_status_code="200",http_url="/tealeaves/search/findByName?name={name}",} 1.0

As you can see the url is templated and path and/or query params do not contain their real values since that can be high cardinality data: https://develotters.com/posts/high-cardinality/ If you want those parameters available to you, you need a signal that can contain high cardinality data like tracing or logs. Metrics poorly handle high cardinality.

[mariusingjer]

Sorry for posting in wrong repo, looked at your example and moving the relative url down to the method attribute like this:

    @PostMapping("/classify", consumes = ["application/json"])

The "http_url" parameters changes to "/classify", which makes it possibly in my case to pick out the requests i want to customize.

[jonatan-ivanov]

Oh, I see. I think this might be a bug. I would open two issues for OpenFeign then:

1. Add the host tag (and additional tags you need)
2. Report the issue with the url, they might need a reproducer for this