search

micrometer-metrics / tracing

Provides tracing abstractions over tracers and tracing system reporters.
https://micrometer.io
Apache License 2.0
244 stars 43 forks source link

OTel dependency convergence issue #819

Closed rayusagi closed 1 month ago

rayusagi commented 1 month ago

Since version 1.3.1, there is a dependency convergence error when trying to use the io.micrometer:micrometer-tracing-bridge-otel dependency :

+-<my_module>
  +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.3.1:compile
    +-io.opentelemetry:opentelemetry-sdk:jar:1.37.0:compile
      +-io.opentelemetry:opentelemetry-sdk-metrics:jar:1.37.0:compile
        +-io.opentelemetry:opentelemetry-api-incubator:jar:1.37.0-alpha:runtime
and
+-<my_module>
  +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.3.1:compile
    +-io.opentelemetry:opentelemetry-sdk:jar:1.37.0:compile
      +-io.opentelemetry:opentelemetry-sdk-logs:jar:1.37.0:compile
        +-io.opentelemetry:opentelemetry-api-incubator:jar:1.37.0-alpha:runtime
and
+-<my_module>
  +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.3.1:compile
    +-io.opentelemetry.instrumentation:opentelemetry-instrumentation-api-semconv:jar:1.33.3-alpha:compile
      +-io.opentelemetry.instrumentation:opentelemetry-instrumentation-api:jar:1.33.3:compile
        +-io.opentelemetry:opentelemetry-api-incubator:jar:1.38.0-alpha:runtime
and
+-<my_module>
  +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.3.1:compile
    +-io.opentelemetry.instrumentation:opentelemetry-instrumentation-api-semconv:jar:1.33.3-alpha:compile
      +-io.opentelemetry:opentelemetry-api-incubator:jar:1.38.0-alpha:runtime

io.opentelemetry:opentelemetry-api-incubator is present both in 1.37-0-alpha and 1.38.0-alpha through different dependency paths.

It seems the problem appeared since version 1.3.1 as I do not reproduce it when using version 1.3.0. It is still present on the latest 1.3.3 version.

For projects using enforcer plugins, this is inconvenient as it forces to downgrade the version of the dependency (which is, in my case, managed by Spring Boot), or to exclude/include one of the transitive dependency. Unfortunately, neither option is ideal.

The solution would be either :

Also, it might be a good idea to introduce such enforcer plugin on the project so that it doesn't happen again in the future.

shakuzen commented 1 month ago

I haven't had a chance to look more closely but something is confusing here. We declare a dependency on version 1.38.0 of OTel for our 1.3.x line - see, for example, https://central.sonatype.com/artifact/io.micrometer/micrometer-tracing-bridge-otel/1.3.0 (same for all 1.3.x patch releases as well). So I'm confused why you're showing 1.37.0 in your dependency tree at all.

rayusagi commented 1 month ago

You're right. The 1.37.0 in my example comes from Spring Boot that overrides the version of OpenTelemetry to this. I might create an issue there aswell.

However, I have made a minimal example and there are still discrepancies :

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.example</groupId>
    <artifactId>demo</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>demo</name>
    <dependencies>
        <dependency>
            <groupId>io.micrometer</groupId>
            <artifactId>micrometer-tracing-bridge-otel</artifactId>
            <version>1.3.3</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-enforcer-plugin</artifactId>
                <version>3.5.0</version>
                <executions>
                    <execution>
                        <id>pom-validation</id>
                        <goals>
                            <goal>enforce</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <rules>
                        <dependencyConvergence/>
                    </rules>
                    <fail>true</fail>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

yields 

[ERROR] Rule 0: org.apache.maven.enforcer.rules.dependency.DependencyConvergence failed with message:
[ERROR] Failed while enforcing releasability.
[ERROR]
[ERROR] Dependency convergence error for io.opentelemetry:opentelemetry-api-incubator:jar:1.38.0-alpha paths to dependency are:
[ERROR] +-com.example:demo:jar:0.0.1-SNAPSHOT
[ERROR]   +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.3.3:compile
[ERROR]     +-io.opentelemetry:opentelemetry-sdk:jar:1.38.0:compile
[ERROR]       +-io.opentelemetry:opentelemetry-sdk-metrics:jar:1.38.0:compile
[ERROR]         +-io.opentelemetry:opentelemetry-api-incubator:jar:1.38.0-alpha:runtime
[ERROR] and
[ERROR] +-com.example:demo:jar:0.0.1-SNAPSHOT
[ERROR]   +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.3.3:compile
[ERROR]     +-io.opentelemetry:opentelemetry-sdk:jar:1.38.0:compile
[ERROR]       +-io.opentelemetry:opentelemetry-sdk-logs:jar:1.38.0:compile
[ERROR]         +-io.opentelemetry:opentelemetry-api-incubator:jar:1.38.0-alpha:runtime
[ERROR] and
[ERROR] +-com.example:demo:jar:0.0.1-SNAPSHOT
[ERROR]   +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.3.3:compile
[ERROR]     +-io.opentelemetry.instrumentation:opentelemetry-instrumentation-api-semconv:jar:1.33.5-alpha:compile
[ERROR]       +-io.opentelemetry.instrumentation:opentelemetry-instrumentation-api:jar:1.33.5:compile
[ERROR]         +-io.opentelemetry:opentelemetry-api-incubator:jar:1.40.0-alpha:runtime
[ERROR] and
[ERROR] +-com.example:demo:jar:0.0.1-SNAPSHOT
[ERROR]   +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.3.3:compile
[ERROR]     +-io.opentelemetry.instrumentation:opentelemetry-instrumentation-api-semconv:jar:1.33.5-alpha:compile
[ERROR]       +-io.opentelemetry:opentelemetry-api-incubator:jar:1.40.0-alpha:runtime
[ERROR]
[ERROR]
[ERROR] Dependency convergence error for io.opentelemetry:opentelemetry-api:jar:1.40.0 paths to dependency are:
[ERROR] +-com.example:demo:jar:0.0.1-SNAPSHOT
[ERROR]   +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.3.3:compile
[ERROR]     +-io.opentelemetry:opentelemetry-api:jar:1.40.0:compile
[ERROR] and
[ERROR] +-com.example:demo:jar:0.0.1-SNAPSHOT
[ERROR]   +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.3.3:compile
[ERROR]     +-io.opentelemetry:opentelemetry-sdk-common:jar:1.38.0:compile
[ERROR]       +-io.opentelemetry:opentelemetry-api:jar:1.38.0:compile
[ERROR] and
[ERROR] +-com.example:demo:jar:0.0.1-SNAPSHOT
[ERROR]   +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.3.3:compile
[ERROR]     +-io.opentelemetry:opentelemetry-sdk-trace:jar:1.38.0:compile
[ERROR]       +-io.opentelemetry:opentelemetry-api:jar:1.38.0:compile
[ERROR] and
[ERROR] +-com.example:demo:jar:0.0.1-SNAPSHOT
[ERROR]   +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.3.3:compile
[ERROR]     +-io.opentelemetry:opentelemetry-sdk:jar:1.38.0:compile
[ERROR]       +-io.opentelemetry:opentelemetry-api:jar:1.38.0:compile
[ERROR] and
[ERROR] +-com.example:demo:jar:0.0.1-SNAPSHOT
[ERROR]   +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.3.3:compile
[ERROR]     +-io.opentelemetry:opentelemetry-sdk:jar:1.38.0:compile
[ERROR]       +-io.opentelemetry:opentelemetry-sdk-metrics:jar:1.38.0:compile
[ERROR]         +-io.opentelemetry:opentelemetry-api:jar:1.38.0:compile
[ERROR] and
[ERROR] +-com.example:demo:jar:0.0.1-SNAPSHOT
[ERROR]   +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.3.3:compile
[ERROR]     +-io.opentelemetry:opentelemetry-sdk:jar:1.38.0:compile
[ERROR]       +-io.opentelemetry:opentelemetry-sdk-logs:jar:1.38.0:compile
[ERROR]         +-io.opentelemetry:opentelemetry-api:jar:1.38.0:compile
[ERROR] and
[ERROR] +-com.example:demo:jar:0.0.1-SNAPSHOT
[ERROR]   +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.3.3:compile
[ERROR]     +-io.opentelemetry.instrumentation:opentelemetry-instrumentation-api-semconv:jar:1.33.5-alpha:compile
[ERROR]       +-io.opentelemetry.instrumentation:opentelemetry-instrumentation-api:jar:1.33.5:compile
[ERROR]         +-io.opentelemetry:opentelemetry-api:jar:1.40.0:compile
[ERROR] and
[ERROR] +-com.example:demo:jar:0.0.1-SNAPSHOT
[ERROR]   +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.3.3:compile
[ERROR]     +-io.opentelemetry:opentelemetry-extension-trace-propagators:jar:1.38.0:compile
[ERROR]       +-io.opentelemetry:opentelemetry-api:jar:1.38.0:compile
[ERROR]

Same problem with versions 1.3.0 and 1.3.2.
1.3.1 seem to be fine because the transitive dependencies luckily match.

In the case of the 1.3.3, both errors can be fixed by upgrading all dependencies under io.opentelemetry group to 1.40.0 instead of 1.38.0 (the opentelemetery-api dep is already at 1.40.0 but the others are not).

shakuzen commented 1 month ago

Something is definitely weird and unexpected. We'll take a closer look.

shakuzen commented 1 month ago

We declare a dependency on version 1.38.0 of OTel for our 1.3.x line - see, for example, https://central.sonatype.com/artifact/io.micrometer/micrometer-tracing-bridge-otel/1.3.0 (same for all 1.3.x patch releases as well).

This wasn't true, at least not for all patch releases. Our 1.3.2 pom declares a dependency on opentelemetry-api 1.39.0 and our 1.3.3 declares a dependency on opentelemetry-api 1.40.0. That's unexpected - we intended for our build to manage the version to 1.38.0.

Also, opentelemetry-instrumentation-api is bumping minor versions of opentelemetry-api in patch releases, which is unexpected from our end (although I don't know if it was intentional on the part of the maintainers of that artifact).

jonatan-ivanov commented 1 month ago

A few notes:

  1. opentelemetry-bom-alpha contains opentelemetry-bom so we don't need to depend on the latter (it's an easy fix but does not help solving this concrete issue).

  2. According to ./gradlew :micrometer-tracing-bridge-otel:dependencies, io.opentelemetry:opentelemetry-api gets bumped: 1.38.0 -> 1.40.0:

    +--- io.opentelemetry:opentelemetry-bom:1.38.0
|    +--- io.opentelemetry:opentelemetry-api:1.38.0 -> 1.40.0 (c)

    because io.opentelemetry.instrumentation:opentelemetry-instrumentation-api-semconv that we also define as a dependency (1.33.5-alpha):

    +--- io.opentelemetry.instrumentation:opentelemetry-instrumentation-api-semconv -> 1.33.5-alpha
|    +--- io.opentelemetry.semconv:opentelemetry-semconv:1.23.1-alpha
|    \--- io.opentelemetry.instrumentation:opentelemetry-instrumentation-api:1.33.5
|         \--- io.opentelemetry:opentelemetry-api:1.40.0 (*)

  3. There is an instrumentation BOM (io.opentelemetry.instrumentation:opentelemetry-instrumentation-bom-alpha) that is not part of opentelemetry-bom-alpha but it does contain it so it seems we could get away by defining one BOM: io.opentelemetry.instrumentation:opentelemetry-instrumentation-bom-alpha, the BOM hierarchy seems to be this:

    io.opentelemetry.instrumentation:opentelemetry-instrumentation-bom-alpha
io.opentelemetry.instrumentation:opentelemetry-instrumentation-bom
opentelemetry-bom-alpha
opentelemetry-bom

    But as Tommy mentioned above, patch version bumps of opentelemetry-instrumentation-bom-alpha will result in minor version bumps of io.opentelemetry:opentelemetry-bom-alpha:

  4. It seems opentelemetry-instrumentation-bom-alpha also got a major version bump (1.x -> 2.x), not sure when the 1.x line will EOL but right now the latest 1.x is a bit behind on io.opentelemetry:opentelemetry-bom-alpha comparing to 2.x.

A single BOM and following the same version strategy would be nice from OTel, until then I think we can use opentelemetry-instrumentation-bom-alpha and don't upgrade it throughout our patch releases. This might not be a huge issue though since OTel rarely issues patch releases for opentelemetry-bom-alpha or opentelemetry-bom.

jonatan-ivanov commented 1 month ago

fyi: https://github.com/micrometer-metrics/tracing/pull/823

jonatan-ivanov commented 1 month ago

This should be fixed by https://github.com/micrometer-metrics/tracing/pull/823. @rayusagi Could you please check if the latest snapshot looks ok on your side?

shakuzen commented 1 month ago

We should make sure to call out in the release notes that the opentelemetry version has been effectively downgraded to 1.31.0 in this patch version to match our dependency version policy and what it was in our 1.2.0 release.

rayusagi commented 1 month ago

@jonatan-ivanov my sample project above builds when using the latest 1.3.4-SNAPSHOT (it does not when using 1.3.3) so that's okay. The dependency tree is that in that case is as follows :

com.example:demo:jar:0.0.1-SNAPSHOT
   io.micrometer:micrometer-tracing-bridge-otel:jar:1.3.4-SNAPSHOT:compile
      io.micrometer:micrometer-tracing:jar:1.3.4-SNAPSHOT:compile
         io.micrometer:context-propagation:jar:1.1.1:compile
         aopalliance:aopalliance:jar:1.0:compile
      org.slf4j:slf4j-api:jar:1.7.36:compile
      io.micrometer:micrometer-observation:jar:1.13.3:compile
         io.micrometer:micrometer-commons:jar:1.13.3:compile
      io.opentelemetry:opentelemetry-api:jar:1.38.0:compile
         io.opentelemetry:opentelemetry-context:jar:1.38.0:compile
      io.opentelemetry.semconv:opentelemetry-semconv:jar:1.23.1-alpha:compile
      io.opentelemetry:opentelemetry-sdk-common:jar:1.38.0:compile
      io.opentelemetry:opentelemetry-sdk-trace:jar:1.38.0:compile
      io.opentelemetry:opentelemetry-sdk:jar:1.38.0:compile
         io.opentelemetry:opentelemetry-sdk-metrics:jar:1.38.0:compile
         io.opentelemetry:opentelemetry-sdk-logs:jar:1.38.0:compile
      io.opentelemetry.instrumentation:opentelemetry-instrumentation-api-semconv:jar:1.33.3-alpha:compile
         io.opentelemetry.instrumentation:opentelemetry-instrumentation-api:jar:1.33.3:compile
         io.opentelemetry:opentelemetry-api-incubator:jar:1.38.0-alpha:runtime
      io.opentelemetry:opentelemetry-extension-trace-propagators:jar:1.38.0:compile

So thank you for that! I still have convergence issues on my real module using Spring Boot because they set the opentelemetry version to 1.37.0 but that's another problem on their part. One I hope they will notice when using your future 1.3.4 release on their next release.

However, when I try to use the latest 1.2.10-SNAPSHOT (which I personally don't need, I was just curious), I still get convergence issues :

Dependency convergence error for io.opentelemetry:opentelemetry-api:jar:1.31.0 paths to dependency are:
+-com.example:demo:jar:0.0.1-SNAPSHOT
  +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.2.10-SNAPSHOT:compile
    +-io.opentelemetry:opentelemetry-api:jar:1.31.0:compile
and
+-com.example:demo:jar:0.0.1-SNAPSHOT
  +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.2.10-SNAPSHOT:compile
    +-io.opentelemetry.semconv:opentelemetry-semconv:jar:1.21.0-alpha:compile
      +-io.opentelemetry:opentelemetry-api:jar:1.29.0:runtime
and
+-com.example:demo:jar:0.0.1-SNAPSHOT
  +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.2.10-SNAPSHOT:compile
    +-io.opentelemetry:opentelemetry-sdk-common:jar:1.31.0:compile
      +-io.opentelemetry:opentelemetry-api:jar:1.31.0:compile
and
+-com.example:demo:jar:0.0.1-SNAPSHOT
  +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.2.10-SNAPSHOT:compile
    +-io.opentelemetry:opentelemetry-sdk-trace:jar:1.31.0:compile
      +-io.opentelemetry:opentelemetry-api:jar:1.31.0:compile
and
+-com.example:demo:jar:0.0.1-SNAPSHOT
  +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.2.10-SNAPSHOT:compile
    +-io.opentelemetry:opentelemetry-sdk:jar:1.31.0:compile
      +-io.opentelemetry:opentelemetry-api:jar:1.31.0:compile
and
+-com.example:demo:jar:0.0.1-SNAPSHOT
  +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.2.10-SNAPSHOT:compile
    +-io.opentelemetry:opentelemetry-sdk:jar:1.31.0:compile
      +-io.opentelemetry:opentelemetry-sdk-metrics:jar:1.31.0:compile
        +-io.opentelemetry:opentelemetry-api:jar:1.31.0:compile
and
+-com.example:demo:jar:0.0.1-SNAPSHOT
  +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.2.10-SNAPSHOT:compile
    +-io.opentelemetry:opentelemetry-sdk:jar:1.31.0:compile
      +-io.opentelemetry:opentelemetry-sdk-logs:jar:1.31.0:compile
        +-io.opentelemetry:opentelemetry-api:jar:1.31.0:compile
and
+-com.example:demo:jar:0.0.1-SNAPSHOT
  +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.2.10-SNAPSHOT:compile
    +-io.opentelemetry:opentelemetry-sdk:jar:1.31.0:compile
      +-io.opentelemetry:opentelemetry-sdk-logs:jar:1.31.0:compile
        +-io.opentelemetry:opentelemetry-api-events:jar:1.31.0-alpha:runtime
          +-io.opentelemetry:opentelemetry-api:jar:1.31.0:runtime
and
+-com.example:demo:jar:0.0.1-SNAPSHOT
  +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.2.10-SNAPSHOT:compile
    +-io.opentelemetry.instrumentation:opentelemetry-instrumentation-api-semconv:jar:1.31.0-alpha:compile
      +-io.opentelemetry.instrumentation:opentelemetry-instrumentation-api:jar:1.31.0:compile
        +-io.opentelemetry:opentelemetry-api:jar:1.31.0:compile
and
+-com.example:demo:jar:0.0.1-SNAPSHOT
  +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.2.10-SNAPSHOT:compile
    +-io.opentelemetry.instrumentation:opentelemetry-instrumentation-api-semconv:jar:1.31.0-alpha:compile
      +-io.opentelemetry:opentelemetry-extension-incubator:jar:1.31.0-alpha:runtime
        +-io.opentelemetry:opentelemetry-api:jar:1.31.0:runtime
and
+-com.example:demo:jar:0.0.1-SNAPSHOT
  +-io.micrometer:micrometer-tracing-bridge-otel:jar:1.2.10-SNAPSHOT:compile
    +-io.opentelemetry:opentelemetry-extension-trace-propagators:jar:1.31.0:compile
      +-io.opentelemetry:opentelemetry-api:jar:1.31.0:compile

The culprit seems to be io.opentelemetry.semconv:opentelemetry-semconv used in 1.21.0-alpha that brings with it a divergent 1.29.0 version of opentelemetry-api.

jonatan-ivanov commented 1 month ago

Thanks for checking! I'm afraid that's an issue we cannot fix on Micrometer side, it seems OTel conflicts within its own versions, here are the details with link to the poms:

Micrometer depends on io.opentelemetry.instrumentation:opentelemetry-instrumentation-bom-alpha:1.31.0-alpha which brings in:

But then, as you pointed out, io.opentelemetry.semconv:opentelemetry-semconv:1.21.0-alpha brings in io.opentelemetry:opentelemetry-api:1.29.0.

Micrometer has no control over this, but build tools should be able to overcome this issue by resolving io.opentelemetry:opentelemetry-api:1.31.0 but upgrading to Micrometer Tracing 1.3.x is also an option.

I will inform OTel devs about this problem, please feel free to open an issue in opentelemetry-java.