Missing transitive dependencies

[SeaJaredCode]

I have dependencies that seem to generate the exact same code for a shared transitive dependency when listed individually, but when listed together they "cancel" each other out and further transitive deps are missed.

Working with only these two dependencies:

dependencies:
  io.grpc:
    grpc-netty-shaded:
      lang: java
      version: 1.14.0

  com.google.firebase:
    firebase-admin:
      lang: java
      version: 6.6.0

After generating, for the shared transitive dependency io.grpc:grpc-core, I get this generated (note the lack of transitive deps):

java_library(
    name = "grpc_core",
    exports = [
        "//external:jar/io/grpc/grpc_core"
    ],
    visibility = [
        "//deps_test/jvm:__subpackages__"
    ]
)

When either of the two dependencies are commented out, I get the correct grpc_core:

java_library(
    name = "grpc_core",
    exports = [
        "//deps_test/jvm/com/google/code/findbugs:jsr305",
        "//deps_test/jvm/com/google/code/gson:gson",
        "//deps_test/jvm/com/google/errorprone:error_prone_annotations",
        "//deps_test/jvm/com/google/guava:guava",
        "//deps_test/jvm/io/opencensus:opencensus_api",
        "//deps_test/jvm/io/opencensus:opencensus_contrib_grpc_metrics",
        "//external:jar/io/grpc/grpc_core",
        ":grpc_context"
    ],
    visibility = [
        "//deps_test/jvm:__subpackages__"
    ]
)

I was initially suspicious of the netty dependency listing grpc as runtime, but some other combinations I found seem to not be troubled by that (like using io.grpc:grpc-protobuf).

Any ideas what might cause this or how to identify when it happens? The workaround is fairly easy (i.e. explicitly specifying grpc-core), but this issue shows up at runtime and is neither easy to diagnose nor be aware of ahead of time.

[johnynek]

Thanks for the bug report.

It is currently a bit challenging to debug these kinds of issues. My first instinct would be to inspect the poms of both of the explicit dependencies. The next thing I would do is try using both the aether resolver and the coursier resolver and see if there are any differences there. If there are, the difference is likely in the code that builds the full graph (Resolvers) vs the code that picks the versions: (Normalizer).

There are two main portions of code likely to be impacted:

1. Normalizer: https://github.com/johnynek/bazel-deps/blob/master/src/scala/com/github/johnynek/bazel_deps/Normalizer.scala this code takes the entire transitive graph of all versions, and normalizes each artifact down to a single version or errors.

2. CoursierResolver: https://github.com/johnynek/bazel-deps/blob/master/src/scala/com/github/johnynek/bazel_deps/CoursierResolver.scala the code that uses coursier to generate a full transitive graph of artifacts.

If you have time to try running with both resolvers, and report back, that would be helpful. See the help on options here: https://github.com/johnynek/bazel-deps#options

[SeaJaredCode]

aether does indeed resolve it "correctly." I'm not sure I know now whether it is correct, but it is what I want!

My guess is that it comes down to one "branch" of dependency that requires exact and another that specifies a minimum that's higher. These can't both be true, so I'm not sure if correct behavior is an error or bumping the version to 1.14.0. Both resolvers do actually bump the version, it's just that the coursier one doesn't include transitive dependencies.

I was able to more narrowly define the issue with two library each with a direct dependency on grpc-core and it has to do with conflicting versions.

dependencies:
  io.grpc:
    grpc-auth:
      lang: java
      version: 1.13.1
    grpc-protobuf:
      lang: java
      version: 1.14.0

The poms are seem straightforward (grpc-auth, grpc-protobuf), with not too many dependencies, but their reference to grpc-core differ on version (and ranges):

grpc-auth:

<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-core</artifactId>
<version>[1.13.1]</version>
<scope>compile</scope>
</dependency>

grpc-protobuf:

<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-core</artifactId>
<version>1.14.0</version>
<scope>compile</scope>
</dependency>

So, it seems that it is specific to the Coursier resolver, but it also seems like it is when it normalizes the version! I haven't yet dug into the code, but I will follow your pointers and see where it leads me. Thx!

[johnynek]

We could try bumping coursier and see if that helps.

I also don’t really know what [0.13.1] means and I wonder if that syntax is playing a role here.

[SeaJaredCode]

According to oracle, it means "exactly".