SBT stuck at Resolving NormalizedArtifact for long time

[huayangdu]

Hi, we want to use this plugin and "com.lightbend.akka" %% "akka-dependencies" bom file to declare dependencies, we use Bom.apply like example. Declare some libs and override use bomDependencies.

But sbt will stuck at this log for long when apply settings and takes about 10 minutes to download jars (that BOM has a lot of libs declared)

Logs: Resolving NormalizedArtifact(com.lightbend.akka,akka-dependencies_2.13,24.05.0).

Seems code here is using IvyDependencyResolution to retrieve BOM file as module. But it will try to download all jars within it even if we don't need. And finally it just uses findLocalPomFile to get BOM original file.

I don't think it's necessary. Actually I change the resolver.retrieve for POM as intransitive, it shows much quicker.

resolver.retrieve(
        resolver.wrapDependencyInModule(moduleId.asModule().intransitive()),
        dummyDir,
        logger
      )

Can we officially support this ?

[molekyla]

Hi @huayangdu. Thank you for pointing out the issues. Feel free to raise a PR in case you want to add/improve something. From our side we will take a look at this improvement, but with no ETA for now.

[molekyla]

@huayangdu I've tried to reproduce the issue but it works for me. Depencencies.scala content:

case class AkkaDependencies(platformBom: Bom) {
  val dependencies: Seq[ModuleID] = Seq(
    "com.typesafe.akka" % "akka-actor_2.13" % platformBom
  )
}

build.sbt file content:

lazy val deps = Bom.read("com.lightbend.akka" % "akka-dependencies_2.13" % "23.05.4")(bom => AkkaDependencies(bom))

lazy val `sbt-bom-example-github` = project
  .in(file("."))
  .settings(deps)
  .settings(
    name := "sbt-bom-example-github",
    resolvers := Resolver.DefaultMavenRepository +: resolvers.value,
    libraryDependencies ++= deps.key.value.dependencies
  )

$ sbt package
[info] welcome to sbt 1.6.2 (Oracle Corporation Java 1.8.0_302)
[info] loading settings for project sbt-bom-example-github-build from plugins.sbt ...
[info] loading project definition from ...projects/sbt-bom-example-github/project
[warn] sbt-bom-1.0.16.jar no longer exists at .../.cache/coursier/v1/https/repo1.maven.org/maven2/com/here/platform/sbt-bom_2.12_1.0/1.0.16/sbt-bom-1.0.16.jar
[info] Updating 
https://repo1.maven.org/maven2/com/here/platform/sbt-bom_2.12_1.0/1.0.16/sbt-bom-1.0.16.pom
  100.0% [##########] 2.5 KiB (5.8 KiB / s)
[info] Resolved  dependencies
[info] Updating 
https://repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.12.15/scala-compiler-2.12.15.pom
  100.0% [##########] 2.4 KiB (31.8 KiB / s)
https://repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.12.15/scala-reflect-2.12.15.pom
  100.0% [##########] 1.8 KiB (27.7 KiB / s)
[info] Resolved  dependencies
[info] Fetching artifacts of 
https://repo1.maven.org/maven2/com/here/platform/sbt-bom_2.12_1.0/1.0.16/sbt-bom-1.0.16.jar
  100.0% [##########] 45.0 KiB (283.3 KiB / s)
https://repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.12.15/scala-compiler-2.12.15.jar
  100.0% [##########] 10.5 MiB (21.0 MiB / s)
https://repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.12.15/scala-reflect-2.12.15.jar
  100.0% [##########] 3.5 MiB (4.9 MiB / s)
[info] Fetched artifacts of 
[info] loading settings for project sbt-bom-example-github from build.sbt ...
[info] set current project to sbt-bom-example-github (in build file:/home/ovyshniak/work/projects/sbt-bom-example-github/)
[warn] akka-actor_2.13-2.8.5.jar no longer exists at .../.cache/coursier/v1/https/repo1.maven.org/maven2/com/typesafe/akka/akka-actor_2.13/2.8.5/akka-actor_2.13-2.8.5.jar
[info] Updating 
https://repo1.maven.org/maven2/com/typesafe/akka/akka-actor_2.13/2.8.5/akka-actor_2.13-2.8.5.pom
  100.0% [##########] 2.3 KiB (38.8 KiB / s)
https://repo1.maven.org/maven2/org/scala-lang/modules/scala-java8-compat_2.13/1.0.0/scala-java8-compat_2.13-1.0.0.pom
  100.0% [##########] 2.4 KiB (52.4 KiB / s)
[info] Resolved  dependencies
[info] Fetching artifacts of 
https://repo1.maven.org/maven2/org/scala-lang/modules/scala-java8-compat_2.13/1.0.0/scala-java8-compat_2.13-1.0.0.jar
  100.0% [##########] 622.6 KiB (7.9 MiB / s)
https://repo1.maven.org/maven2/com/typesafe/akka/akka-actor_2.13/2.8.5/akka-actor_2.13-2.8.5.jar
  100.0% [##########] 3.6 MiB (12.1 MiB / s)
[info] Fetched artifacts of 
[success] Total time: 1 s, completed Nov 8, 2024 5:09:01 PM

The build is not stuck and downloaded only used dependecies. Could you please provide minimal example to reproduce the issue?

[huayangdu]

Hey @molekyla thanks, our example is almost the same as yours, but the thing is when your cache doesn't contain this dependencies bom at first, it will get this log in this code Resolving NormalizedArtifact(com.lightbend.akka,akka-dependencies_2.13,24.05.0 If there is no local pom file, it will download all dependencies by retrieve which we stuck at, although it's related to network. But seems it only need pom file. If there is, it will like your example. I reproduce by removing .ivy2/cache/com.lightbend.akka/akka-dependencies_2.13 folder.

From your log, seems I haven't seen this ?

[huayangdu]

Feel free to raise a PR in case you want to add/improve something.

I just add code here, it's resolved by intransitive resolver.wrapDependencyInModule(moduleId.asModule().intransitive())

[huayangdu]

  lazy val akkaBom = Bom("com.lightbend.akka" %% "akka-dependencies" % '24.05.0')
  def akkaDependencies(bom: Bom) = {
    Seq(
      "com.typesafe.akka" %% "akka-cluster-sharding" % bom,
      "com.lightbend.akka" %% "akka-diagnostics" % bom,
      "com.lightbend.akka.grpc" %% "akka-grpc-runtime" % bom,
      "com.lightbend.akka.management" %% "akka-lease-kubernetes" % bom % Runtime,
      "com.lightbend.akka.management" %% "akka-management-cluster-http" % bom % Runtime,
      "com.lightbend.akka.management" %% "akka-management-cluster-bootstrap" % bom % Runtime,
      "com.lightbend.akka.management" %% "akka-management-loglevels-logback" % bom % Runtime,
      "com.typesafe.akka" %% "akka-cluster-metrics" % bom % Runtime,
      "com.typesafe.akka" %% "akka-serialization-jackson" % bom % Runtime,
      "com.typesafe.akka" %% "akka-slf4j" % bom % Runtime,
      "com.typesafe.akka" %% "akka-http-testkit" % bom % Test,
      "com.typesafe.akka" %% "akka-stream-testkit" % bom % "test,it",
    )
  }
  project.inxxxx.settings(
    akkaBom,
    libraryDependencies ++= akkaDependencies(akkaBom.key.value),
    dependencyOverrides ++= akkaBom.key.value.bomDependencies
  )

our example like this

[molekyla]

Hi @huayangdu. Could you check new 1.0.17 version? The issue should be fixed there.

[huayangdu]

OK, thanks!