aiyanbo / sbt-dependency-updates

⬆️ SBT plugin that can check Maven and Ivy repositories for dependency and plugin updates
Apache License 2.0
81 stars 10 forks source link

Too many connections error #23

Open jgulotta opened 2 years ago

jgulotta commented 2 years ago

I constantly get errors like this running dependencyUpdates

[error     ] com.typesafe.akka:akka-stream:2.6.19
Too many connections: 100
Too many connections: 100
Too many connections: 100
Too many connections: 100
Too many connections: 100
Too many connections: 100

[error     ] dev.zio:zio-stream:2.0.2
Too many connections: 100
Too many connections: 100
Too many connections: 100
Too many connections: 100
Too many connections: 100
Too many connections: 100
Too many connections: 100
Too many connections: 100
Too many connections: 100

I have less than 20 direct library/test/plugin dependencies for this project and have not been able to determine what causes the explosion in connections.

The message itself appears to come from async-http-client's TooManyConnectionsException and/or TooManyConnectionsPerHostException

I realize the information here is sparse but it's all I have been able to gather. I'm happy to gather more if you have ideas for things I can check.

If it's not possible to track down the real issue from such sparse info, maybe it's possible to alter how many requests are sent in parallel to stay under the limit, or maybe to issue backoff/retries for TooManyConnectionsException and TooManyConnectionsPerHostException?

aiyanbo commented 2 years ago

Fixed in 1.2.7

jgulotta commented 2 years ago

It does not appear solved for me

> dependencyUpdates
[info] Done checking.
[info] ---------------------------------- Global Plugins ----------------------------------
[success   ] org.jmotor.sbt:sbt-dependency-updates:1.2.7 √
[error     ] io.spray:sbt-revolver:0.9.1
Too many connections: 100
Too many connections: 100
Too many connections: 100
Too many connections: 100
Too many connections: 100
Too many connections: 100
Too many connections: 100
Too many connections: 100
Too many connections: 100
Too many connections: 100
jgulotta commented 2 years ago

Any chance this can be reopened?

aiyanbo commented 2 years ago

@jgulotta Can you show your dependencies and plugins?

jgulotta commented 2 years ago

This happens to me on all projects starting around 7 dependencies, usually after SBT is first launched.

I have a few global plugins

addSbtPlugin("org.jmotor.sbt" % "sbt-dependency-updates" % "1.2.7")

// conflicts with sbt-dependency-updates 1.2.7 because of transitive scala-xml 2
//addSbtPlugin("com.github.duhemm" % "sbt-errors-summary" % "0.6.5")

addSbtPlugin("io.spray" % "sbt-revolver" % "0.9.1")

We have an internal plugin at work that has a few plugin dependencies

addSbtPlugin("com.github.sbt" % "sbt-native-packager" % "1.9.11")
addSbtPlugin("com.github.sbt" % "sbt-release" % "1.1.0")

One project that shows the issue with relatively few and non-internal dependencies (other than the plugin) has

enablePlugins(InternalPlugin)

val slickVersion     = "3.3.3"

libraryDependencies += ("com.typesafe.slick" %% "slick" % slickVersion).cross(CrossVersion.for3Use2_13)
libraryDependencies += ("com.typesafe.slick" %% "slick-hikaricp" % slickVersion).cross(CrossVersion.for3Use2_13)
libraryDependencies += "com.zaxxer" % "HikariCP" % "3.4.5" % Provided

val awsSdkVersion = "2.17.193"
libraryDependencies += "software.amazon.awssdk" % "iam" % awsSdkVersion % Optional
libraryDependencies += "software.amazon.awssdk" % "rds" % awsSdkVersion % Optional
libraryDependencies += "software.amazon.awssdk" % "sts" % awsSdkVersion % Optional

libraryDependencies += "org.postgresql" % "postgresql" % "42.3.5" % Optional
libraryDependencies += "mysql" % "mysql-connector-java" % "8.0.29" % Optional
libraryDependencies += "org.mariadb.jdbc" % "mariadb-java-client" % "2.7.4" % Optional

If I remove the AWS dependencies, I can run the task without error. If I add even one back I start seeing connection exhaustion errors.