search

scala-steward-org / scala-steward

:robot: A bot that helps you keep your projects up-to-date
Apache License 2.0
1.15k stars 503 forks source link

scala-steward is incompatible with sbt-dependency-updates #178

Closed neko-kai closed 5 years ago

neko-kai commented 5 years ago

Abstract: When a ModuleID is written with version detached into another variable, scala-steward seems to be unable update the version:

val cats_effect_version = "1.1.0"
val cats_effect = "org.typelevel" %% "cats-effect" % cats_effect_version

Detail: See https://github.com/pshirshov/izumi-r2/blob/develop/sbt/sbt-izumi-deps/src/main/scala/com/github/pshirshov/izumi/sbt/deps/IzumiDeps.scala#L21

After importing the Izumi Project here, we've got two PRs from scala-steward: https://github.com/pshirshov/izumi-r2/pull/460 https://github.com/pshirshov/izumi-r2/pull/461 both only concern ModuleID's that were written all at once while the large IzumiDeps object above was not processed. (Note, there's another bad case in IzumiDeps - constructing ModuleIDs with .map - however, this case likely can't be handled sanely, so we'll just flattenize those.)

fthomas commented 5 years ago

I think the main reason why scala-steward didn't update any library versions is because it doesn't understood the output of sbt dependencyUpdates which comes from https://github.com/aiyanbo/sbt-dependency-updates but scala-steward expects output that is generated by https://github.com/rtimush/sbt-updates 's dependencyUpdates which it has installed as global plugin.

neko-kai commented 5 years ago

Hmm, is there a way to unambiguously refer to sbt-updates's key? Worst comes to worst, scala-steward can create an .sbt file in root that imports sbt-updates's Task and assigns a unique key to it before importing the project. I'm not sure, but It might be possible to do it globally, too.

fthomas commented 5 years ago

I tried your second suggestion but it doesn't work.

izumi-r2 (develop)> cat steward.sbt 
val stewardDependencyUpdates2 = taskKey[Unit]("")

stewardDependencyUpdates2 := {
  _root_.com.timushev.sbt.updates.UpdatesKeys.dependencyUpdates.value
}

sbt stewardDependencyUpdates2 still runs dependencyUpdates from sbt-dependency-updates.

fthomas commented 5 years ago

I'm out of ideas how to fix this on scala-steward's side. I think the simplest fix would be if sbt-dependency-updates uses a task name that is different from sbt-updates' task name.

fthomas commented 5 years ago

As last resort to "fix" this, I've asked to rename sbt-dependency-updates' task: https://github.com/aiyanbo/sbt-dependency-updates/issues/11

neko-kai commented 5 years ago

@fthomas Thanks for caring!

neko-kai commented 5 years ago

@fthomas we removed sbt-dependency-updates, however, new scala steward PRs still seem to be limited to SBT plugin updates, not library updates. (There is at least one candidate for scala-steward, zio to RC9)

fthomas commented 5 years ago

@kaishh Here is what I get when running sbt-updates' dependencyUpdates:

[info] Found 1 dependency update for izumi-r2
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for fundamentals-collections
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for fundamentals-functional
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for fundamentals-collections
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for fundamentals-functional
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for sbt-izumi-deps
[info]   org.scala-lang:scala-library:provided : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for fundamentals-platform
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for fundamentals-platform
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 5 dependency updates for sbt-izumi
[info]   com.github.gseitz:sbt-release         : 1.0.9    -> 1.0.11               
[info]   io.get-coursier:sbt-coursier          : 1.1.0-M8 -> 1.1.0-M14-4          
[info]   org.portable-scala:sbt-crossproject   : 0.6.0    -> 0.6.1                
[info]   org.scala-lang:scala-library:provided : 2.12.8                  -> 2.13.0
[info]   org.scoverage:sbt-scoverage           : 1.5.1                   -> 1.6.0 
[info] Found 1 dependency update for sbt-tests
[info]   org.scala-lang:scala-library:provided : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for fundamentals-bio
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for fundamentals-bio
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for idealingua-v1-runtime-rpc-c-sharp
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for idealingua-v1-model
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for fundamentals-json-circe
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for fundamentals-json-circe
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for idealingua-v1-runtime-rpc-typescript
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for idealingua-v1-model
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for idealingua-v1-runtime-rpc-go
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 2 dependency updates for fundamentals-reflection
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info]   org.scala-lang:scala-reflect : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for fundamentals-typesafe-config
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for idealingua-v1-runtime-rpc-scala
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for idealingua-v1-runtime-rpc-scala
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for logstage-api
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for distage-model
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for idealingua-v1-test-defs
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for idealingua-v1-core
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for idealingua-v1-core
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for idealingua-v1-transpilers
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for distage-roles-api
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for logstage-core
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for logstage-config
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for logstage-rendering-circe
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for distage-core
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for idealingua-v1-transpilers
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for logstage-adapter-slf4j
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for logstage-sink-slf4j
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for distage-config
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for idealingua-v1-compiler
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for distage-proxy-cglib
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for logstage-di
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for idealingua-v1-runtime-rpc-http4s
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for distage-roles
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for distage-plugins
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for microsite
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for distage-static
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0
[info] Found 1 dependency update for distage-testkit
[info]   org.scala-lang:scala-library : 2.12.8 -> 2.13.0

It doesn't list an update for zio and that is the reason Scala Steward hasn't created a PR for that.

fthomas commented 5 years ago

https://github.com/7mind/izumi/pull/569 seems to be the first non-plugin update by Scala Steward. :tada: The build failed I think because you're using one version number for circe and circe-derivation although these are two different projects with different release cycles. But that is not an issue with Scala Steward.

I think we can close this now.

neko-kai commented 5 years ago

@fthomas I think we can't close yet, because sbt-updates is clearly ignoring all Optional dependencies - i.e. cats and ZIO dependencies there. The only dependencies I get listed are non-Optional.

sbt-updates knows about ZIO-RC9 since there was a PR here - https://github.com/zio/interop-cats/pull/8

neko-kai commented 5 years ago

Actually, I didn't check yet if promoting to non-Optional gets dependencies listed... lemme do that right now

neko-kai commented 5 years ago

Ok, I'm wrong. Downgrading cats causes it to appear even if Optional:

[info] Found 3 dependency updates for fundamentals-bio
[info]   org.scala-lang:scala-library       : 2.12.8               -> 2.13.0
[info]   org.typelevel:cats-core:optional   : 2.0.0-M1 -> 2.0.0-M4          
[info]   org.typelevel:cats-effect:optional : 2.0.0-M1 -> 2.0.0-M4

It's only zio update 1.0.0-RC8-12 -> 1.0.0-RC9 that isn't detected... 🤔 Even though the same was upgrade was detected in https://github.com/zio/interop-cats/pull/8/files !

@fthomas Thank you! I guess you can close now.

fthomas commented 5 years ago

Is zio 1.0.0-RC9 available for all crossScalaVersions in your project? Maybe that is the reason sbt-updates does not report it.

neko-kai commented 5 years ago

Yeah, it's bound for both JS and JVM, same as cats:

lazy val fundamentalsBio = inFundamentals.as.cross(platforms)
  .dependsOn(fundamentalsFunctional)
  .settings(
    libraryDependencies ++= R.cats_all.map(_.cross(platformDepsCrossVersion.value) % Optional),
    libraryDependencies ++= Seq(R.zio_core).map(_.cross(platformDepsCrossVersion.value) % Optional),
  )
fthomas commented 5 years ago

I've no other idea why it isn't reported then. But as soon as sbt-updates reports it, Scala Steward will create a PR for it.

If you ever find out the reason why, please report back here. I'm curious what the problem here is.