search

VirtusLab / scala-steward-repos

24 stars 145 forks source link

PRs not being sent in for various projects #98

Open ckipp01 opened 1 year ago

ckipp01 commented 1 year ago

I'm not sure if the logs are public or not since I don't see reference to them anywhere, but I noticed today that no PRs are being sent in to https://github.com/DavidGregory084/mill-tpolecat even though it's listed in https://github.com/VirtusLab/scala-steward-repos/blob/main/repos-github.md. Could this be checked to see if it's erroring out on that repo for some reason? If so, pasting the error would be super helpful as I'm happy to look into it.

tgodzik commented 1 year ago

Thanks for reporting! I am having issues getting full logs for Azure, it seems I need to configure logging workspace? This is a bit too complicated, it actually gives you less logs than github actions.

tgodzik commented 1 year ago

Ok, it all seems complicated to get logs :|

Could you maybe try running the ScalaSteward mill plugin locally to see if it breaks? Maybe it's reproducible locally?

ckipp01 commented 1 year ago

Sure, running the extractDeps task which is what is called in the Mill plugin seems to work fine:

❯ mill --import ivy:org.scala-steward:scala-steward-mill-plugin_2.13:0.15.0 org.scalasteward.mill.plugin.StewardPlugin/extractDeps
No mill version specified.
You should provide a version via '.mill-version' file or --mill-version option.
Using mill version 0.10.8
[11/11] org.scalasteward.mill.plugin.StewardPlugin.extractDeps

So no idea really without actually seeing the logs.

ckipp01 commented 1 year ago

Alright, so I actually just saw that this is happening on another repo. https://github.com/com-lihaoyi/scalatags isn't getting any prs sent in and it's on the repo.md.

tgodzik commented 1 year ago

Ok, I finally managed to get a follow on container logs. Let's see if I find the issue.

tgodzik commented 1 year ago

Yesterday I only got a part of it and it hasn't yielded any results, I am rerunning the whole job and should get the full logs.

tgodzik commented 1 year ago

Got it! I think I should actually find all errors and let the people know. That's for scalatags

2022-12-21 09:25:07,660 ERROR Steward com-lihaoyi/scalatags failed
java.io.IOException: 'sbt -Dsbt.color=false -Dsbt.log.noformat=true -Dsbt.supershell=false ;+ stewardDependencies;reload plugins;stewardDependencies' exited with code 1
[info] Loading global plugins from /root/.sbt/1.0/plugins
[info] Compiling 1 Scala source to /root/.sbt/1.0/plugins/target/scala-2.12/sbt-1.0/classes ...
[error] /root/.sbt/1.0/plugins/StewardPlugin.scala:67:15: not found: value csrConfiguration
[error]               csrConfiguration.value.authenticationByRepositoryId.toList
[error]               ^
[error] /root/.sbt/1.0/plugins/StewardPlugin.scala:69:56: value headers is not a member of Any
[error]             (headerKey, headerValue) <- authentication.headers
[error]                                                        ^
[error] /root/.sbt/1.0/plugins/StewardPlugin.scala:70:35: type mismatch;
[error]  found   : Any
[error]  required: String
[error]           } yield Resolver.Header(headerKey, headerValue)
[error]                                   ^
[error] /root/.sbt/1.0/plugins/StewardPlugin.scala:70:46: type mismatch;
[error]  found   : Any
[error]  required: String
[error]           } yield Resolver.Header(headerKey, headerValue)
[error]                                              ^
[error] four errors found
[error] (Compile / compileIncremental) Compilation failed
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? 
    at org.scalasteward.core.io.process$.$anonfun$slurp$9(process.scala:54)
    at org.scalasteward.core.io.process$.$anonfun$slurp$9$adapted(process.scala:50)
    at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
    at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
    at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
    at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
    at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
    at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
    at modify @ fs2.internal.Scope.close(Scope.scala:262)
    at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
    at rethrow$extension @ fs2.Compiler$Target.$anonfun$compile$1(Compiler.scala:157)
    at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
    at flatMap @ fs2.Pull$.$anonfun$compile$21(Pull.scala:1209)
    at update @ fs2.internal.Scope.releaseChildScope(Scope.scala:224)
    at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
tgodzik commented 1 year ago

Interestingly for mill-tpolcat it's just this:

2022-12-21 09:34:18,213 INFO  ──────────── Steward DavidGregory084/mill-tpolecat ────────────
2022-12-21 09:34:18,213 INFO  Check cache of DavidGregory084/mill-tpolecat
2022-12-21 09:34:18,697 INFO  Find updates for DavidGregory084/mill-tpolecat
2022-12-21 09:34:18,698 INFO  Found 0 updates
2022-12-21 09:34:18,698 INFO  DavidGregory084/mill-tpolecat is up-to-date
2022-12-21 09:34:18,699 INFO  ──────────── Total time: Steward DavidGregory084/mill-tpolecat: 486ms ────────────
2022-12-21 09:34:18,699 INFO  ──────────── Steward deanwampler/programming-scala-book-code-examples ────────────

I can send you the full logs if you want.

Packed size is 150 mb and unpacked 10x more :fearful:

tgodzik commented 1 year ago

Full list of failing repos to check is:

Edit: started removing some: https://github.com/VirtusLab/scala-steward-repos/pull/136

ckipp01 commented 1 year ago

Ah super helpful, thanks for sharing that. Regarding scalatags and a lot of the other com-lihaoyi ones, the logs make it seem like it's trying to run sbt on them, but looking at them they do indeed have a build.sbt file, only the docs, which stinks. I don't really know if there is a way to force a build tool, but this at least explains why all those are failing.

tgodzik commented 1 year ago

Ah super helpful, thanks for sharing that. Regarding scalatags and a lot of the other com-lihaoyi ones, the logs make it seem like it's trying to run sbt on them, but looking at them they do indeed have a build.sbt file, only the docs, which stinks. I don't really know if there is a way to force a build tool, but this at least explains why all those are failing.

We might need to try and run both :thinking: but that needs a change in the steward I guess. I updated the list, will try tackle that slowly. We can open issues in the repos that are failing.

ckipp01 commented 1 year ago

Ah weird that it failed on Mill. Actually, do you mind dropping me the entire logs thing? I can try to take care of the mill ones.

mtomko commented 1 year ago

One of those repos (https://github.com/broadinstitute/poolq) "belongs" to me and I think it should work. I mean, we use it all the time. Also, it doesn't use Mill - it has a regular build.sbt, although its CI job runs a make target that does some extra integration tests as well. It's CI build passed the last time it ran: https://github.com/broadinstitute/poolq/actions/workflows/ci.yml

Anything I can do to help? I'm not sure how to see these logs.

tgodzik commented 1 year ago

@mtomko I haven't put them anywhere since they're huge and not sure if it's not a security breach to put it up opnely :sweat_smile:

For your project it seems it's failing because of:

[info] set current project to poolq3 (in build file:/opt/workspace/repos/broadinstitute/poolq/)
[error] java.lang.RuntimeException: unable to locate a valid GitHub token from Or(Environment(GITHUB_TOKEN),GitConfig(github.token))
[error]     at scala.sys.package$.error(package.scala:30)
[error]     at sbtghpackages.GitHubPackagesPlugin$.$anonfun$authenticationSettings$2(GitHubPackagesPlugin.scala:53)
[error]     at scala.Function1.$anonfun$compose$1(Function1.scala:49)
[error]     at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:62)
[error]     at sbt.std.Transform$$anon$4.work(Transform.scala:68)
[error]     at sbt.Execute.$anonfun$submit$2(Execute.scala:282)
[error]     at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:23)
[error]     at sbt.Execute.work(Execute.scala:291)
[error]     at sbt.Execute.$anonfun$submit$1(Execute.scala:282)
[error]     at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
[error]     at sbt.CompletionService$$anon$2.call(CompletionService.scala:64)
[error]     at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error]     at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
[error]     at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error]     at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[error]     at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[error]     at java.base/java.lang.Thread.run(Thread.java:829)
[error] (credentials) unable to locate a valid GitHub token from Or(Environment(GITHUB_TOKEN),GitConfig(github.token))

GITHUB_TOKEN will not be available in scala steward job and it needs to be able to run sbt.

julienrf commented 1 year ago

Got it! I think I should actually find all errors and let the people know. That's for scalatags

2022-12-21 09:25:07,660 ERROR Steward com-lihaoyi/scalatags failed
java.io.IOException: 'sbt -Dsbt.color=false -Dsbt.log.noformat=true -Dsbt.supershell=false ;+ stewardDependencies;reload plugins;stewardDependencies' exited with code 1
[info] Loading global plugins from /root/.sbt/1.0/plugins
[info] Compiling 1 Scala source to /root/.sbt/1.0/plugins/target/scala-2.12/sbt-1.0/classes ...
[error] /root/.sbt/1.0/plugins/StewardPlugin.scala:67:15: not found: value csrConfiguration
[error]               csrConfiguration.value.authenticationByRepositoryId.toList
[error]               ^
[error] /root/.sbt/1.0/plugins/StewardPlugin.scala:69:56: value headers is not a member of Any
[error]             (headerKey, headerValue) <- authentication.headers
[error]                                                        ^
[error] /root/.sbt/1.0/plugins/StewardPlugin.scala:70:35: type mismatch;
[error]  found   : Any
[error]  required: String
[error]           } yield Resolver.Header(headerKey, headerValue)
[error]                                   ^
[error] /root/.sbt/1.0/plugins/StewardPlugin.scala:70:46: type mismatch;
[error]  found   : Any
[error]  required: String
[error]           } yield Resolver.Header(headerKey, headerValue)
[error]                                              ^
[error] four errors found
[error] (Compile / compileIncremental) Compilation failed
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? 
  at org.scalasteward.core.io.process$.$anonfun$slurp$9(process.scala:54)
  at org.scalasteward.core.io.process$.$anonfun$slurp$9$adapted(process.scala:50)
  at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
  at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
  at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
  at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
  at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
  at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
  at modify @ fs2.internal.Scope.close(Scope.scala:262)
  at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
  at rethrow$extension @ fs2.Compiler$Target.$anonfun$compile$1(Compiler.scala:157)
  at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
  at flatMap @ fs2.Pull$.$anonfun$compile$21(Pull.scala:1209)
  at update @ fs2.internal.Scope.releaseChildScope(Scope.scala:224)
  at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)

That one is related to a recent change: https://github.com/scala-steward-org/scala-steward/issues/2847

mtomko commented 1 year ago

@tgodzik thanks, I probably won't get to it before January but I'm sure I can find a way to work around that; the build should only need those for publishing so maybe I can stub it out or delay initialization

ckipp01 commented 1 year ago

That one is related to a recent change: https://github.com/scala-steward-org/scala-steward/issues/2847

Yes and no, we actually spoke about this on Discord last night, but the larger issue is that the build still has a build.sbt file in it only to generate docs using the very old scalatext. That should really be addressed because there is no incentive to update the sbt version there, since it's barely doing anything and the build is using Mill.

tgodzik commented 1 year ago

I am ticking off all the repos that either can be removed or are notified. Will try to slowly churn through all of them

tgodzik commented 1 year ago

Btw. @ckipp01 I see some jobs fail with:

[1/1] show 
Exception in thread "main" java.lang.NoSuchMethodError: 'void mainargs.arg.<init>(java.lang.String, char, java.lang.String, boolean)'
    at org.scalasteward.mill.plugin.StewardPlugin$.$anonfun$millDiscover$1(StewardPlugin.scala:86)
    at org.scalasteward.mill.plugin.StewardPlugin$.millDiscover$lzycompute(StewardPlugin.scala:86)
    at org.scalasteward.mill.plugin.StewardPlugin$.millDiscover(StewardPlugin.scala:86)

is that because of using an older mill version?

ckipp01 commented 1 year ago

Btw. @ckipp01 I see some jobs fail with:

[1/1] show 
Exception in thread "main" java.lang.NoSuchMethodError: 'void mainargs.arg.<init>(java.lang.String, char, java.lang.String, boolean)'
  at org.scalasteward.mill.plugin.StewardPlugin$.$anonfun$millDiscover$1(StewardPlugin.scala:86)
  at org.scalasteward.mill.plugin.StewardPlugin$.millDiscover$lzycompute(StewardPlugin.scala:86)
  at org.scalasteward.mill.plugin.StewardPlugin$.millDiscover(StewardPlugin.scala:86)

is that because of using an older mill version?

Yea, actually many of the Mill jobs should have greatly been improved since https://github.com/scala-steward-org/scala-steward/pull/2821 was merged. Previously there was no detection of Mill version, so it was sort of best effort if it'd actually work or not. Now that there is an external plugin it correctly identifies the version of Mill and therefore brings in the right plugin. Is that error using the newest steward?

tgodzik commented 1 year ago

I think we always run the newest steward :thinking: since we use the lastest image fthomas/scala-steward:latest

ckipp01 commented 1 year ago

I think we always run the newest steward 🤔 since we use the lastest image fthomas/scala-steward:latest

Weird, do you have an example of a repo that it's failing on. I'll try to reproduce it locally.

tgodzik commented 1 year ago

That's for example coursier/dependency

ckipp01 commented 1 year ago

That's for example coursier/dependency

Alright, this should fix it https://github.com/coursier/dependency/pull/16.

tgodzik commented 1 year ago

Thanks! Alex should be back next week to take a look.

mkurz commented 1 year ago

You can tick three checkboxes in the above list after merging #180.

Can someone please paste me the log for playframework/play-samples so I can take a look why its failing? Thanks!

AlexITC commented 1 year ago

@mkurz I added play-samples to my hosted steward and it throws this (be aware my instance is old):

  2023-02-28 14:42:06,965 INFO  Get dependencies in play-java-akka-cluster-example from sbt
  2023-02-28 14:42:39,857 INFO  Get dependencies in play-java-chatroom-example from sbt
  2023-02-28 14:43:01,810 INFO  Get dependencies in play-java-compile-di-example from sbt
  2023-02-28 14:43:25,897 INFO  Get dependencies in play-java-dagger2-example from sbt
  2023-02-28 14:43:48,158 INFO  Get dependencies in play-java-ebean-example from sbt
  2023-02-28 14:44:13,835 INFO  Get dependencies in play-java-fileupload-example from sbt
  2023-02-28 14:44:35,051 INFO  Get dependencies in play-java-forms-example from sbt
  2023-02-28 14:44:57,853 INFO  Get dependencies in play-java-grpc-example from sbt
  2023-02-28 14:45:25,934 INFO  Get dependencies in play-java-grpc-example/docs from sbt
  2023-02-28 14:45:51,017 ERROR Steward playframework/play-samples failed
  org.scalasteward.core.io.process$ProcessFailedException: '"SBT_OPTS=-Xmx2048m -Xss8m -XX:MaxMetaspaceSize=512m" sbt -Dsbt.color=false -Dsbt.log.noformat=true -Dsbt.supershell=false ;+ stewardDependencies;reload plugins;stewardDependencies' exited with code 1.
  [info] [launcher] getting org.scala-sbt sbt 1.8.2  (this may take some time)...
  [info] Updated file /home/runner/scala-steward/workspace/repos/playframework/play-samples/play-java-grpc-example/docs/project/build.properties: set sbt.version to 1.8.2
  [info] welcome to sbt 1.8.2 (Eclipse Adoptium Java 11.0.18)
  [info] loading project definition from /home/runner/scala-steward/workspace/repos/playframework/play-samples/play-java-grpc-example/docs/project/project
  [info] compiling 1 Scala source to /home/runner/scala-steward/workspace/repos/playframework/play-samples/play-java-grpc-example/docs/project/project/target/scala-2.12/sbt-1.0/classes ...
  [info] Non-compiled module 'compiler-bridge_2.12' for Scala 2.12.17. Compiling...
  [info]   Compilation completed in 10.061s.
  [info] done compiling
  [info] loading project definition from /home/runner/scala-steward/workspace/repos/playframework/play-samples/play-java-grpc-example/docs/project
  [info] compiling 1 Scala source to /home/runner/scala-steward/workspace/repos/playframework/play-samples/play-java-grpc-example/docs/project/target/scala-2.12/sbt-1.0/classes ...
  [info] done compiling
  /home/runner/scala-steward/workspace/repos/playframework/play-samples/play-java-grpc-example/docs/build.sbt:1: error: not found: value paradoxTheme
  paradoxTheme := Some(builtinParadoxTheme("generic"))
  ^
  /home/runner/scala-steward/workspace/repos/playframework/play-samples/play-java-grpc-example/docs/build.sbt:1: error: not found: value builtinParadoxTheme
  paradoxTheme := Some(builtinParadoxTheme("generic"))
                       ^
  Error:  Type error in expression
  [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r)
tgodzik commented 1 year ago

Looks like it's the same in the container logs:

2022-12-21 11:24:46,766 ERROR Steward playframework/play-samples failed
java.io.IOException: 'sbt -Dsbt.color=false -Dsbt.log.noformat=true -Dsbt.supershell=false ;+ stewardDependencies;reload plugins;stewardDependencies' exited with code 1
[info] Updated file /opt/workspace/repos/playframework/play-samples/play-java-grpc-example/docs/project/build.properties: set sbt.version to 1.8.0
[info] welcome to sbt 1.8.0 (Eclipse Adoptium Java 11.0.16.1)
[info] loading global plugins from /root/.sbt/1.0/plugins
[info] compiling 1 Scala source to /root/.sbt/1.0/plugins/target/scala-2.12/sbt-1.0/classes ...
[info] done compiling
[info] loading project definition from /opt/workspace/repos/playframework/play-samples/play-java-grpc-example/docs/project
/opt/workspace/repos/playframework/play-samples/play-java-grpc-example/docs/build.sbt:1: error: not found: value paradoxTheme
paradoxTheme := Some(builtinParadoxTheme("generic"))
^
/opt/workspace/repos/playframework/play-samples/play-java-grpc-example/docs/build.sbt:1: error: not found: value builtinParadoxTheme
paradoxTheme := Some(builtinParadoxTheme("generic"))
                     ^
[error] Type error in expression
[warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r)
mkurz commented 1 year ago

@AlexITC @tgodzik thanks! I configured and started Scala Steward locally (because I was working on https://github.com/scala-steward-org/scala-steward/pull/2995 anyway) and got exactly the same error. So I went ahead and fixed all problems until I was able to run scala-steward locally against the play-sample repo again:

Now it works! The logs:

Click to show the logs ```java ... 2023-03-01 08:11:38,081 INFO ──────────── Steward playframework/play-samples ──────────── 2023-03-01 08:11:38,081 INFO Check cache of playframework/play-samples 2023-03-01 08:11:38,807 INFO Clone playframework/play-samples 2023-03-01 08:11:40,909 INFO Refresh cache of playframework/play-samples 2023-03-01 08:11:40,949 INFO Parsed repo config { "commits" : { "message" : null }, "pullRequests" : { "frequency" : null, "grouping" : [ ], "includeMatchedLabels" : null }, "scalafmt" : { "runAfterUpgrading" : null }, "updates" : { "pin" : [ ], "allow" : [ ], "allowPreReleases" : [ ], "ignore" : [ ], "limit" : null, "fileExtensions" : null }, "postUpdateHooks" : null, "updatePullRequests" : null, "buildRoots" : [ "play-java-akka-cluster-example", "play-java-chatroom-example", "play-java-compile-di-example", "play-java-dagger2-example", "play-java-ebean-example", "play-java-fileupload-example", "play-java-forms-example", "play-java-grpc-example", "play-java-hello-world-tutorial", "play-java-jpa-example", "play-java-rest-api-example", "play-java-starter-example", "play-java-streaming-example", "play-java-telemetry-example", "play-java-websocket-example", "play-scala-anorm-example", "play-scala-chatroom-example", "play-scala-compile-di-example", "play-scala-fileupload-example", "play-scala-forms-example", "play-scala-grpc-example", "play-scala-hello-world-tutorial", "play-scala-isolated-slick-example", "play-scala-log4j2-example", "play-scala-macwire-di-example", "play-scala-rest-api-example", "play-scala-secure-session-example", "play-scala-slick-example", "play-scala-starter-example", "play-scala-streaming-example", "play-scala-telemetry-example", "play-scala-tls-example", "play-scala-websocket-example" ], "assignees" : [ ], "reviewers" : [ ], "dependencyOverrides" : [ ] } 2023-03-01 08:11:41,286 INFO Get dependencies in play-java-akka-cluster-example from sbt 2023-03-01 08:11:53,785 INFO Get dependencies in play-java-chatroom-example from sbt 2023-03-01 08:12:05,639 INFO Get dependencies in play-java-compile-di-example from sbt 2023-03-01 08:12:18,200 INFO Get dependencies in play-java-dagger2-example from sbt 2023-03-01 08:12:29,724 INFO Get dependencies in play-java-ebean-example from sbt 2023-03-01 08:12:42,396 INFO Get dependencies in play-java-fileupload-example from sbt 2023-03-01 08:12:53,667 INFO Get dependencies in play-java-forms-example from sbt 2023-03-01 08:13:05,993 INFO Get dependencies in play-java-grpc-example from sbt 2023-03-01 08:13:19,572 INFO Get dependencies in play-java-hello-world-tutorial from sbt 2023-03-01 08:13:31,418 INFO Get dependencies in play-java-jpa-example from sbt 2023-03-01 08:13:43,595 INFO Get dependencies in play-java-rest-api-example from sbt 2023-03-01 08:13:55,749 INFO Get dependencies in play-java-starter-example from sbt 2023-03-01 08:14:07,355 INFO Get dependencies in play-java-streaming-example from sbt 2023-03-01 08:14:19,642 INFO Get dependencies in play-java-telemetry-example from sbt 2023-03-01 08:14:31,713 INFO Get dependencies in play-java-websocket-example from sbt 2023-03-01 08:14:43,862 INFO Get dependencies in play-scala-anorm-example from sbt 2023-03-01 08:14:55,391 INFO Get dependencies in play-scala-chatroom-example from sbt 2023-03-01 08:15:07,303 INFO Get dependencies in play-scala-compile-di-example from sbt 2023-03-01 08:15:19,255 INFO Get dependencies in play-scala-fileupload-example from sbt 2023-03-01 08:15:31,344 INFO Get dependencies in play-scala-forms-example from sbt 2023-03-01 08:15:43,486 INFO Get dependencies in play-scala-grpc-example from sbt 2023-03-01 08:15:56,764 INFO Get dependencies in play-scala-hello-world-tutorial from sbt 2023-03-01 08:16:09,331 INFO Get dependencies in play-scala-isolated-slick-example from sbt 2023-03-01 08:16:23,062 INFO Get dependencies in play-scala-log4j2-example from sbt 2023-03-01 08:16:35,349 INFO Get dependencies in play-scala-macwire-di-example from sbt 2023-03-01 08:16:46,986 INFO Get dependencies in play-scala-rest-api-example from sbt 2023-03-01 08:17:00,000 INFO Get dependencies in play-scala-secure-session-example from sbt 2023-03-01 08:17:11,618 INFO Get dependencies in play-scala-slick-example from sbt 2023-03-01 08:17:24,678 INFO Get dependencies in play-scala-starter-example from sbt 2023-03-01 08:17:36,684 INFO Get dependencies in play-scala-streaming-example from sbt 2023-03-01 08:17:48,619 INFO Get dependencies in play-scala-telemetry-example from sbt 2023-03-01 08:18:00,736 INFO Get dependencies in play-scala-tls-example from sbt 2023-03-01 08:18:13,207 INFO Get dependencies in play-scala-websocket-example from sbt 2023-03-01 08:18:30,345 INFO Find updates for playframework/play-samples ... ```

I think you can tick the play-samples checkbox in above list now :wink: Also, is it actually possible that you now trigger a scala-steward run from the VirtusLab scala-steward instance against the playframework/play-samples repo? Or may I ask how often do you actually run the instance against a specific repository? (I know about the interval config, but still I would like to know how often scala-steward visits a repo if that setting is not defined)

Also I think it would be a good idea to somehow catch errors that occur when trying to run scala-steward and notifiy the repo owners about the problem, for example by just automatically open an issue by the scala-steward bot. What do you think? Would that be possible?

Thanks!

mkurz commented 1 year ago

Ah and btw, maybe give this issue another title? It's not just about mill. If the issue would have had a better title I wouldn't even had opened #179 first...

tgodzik commented 1 year ago

Ah and btw, maybe give this issue another title? It's not just about mill. If the issue would have had a better title I wouldn't even had opened https://github.com/VirtusLab/scala-steward-repos/issues/179 first...

Thanks! That's changed!

Also, is it actually possible that you now trigger a scala-steward run from the VirtusLab scala-steward instance against the playframework/play-samples repo? Or may I ask how often do you actually run the instance against a specific repository? (I know about the interval config, but still I would like to know how often scala-steward visits a repo if that setting is not defined)

It runs every couple of days since it's a heavy beast and we are rate limited by Github

Also I think it would be a good idea to somehow catch errors that occur when trying to run scala-steward and notifiy the repo owners about the problem, for example by just automatically open an issue by the scala-steward bot. What do you think? Would that be possible?

It should be possible to do inside Scala Steward itself as it has access to Github API already, though I worry it would create a lot of issues currently.

mkurz commented 1 year ago

It should be possible to do inside Scala Steward itself as it has access to Github API already, though I worry it would create a lot of issues currently.

I wouldn't care about creating lots of issues. IMHO it's more important to let people know that there a problems. People may just rely on the fact they once set up scala-steward and think they are good and forget about it (specially for dependencies that see new releases every couple of years).

mkurz commented 1 year ago

@tgodzik Can you please tick playframework/play-samples in the list above :wink: I am worried it will be removed one day otherwise... Thanks!

tgodzik commented 1 year ago

@tgodzik Can you please tick playframework/play-samples in the list above wink I am worried it will be removed one day otherwise... Thanks!

It shouldn't be removed, that was my mistake. I wanted only to remove archived/deleted repos. And it's ticked!

mkurz commented 1 year ago

@tgodzik A bit offtopic, but ScalaSteward v0.23.0 was just released, just curious: when do you usually upgrade you public instance? Asking because the release contains a fix our repos will benefit from. Thanks!

tgodzik commented 1 year ago

I think we always use the newest image, which has the newest steward.

tgodzik commented 1 year ago

This one I think https://hub.docker.com/r/fthomas/scala-steward/

SethTisue commented 10 months ago

Was this resolved...?

mtomko commented 10 months ago

Was this resolved...?

I can't speak to the larger issue; for the repository I'd commented on, there's not a clear path forward without more build engineering than I have time for, so if it's in the way, I can just submit a PR to remove it from Scala Steward's care.

tgodzik commented 10 months ago

I haven't had the time to look at all the repositories unfortunately, I was hoping that maybe people with issues on the active repositories would at least be able to see that there is an issue.

lefou commented 7 months ago

Just a short note, that Mill repo itself is not receiving any PRs for a very long time. It's on the repos-github.md list, of course.

tgodzik commented 7 months ago

I think the issue previously was that both sbt and mill was present. Is that still the case? I will try to run scala-steward and save all the logs I can.

lefou commented 7 months ago

I think the issue previously was that both sbt and mill was present. Is that still the case? I will try to run scala-steward and save all the logs I can.

This was never the case for Mill. We removed all sbt stuff long time before we used scala-steward. But it might be correct for other com-lihaoyi project, e.g. upickle, but interestingly, this one gets PRs.

fthomas commented 7 months ago

I just tried running my Scala Steward dev instance on com-lihaoyi/mill and it failed to extract dependencies from the build because its internal buffer was too small. Running Scala Steward with --max-buffer-size 32768 fixed it for me. We should probably increase the default value and/or maybe change https://github.com/scala-steward-org/mill-plugin so that it outputs fewer lines.

lefou commented 7 months ago

That's probably an issue with all larger projects. What exactly should we cut off? We could make the plugin aware of the buffer length, let it report back to scala-steward and steward needs to run it again. Some kind of paging. But I think this is too complicated. We should either increase the buffer or just support output to files.

sageserpent-open commented 3 weeks ago

The sageserpent-open/americium project was bitten by this too.

As an FYI to latecomers, the workaround used was to suppress writing to standard-out when running a very verbose set of tests (including a lot of noisy parameterised tests).

So if your SBT build forks tests, you could try setting Test / forkOptions to:

(Test / forkOptions).value
              .withOutputStrategy(
                OutputStrategy.CustomOutput(OutputStream.nullOutputStream)
              )

That works for Scalatest test suites, for JUnit5 via the SBT / Jupiter integration you would need something like:

    testOptions += Tests.Argument(jupiterTestFramework, "-q")

I imagine Mill etc have their own ways of doing the same thing.