shanman190
commented
6 months ago Closed erdi closed 6 months ago
shanman190
commented
6 months ago 
timtebeek
commented
6 months ago Hi @erdi ; Interesting use case there! I can see how you'd like to speed up a run across 1300 projects ; we'll of course see what we can do with the Gradle plugin to make that possible, and any help towards that is appreciated.
But even if we get parallel to work there might still be challenges with a project of that size, as building up the lossless semantic tree take up a lot of time. You'd also have to ensure that it all fits into memory, and build up that LST for every recipe run, both no easy feat.
Have you ever looked at Moderne.io? There we serialize the LST to disk, and allow you to speed up and do multiple recipe runs using that same model. Could be interesting given the size of what you have to work with, and the likely challenges involved.
erdi
commented
6 months ago Related issue #183.
Oh, sorry @shanman190, I should have checked other issues in the tracker first, this clearly is a duplicate, feel free to close.
erdi
commented
6 months ago Yeah, I've noticed it's very, very slow, @timtebeek. In the end I got fed up and just changed the javax to jakarta imports using search and replace in the IDE. Bit of a shame because OpenRewrite looks like a cool tool, I always wanted to use it and had somewhat bad experience when I tried.
timtebeek
commented
6 months ago Yes for a bit of context: we're building up a detailed model of your code including types and formatting, such that we can do exact replacements only where we are sure it's safe to do so (but it does take some time). That allows us to do library and framework migrations far beyond what a text based replacement can do. Plus we can compose together recipes, such that our Java 21 migration recipe for instance adopts SequencedCollection methods, introduces pattern matching for instance of and text blocks, among a couple hundred other changes going back to Java 5.
We've found OpenRewrite to work very well for individual projects. But as you've found when scaling up to monorepos or many-multi-repositories you might want to look at Moderne to help you there. The recipes are the same and open source for both, but the capabilities of the platform stretch far beyond what you can do with the OSS Maven and Gradle plugins.
We can indeed track progress towards parallel runs in https://github.com/openrewrite/rewrite-gradle-plugin/issues/183, and in the mean time hope to see around on other issues or our Slack
What version of OpenRewrite are you using?
Using
org.openrewrite.rewriteGradle plugin in version 6.8.4 and whatever OpenRewrite version that is using by default. Applyingorg.openrewrite.java.migrate.jakarta.JavaxMigrationToJakartarecipe fromorg.openrewrite.recipe:rewrite-migrate-java:2.9.1.How are you running OpenRewrite?
./gradlew rewriteRunon a large monorepo with 1300+ projects in the build with parallel execution enabled.What is the smallest, simplest way to reproduce the problem?
Run
rewriteRunon a repo with multiple subprojects all havingorg.openrewrite.rewriteplugin applied to them with--parallelswitch to enable parallel execution.What did you expect to see?
No build errors.
What did you see instead?
Context
As far as I understand, implementations of
java.lang.ClassLoader#loadClass(java.lang.String, boolean)need to be thread safe, that's why there is a synchronized block ongetClassLoadingLock(name)wrapping the whole method in the default implementation.What is the full stack trace of any errors you encountered?