FEAT: Recipe discovery, management, updates and downloads

[mmoayyed]

What problem are you trying to solve?

I am experimenting with OpenRewrite to allow CAS deployers to upgrade their CAS installation and build from one version to another. I realize this is perhaps different from the more common scenario of rewriting code elements in a codebase, and the recipes we have are more about upgrading software rather than fixing bits of code here and there. The installations in fact hardly have any Java code in them and that should not matter.

So the intention is to build and collect a large number of recipes that would allow one to go from CAS X to X+1, and these recipes would know how to upgrade the build, Gradle version, etc to make that upgrade as smooth as possible. Often, it's just a matter of changing version numbers, but sometimes it can be more complicated especially if there are breaking changes in Gradle plugins. Here is one such recipe that is more on the simpler side of things.

So the question then becomes: how does one discover recipes? If I am on version X and want to go to X+1, where do I find the recipe that would do that, and how can I find and run that recipe with as little technical effort as possible?

Presently, recipes are maintained as physical YAML files, and the OpenRewrite plugin is slightly massaged to run a recipe on command:

rewrite {
    def activeRecipe = providers.systemProperty("activeRecipe").getOrNull()
    if (activeRecipe != null) {
        def file = project.getRootProject().file("openrewrite/${activeRecipe}.yml")
        configFile = file
    }
}

Recipe discovery using this mode is somewhat cumbersome. One would have to go find the YAML files, read and study them to figure out what their name is, and only then one can run those. If a recipe is not available directly in the repo, one would have to hunt around the docs, etc to figure out where the recipes are or perhaps do fancy things with a Gradle build to download and fetch the latest collection of recipes. This is doable but hard for a novice user.

Describe the solution you'd like

Allow the plugin to do this automatically; that is, given some kind of URL, it should be able to download recipes and put them in the right places for use. I don't know yet if the https://www.moderne.io/ platform has this ability, but one could imagine, (similar to Maven Central), there would be a repository of sorts where recipes can be found, stored, managed, and downloaded for all to use.

Then, it would perhaps be something like this:

rewrite {
    repository = "apereo/cas"
    def activeRecipe = providers.systemProperty("activeRecipe").getOrNull()
    if (activeRecipe != null) {
        def file = project.getRootProject().file("openrewrite/${activeRecipe}.yml")
        configFile = file
    }
}

...and:

./gradlew rewriteFetchRecipes

Or:

rewrite {
    repository = "https://somewhere.example.org/apereo/cas"
    def activeRecipe = providers.systemProperty("activeRecipe").getOrNull()
    if (activeRecipe != null) {
        def file = project.getRootProject().file("openrewrite/${activeRecipe}.yml")
        configFile = file
    }
}

...and:

./gradlew rewriteFetchRecipes

Have you considered any alternatives or workarounds?

Yes, we can modify the Gradle build with a custom task to download the latest versions of recipes from somewhere, unpack them, and put them in the right place for the plugin to find and activate them.

Are you interested in contributing this feature to OpenRewrite?

Sure!

[shanman190]

So this is already how the rewrite plugin works. It relies upon the dependency management ecosystem, more specifically for Gradle using the rewrite configuration to download jars that contain recipes and execute any named recipe via it's fully qualified name. Recipes can be both declarative or programmatic in this way.

You can take a look here on how the yaml recipes can be packed into the jar file and are automatically discovered by the plugin. Then it's just activating the desired recipe. https://github.com/openrewrite/rewrite-testing-frameworks/tree/main/src/main/resources/META-INF/rewrite

Here's an example that shows this in practice: https://docs.openrewrite.org/running-recipes/running-rewrite-on-a-gradle-project-without-modifying-the-build

Also a note, in the script that you shared, you can already set the JVM argument activeRecipe, activeRecipes, rewrite.activeRecipe, or rewrite.activeRecipes and the Gradle plugin will load the configured recipe. You don't actually need to do that part yourself.

Then for the use case that you described on a transitive set of version upgrades, that is how the Spring Boot recipes are organized. The first recipe of each new version is to apply the previous migration. This means that no matter what your original source version is, you'll acquire all of the transformations necessary to reach the desired version. You can see that here: https://github.com/openrewrite/rewrite-spring/blob/5af999be8cc3adb35443ead813610d4c31ba0891/src/main/resources/META-INF/rewrite/spring-boot-31.yml#L26

[shanman190]

There's also the rewriteDiscover task which can help you find recipes that exist on your classpath already as well.

[mmoayyed]

Outstanding, thank you so much @shanman190.