Maven ci friendly version numbers

delanym commented 2 years ago

Im using

 <plugin>
   <groupId>org.openrewrite.maven</groupId>
   <artifactId>rewrite-maven-plugin</artifactId>
   <version>4.19.0</version>
   <configuration>
     <activeRecipes>
       <recipe>org.openrewrite.java.logging.slf4j.Log4jToSlf4j</recipe>
     </activeRecipes>
   </configuration>
   <dependencies>
     <dependency>
       <groupId>org.openrewrite.recipe</groupId>
       <artifactId>rewrite-logging-frameworks</artifactId>
       <version>1.2.0</version>

But I get Caused by: java.net.URISyntaxException: Illegal character in path at index 74

The new parser is unable to handle the ${changelist} property inherent in all my pom.xml files. See https://maven.apache.org/maven-ci-friendly.html

I set this property in \<root>/.mvn/maven.config with the line -Dchangelist=-SNAPSHOT

sambsnyd commented 2 years ago

Thanks for letting us know about this @delanym , we will look into it. I appreciate the link to maven's ci-friendly documentation

tkvangorder commented 2 years ago

I've loaded the source of the warning (this was actually due to having a property in a repository URL). It was not resolving these correctly when downloading imported boms.

delanym commented 2 years ago

This is still not working in rewrite-maven-plugin:4.20.0 The changelist property as well as other properties that get set in a Maven extension are picked up by Maven but not by this plugin. As I mentioned before it seems like you're stepping over the model and trying to do it yourself.

tkvangorder commented 2 years ago

Hi @delanym . I may need some help getting a reproducible use case.

You are correct, rewrite has its own semantic model for Maven build files.

All of the properties that are defined in maven.config are added to the system properties. I have confirmed those properties are indeed being resolved as part of rewrite's model:

My example:

./mvn/maven.config:

-Dsnapshot=https://oss.sonatype.org/content/repositories/snapshots/ -Dchangelist=-SNAPSHOT -Dtest=123

./pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.openrewrite</groupId>
    <artifactId>eclipse-talk</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <kotlin.code.style>official</kotlin.code.style>
        <kotlin.version>1.5.10</kotlin.version>
        <kotlin.compiler.jvmTarget>11</kotlin.compiler.jvmTarget>
        <maven-compiler-plugin.version>3.8.1</maven-compiler-plugin.version>
        <maven.compiler.showWarnings>true</maven.compiler.showWarnings>
        <java.version>11</java.version>
        <rewrite.version>7.19.0${changelist}</rewrite.version>
    </properties>

    <build>
        <plugins>
            <plugin>
                <groupId>com.github.ekryd.echo-maven-plugin</groupId>
                <artifactId>echo-maven-plugin</artifactId>
                <version>1.2.0</version>
                <executions>
                    <execution>
                        <phase>validate</phase>
                        <goals>
                            <goal>echo</goal>
                        </goals>
                        <configuration>
                            <message>snapshot = ${snapshot}, test = ${test}, changelist=${changelist},</message>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.openrewrite.maven</groupId>
                <artifactId>rewrite-maven-plugin</artifactId>
                <version>4.20.0</version>
                <configuration>
                    <activeRecipes>
                            <recipe>org.openrewrite.java.format.AutoFormat</recipe>
                    </activeRecipes>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>${maven-compiler-plugin.version}</version>
                <configuration>
                    <release>11</release>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.jetbrains.kotlin</groupId>
                <artifactId>kotlin-maven-plugin</artifactId>
                <version>${kotlin.version}</version>
                <executions>
                    <execution>
                        <id>compile</id>
                        <phase>process-sources</phase>
                        <goals>
                            <goal>compile</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>test-compile</id>
                        <phase>test-compile</phase>
                        <goals>
                            <goal>test-compile</goal>
                        </goals>
                        <configuration>
                            <sourceDirs>
                                <sourceDir>${project.basedir}/src/test/kotlin</sourceDir>
                                <sourceDir>${project.basedir}/src/test/java</sourceDir>
                            </sourceDirs>
                        </configuration>

                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.22.2</version>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>2.22.2</version>
            </plugin>
        </plugins>
    </build>
    <dependencies>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.18.20</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>5.6.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>5.6.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.openrewrite</groupId>
            <artifactId>rewrite-maven</artifactId>
            <version>${rewrite.version}</version>
        </dependency>
        <dependency>
            <groupId>org.openrewrite</groupId>
            <artifactId>rewrite-java</artifactId>
            <version>${rewrite.version}</version>
        </dependency>
        <dependency>
            <groupId>org.openrewrite</groupId>
            <artifactId>rewrite-java-11</artifactId>
            <version>${rewrite.version}</version>
        </dependency>
        <dependency>
            <groupId>org.openrewrite</groupId>
            <artifactId>rewrite-test</artifactId>
            <version>${rewrite.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.assertj</groupId>
            <artifactId>assertj-core</artifactId>
            <version>3.19.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.jetbrains.kotlin</groupId>
            <artifactId>kotlin-test-junit5</artifactId>
            <version>${kotlin.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.jetbrains.kotlin</groupId>
            <artifactId>kotlin-stdlib-jdk8</artifactId>
            <version>${kotlin.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.jetbrains.kotlin</groupId>
            <artifactId>kotlin-reflect</artifactId>
            <version>${kotlin.version}</version>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <pluginRepositories>
      <pluginRepository>
        <id>sonatype-snapshots</id>
        <name>snapshots</name>
        <url>https://oss.sonatype.org/content/repositories/snapshots/</url>
        <snapshots><enabled>true</enabled></snapshots>
      </pluginRepository>
    </pluginRepositories>
    <repositories>
      <repository>
        <id>sonatype-snapshots</id>
        <name>snapshots</name>
        <url>${snapshot}</url>
        <snapshots><enabled>true</enabled></snapshots>
      </repository>
    </repositories>
</project>

tkvangorder commented 2 years ago

Note that in the above example, the changeset property is used to suffix the rewrite-version property and that the snapshot property is used in the repositories section of the pom.

This runs correctly for me:

 ~/work/example  mvn rewrite:dryRun
[INFO] Scanning for projects...
[INFO]
[INFO] --------------------< org.openrewrite:eclipse-talk >--------------------
[INFO] Building eclipse-talk 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] >>> rewrite-maven-plugin:4.20.0:dryRun (default-cli) > process-test-classes @ eclipse-talk >>>
Downloading from sonatype-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/openrewrite/rewrite-maven/7.19.0-SNAPSHOT/maven-metadata.xml
Downloaded from sonatype-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/openrewrite/rewrite-maven/7.19.0-SNAPSHOT/maven-metadata.xml (2.0 kB at 3.1 kB/s)
Downloading from sonatype-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/openrewrite/rewrite-maven/7.19.0-SNAPSHOT/rewrite-maven-7.19.0-20220225.193518-44.pom
Downloaded from sonatype-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/openrewrite/rewrite-maven/7.19.0-SNAPSHOT/rewrite-maven-7.19.0-20220225.193518-44.pom (9.1 kB at 51 kB/s)
Downloading from sonatype-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/openrewrite/rewrite-xml/7.19.0-SNAPSHOT/maven-metadata.xml
Downloaded from sonatype-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/openrewrite/rewrite-xml/7.19.0-SNAPSHOT/maven-metadata.xml (2.0 kB at 12 kB/s)
Downloading from sonatype-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/openrewrite/rewrite-xml/7.19.0-SNAPSHOT/rewrite-xml-7.19.0-20220225.193518-44.pom
Downloaded from sonatype-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/openrewrite/rewrite-xml/7.19.0-SNAPSHOT/rewrite-xml-7.19.0-20220225.193518-44.pom (5.2 kB at 29 kB/s)
Downloading from sonatype-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/openrewrite/rewrite-core/7.19.0-SNAPSHOT/maven-metadata.xml
...
[INFO]
[INFO] --- echo-maven-plugin:1.2.0:echo (default) @ eclipse-talk ---
[INFO] snapshot = https://oss.sonatype.org/content/repositories/snapshots/, test = 123, changelist=-SNAPSHOT,
[INFO]

...

[INFO] --- rewrite-maven-plugin:4.20.0:dryRun (default-cli) @ eclipse-talk ---
[INFO] Using active recipe(s) [org.openrewrite.java.format.AutoFormat]
[INFO] Using active styles(s) []
[INFO] Validating active recipes...
[INFO] Parsing Java main files...
[INFO] Parsing Java test files...
[INFO] Running recipe(s)...
[WARNING] These recipes would make changes to src/main/java/org/example/RemoveGetDataPointsParamRecipe.java:
[WARNING]     org.openrewrite.java.format.AutoFormat
[WARNING] These recipes would make changes to src/main/java/org/example/MissingOverrideAnnotation.java:
[WARNING]     org.openrewrite.java.format.AutoFormat
[WARNING] Patch file available:
[WARNING]     /Users/tyler.vangorder/work/example/target/site/rewrite/rewrite.patch
[WARNING] Run 'mvn rewrite:run' to apply the recipes.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  50.788 s
[INFO] Finished at: 2022-02-28T12:03:23-08:00
[INFO] ------------------------------------------------------------------------

tkvangorder commented 2 years ago

It is possible that you are using the changeset property in a place where we are not correctly doing property substiution.

Can you:

Provide your maven.config?
Provide all places within your pom.xml where you are using the ${changeset} property?

TY!

delanym commented 2 years ago

Hi @tkvangorder . "changelist" not "changeset" - Im not able to reproduce the error yet, but here is a project to demonstrate the warnings from unresolved properties https://github.com/delanym/rewrite-me

I suspect if you sort that out the rest will follow.

The error I get on my main project is

Caused by: java.net.URISyntaxException: Illegal character in path at index 73: file:/home/sol/.m2/repository/com/example/example-maven-super/2.2.14${changelist}/example-maven-super-2.2.14${changelist}.pom
    at java.net.URI$Parser.fail (URI.java:2873)
    at java.net.URI$Parser.checkChars (URI.java:3046)
    at java.net.URI$Parser.parseHierarchical (URI.java:3130)
    at java.net.URI$Parser.parse (URI.java:3078)
    at java.net.URI.<init> (URI.java:588)
    at java.net.URI.create (URI.java:850)
    at org.openrewrite.maven.internal.MavenPomDownloader.download (MavenPomDownloader.java:228)
    at org.openrewrite.maven.tree.ResolvedPom$Resolver.resolveParentPropertiesAndRepositoriesRecursively (ResolvedPom.java:322)
    at org.openrewrite.maven.tree.ResolvedPom$Resolver.resolveParentPropertiesAndRepositoriesRecursively (ResolvedPom.java:333)
    at org.openrewrite.maven.tree.ResolvedPom$Resolver.resolveParentsRecursively (ResolvedPom.java:295)
    at org.openrewrite.maven.tree.ResolvedPom$Resolver.resolve (ResolvedPom.java:288)
    at org.openrewrite.maven.tree.ResolvedPom.resolve (ResolvedPom.java:133)
    at org.openrewrite.maven.UpdateMavenModel.updateResult (UpdateMavenModel.java:126)
    at org.openrewrite.maven.UpdateMavenModel.lambda$visitDocument$3 (UpdateMavenModel.java:100)
    at org.openrewrite.marker.Markers.lambda$computeByType$0 (Markers.java:97)
    at org.openrewrite.internal.ListUtils.lambda$map$0 (ListUtils.java:149)
    at org.openrewrite.internal.ListUtils.map (ListUtils.java:131)
    at org.openrewrite.internal.ListUtils.map (ListUtils.java:149)
    at org.openrewrite.marker.Markers.computeByType (Markers.java:92)
    at org.openrewrite.maven.UpdateMavenModel.visitDocument (UpdateMavenModel.java:39)
    at org.openrewrite.xml.tree.Xml$Document.acceptXml (Xml.java:114)
    at org.openrewrite.xml.tree.Xml.accept (Xml.java:51)
    at org.openrewrite.TreeVisitor.visit (TreeVisitor.java:199)
    at org.openrewrite.TreeVisitor.visit (TreeVisitor.java:218)
    at org.openrewrite.TreeVisitor.visit (TreeVisitor.java:218)
    at org.openrewrite.TreeVisitor.visitNonNull (TreeVisitor.java:159)
    at org.openrewrite.maven.AddDependency$1.visitDocument (AddDependency.java:200)
    at org.openrewrite.maven.AddDependency$1.visitDocument (AddDependency.java:175)
    at org.openrewrite.xml.tree.Xml$Document.acceptXml (Xml.java:114)
    at org.openrewrite.xml.tree.Xml.accept (Xml.java:51)
    at org.openrewrite.TreeVisitor.visit (TreeVisitor.java:199)
    at org.openrewrite.maven.AddDependency.lambda$visit$3 (AddDependency.java:202)
    at java.util.Optional.map (Optional.java:215)
    at org.openrewrite.maven.AddDependency.lambda$visit$4 (AddDependency.java:175)
    at org.openrewrite.internal.ListUtils.lambda$map$0 (ListUtils.java:149)
    at org.openrewrite.internal.ListUtils.map (ListUtils.java:131)
    at org.openrewrite.internal.ListUtils.map (ListUtils.java:149)
    at org.openrewrite.maven.AddDependency.visit (AddDependency.java:174)
    at org.openrewrite.RecipeScheduler.scheduleVisit (RecipeScheduler.java:244)
    at org.openrewrite.RecipeScheduler.scheduleVisit (RecipeScheduler.java:287)
    at org.openrewrite.RecipeScheduler.scheduleVisit (RecipeScheduler.java:287)
    at org.openrewrite.RecipeScheduler.scheduleRun (RecipeScheduler.java:79)
    at org.openrewrite.Recipe.run (Recipe.java:247)
    at org.openrewrite.Recipe.run (Recipe.java:238)
    at org.openrewrite.Recipe.run (Recipe.java:234)
    at org.openrewrite.maven.AbstractRewriteMojo.listResults (AbstractRewriteMojo.java:179)
    at org.openrewrite.maven.RewriteDryRunMojo.execute (RewriteDryRunMojo.java:51)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:196)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:186)
    at java.util.concurrent.FutureTask.run (FutureTask.java:266)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)
    at java.util.concurrent.FutureTask.run (FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
    at java.lang.Thread.run (Thread.java:748)

openrewrite / rewrite

Maven ci friendly version numbers #1399