search

openrewrite / rewrite

Automated mass refactoring of source code.
https://docs.openrewrite.org
Apache License 2.0
2.23k stars 332 forks source link

Groovy parsing Exception while reading jenkins file - running custom recipe #4116

Open SDamasani opened 7 months ago

SDamasani commented 7 months ago

What version of OpenRewrite are you using?

I am using

How are you running OpenRewrite?

I am using the Maven plugin, and my project is a single module project.

<plugin>
    <groupId>org.openrewrite.maven</groupId>
    <artifactId>rewrite-maven-plugin</artifactId>
    <version>5.17.1</version>
    <configuration>
        <activeRecipes>
            <recipe>net.test.sample.UpdateJenkinsRecipe</recipe>
        </activeRecipes>
    </configuration>
    <dependencies>
        <dependency>
            <groupId>net.test.sample</groupId>
            <artifactId>jenkinsrecipe</artifactId>
            <version>1.0.0-SNAPSHOT</version>
        </dependency>
    </dependencies>
</plugin>

What is the smallest, simplest way to reproduce the problem?

please unit test with this below content which is from from jenkins file - you shoule be able to reproduce the problem.

#!groovy
def getEndBuild() {
    { steps , config , domain ->
        new net.one.gtu.jenkins.helper.ConsoleLogger(steps).logInfo("Intentionally exiting the build early.  Please" +
                " disregard subsequent errors; they do not represent real concerns/issues");
        throw new org.jenkinsci.plugins.workflow.steps.FlowInterruptedException(hudson.model.Result.SUCCESS);

    }
}
def buildPipeline1() {
    pipelineRunner {
        yml = 'jenkins_old.yml'
        endBuild = getEndBuild();
    }
}

created customVisitor which extend GroovyVisitor and i'm trying to read jenkinsfile and update some static content back to the file.Below is the myvisitor code .the error is occurring during the Parse, not during my custom visit.

@Override
public TreeVisitor<?, ExecutionContext> getVisitor() {
    log.info("I'm in the recipe for jenkins file update");
    return Preconditions.check(new FindSourceFiles("Jenkinsfile"), new GroovyVisitor<ExecutionContext>() {

        @Override
        public G.CompilationUnit visitCompilationUnit(G.CompilationUnit cu, ExecutionContext ctx) {
}
)};
}

The issue is happening in below block of code inside the sourceBefore method 

 @Override
  public void visitBlockStatement(BlockStatement block) {
      Space fmt = EMPTY;
      Object parent = nodeCursor.getParentOrThrow().getValue();
      if (!(parent instanceof ClosureExpression)) {
          fmt = sourceBefore("{");
      }

What did you expect to see?

Expected to see Parsing successful , the error is occurring during the Parse, not during my custom visit. Which implies that there's an issue either in the source file I'm trying to parse, or, in OpenRewrite's groovy-parsing logic. so i added the content from source file and this file is being used my teams accross my firm

What did you see instead?

org.openrewrite.groovy.GroovyParsingException: Failed to parse Jenkinsfile at cursor position 592. The next 10 characters in the original source are `{
       `
  org.openrewrite.groovy.GroovyParserVisitor.visit(GroovyParserVisitor.java:174)

What is the full stack trace of any errors you encountered?

org.openrewrite.groovy.GroovyParsingException: Failed to parse Jenkinsfile at cursor position 592. The next 10 characters in the original source are `{
       `
  org.openrewrite.groovy.GroovyParserVisitor.visit(GroovyParserVisitor.java:174)

During debug found out that inside this closure after domain -> the code is trying to get the nextSourceBefore using delim '{' which is avaialble only in the next method which is buildPipeline1(). so that's where the parsing issue happening.

  { steps , config , domain ->
        **new net.jpmchase.gti.jules.helper.ConsoleLogger(steps).**logInfo**("Intentionally exiting the build early.  Please" +
                " disregard subsequent errors; they do not represent real concerns/issues");
    } 

def buildPipeline1()** {
    pipelineRunner {
        yml = 'jenkins_old.yml'
        endBuild = getEndBuild();
    }
}
SDamasani commented 7 months ago

Adding more context. in the JenkinsFile if this is the last method it works .

def pvtClosure() {
    { steps, domain, config ->
        String url = "https://${config.greenRoute}.${config.domain}/actuator/health"
        def response = steps.httpRequest([url: url,
                                          validResponseCodes: '200'])
        steps.echo response.content
    }
}

if i add one more below method to the file as shown below it fails . even if i copy paste the above working method and rename the method name it still fails during parsing. 

 def jenkinsIntegration() { 
    { steps, domain, config ->
     steps.echo("######### Invoking Jira Integration Closure ########")
    }
  }

Adding detailed message and stacktrace 

java.lang.IllegalArgumentException: Unable to parse method call

stacktrace:

0 = {StackTraceElement@16114} "org.openrewrite.groovy.GroovyParserVisitor$RewriteGroovyVisitor.visitMethodCallExpression(GroovyParserVisitor.java:1510)"
1 = {StackTraceElement@16115} "org.codehaus.groovy.ast.expr.MethodCallExpression.visit(MethodCallExpression.java:76)"
2 = {StackTraceElement@16116} "org.codehaus.groovy.ast.CodeVisitorSupport.visitExpressionStatement(CodeVisitorSupport.java:117)"
3 = {StackTraceElement@16117} "org.openrewrite.groovy.GroovyParserVisitor$RewriteGroovyVisitor.visitExpressionStatement(GroovyParserVisitor.java:1275)"
4 = {StackTraceElement@16118} "org.codehaus.groovy.ast.stmt.ExpressionStatement.visit(ExpressionStatement.java:40)"
5 = {StackTraceElement@16119} "org.openrewrite.groovy.GroovyParserVisitor$RewriteGroovyVisitor.visit(GroovyParserVisitor.java:561)"
6 = {StackTraceElement@16120} "org.openrewrite.groovy.GroovyParserVisitor$RewriteGroovyVisitor.visitBlockStatement(GroovyParserVisitor.java:887)"
7 = {StackTraceElement@16121} "org.codehaus.groovy.ast.stmt.BlockStatement.visit(BlockStatement.java:69)"
8 = {StackTraceElement@16122} "org.openrewrite.groovy.GroovyParserVisitor$RewriteGroovyVisitor.visit(GroovyParserVisitor.java:561)"
9 = {StackTraceElement@16123} "org.openrewrite.groovy.GroovyParserVisitor$RewriteGroovyVisitor.visitClosureExpression(GroovyParserVisitor.java:1079)"
10 = {StackTraceElement@16124} "org.codehaus.groovy.ast.expr.ClosureExpression.visit(ClosureExpression.java:46)"
11 = {StackTraceElement@16125} "org.codehaus.groovy.ast.CodeVisitorSupport.visitExpressionStatement(CodeVisitorSupport.java:117)"
12 = {StackTraceElement@16126} "org.openrewrite.groovy.GroovyParserVisitor$RewriteGroovyVisitor.visitExpressionStatement(GroovyParserVisitor.java:1275)"
13 = {StackTraceElement@16127} "org.codehaus.groovy.ast.stmt.ExpressionStatement.visit(ExpressionStatement.java:40)"
14 = {StackTraceElement@16128} "org.openrewrite.groovy.GroovyParserVisitor$RewriteGroovyVisitor.visit(GroovyParserVisitor.java:561)"
15 = {StackTraceElement@16129} "org.openrewrite.groovy.GroovyParserVisitor$RewriteGroovyVisitor.visitBlockStatement(GroovyParserVisitor.java:887)"
16 = {StackTraceElement@16130} "org.codehaus.groovy.ast.stmt.BlockStatement.visit(BlockStatement.java:69)"
17 = {StackTraceElement@16131} "org.openrewrite.groovy.GroovyParserVisitor$RewriteGroovyVisitor.visit(GroovyParserVisitor.java:561)"
18 = {StackTraceElement@16132} "org.openrewrite.groovy.GroovyParserVisitor$RewriteGroovyVisitor.access$1200(GroovyParserVisitor.java:549)"
19 = {StackTraceElement@16133} "org.openrewrite.groovy.GroovyParserVisitor$RewriteGroovyClassVisitor.visitMethod(GroovyParserVisitor.java:526)"
20 = {StackTraceElement@16134} "org.openrewrite.groovy.GroovyParserVisitor.convertTopLevelStatement(GroovyParserVisitor.java:2000)"
21 = {StackTraceElement@16135} "org.openrewrite.groovy.GroovyParserVisitor.visit(GroovyParserVisitor.java:160)"
22 = {StackTraceElement@16136} "org.openrewrite.groovy.GroovyParser.lambda$parseInputs$4(GroovyParser.java:154)"
23 = {StackTraceElement@16137} "java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)"
24 = {StackTraceElement@16138} "java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)"
25 = {StackTraceElement@16139} "java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)"
26 = {StackTraceElement@16140} "java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)"
27 = {StackTraceElement@16141} "java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:310)"
28 = {StackTraceElement@16142} "java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:735)"
29 = {StackTraceElement@16143} "java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734)"
30 = {StackTraceElement@16144} "java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734)"
31 = {StackTraceElement@16145} "java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:735)"
32 = {StackTraceElement@16146} "java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)"
33 = {StackTraceElement@16147} "java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)"
34 = {StackTraceElement@16148} "java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:310)"
35 = {StackTraceElement@16149} "java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:735)"
36 = {StackTraceElement@16150} "java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)"
37 = {StackTraceElement@16151} "java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)"
38 = {StackTraceElement@16152} "java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)"
39 = {StackTraceElement@16153} "java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)"
40 = {StackTraceElement@16154} "java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)"
41 = {StackTraceElement@16155} "java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)"
42 = {StackTraceElement@16156} "java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276)"
43 = {StackTraceElement@16157} "java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)"
44 = {StackTraceElement@16158} "java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)"
45 = {StackTraceElement@16159} "java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)"
46 = {StackTraceElement@16160} "java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)"
47 = {StackTraceElement@16161} "java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)"
48 = {StackTraceElement@16162} "java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)"
49 = {StackTraceElement@16163} "org.openrewrite.maven.AbstractRewriteMojo.sourcesWithAutoDetectedStyles(AbstractRewriteMojo.java:290)"
50 = {StackTraceElement@16164} "org.openrewrite.maven.AbstractRewriteMojo.loadSourceSet(AbstractRewriteMojo.java:259)"
51 = {StackTraceElement@16165} "org.openrewrite.maven.AbstractRewriteMojo.listResults(AbstractRewriteMojo.java:240)"
52 = {StackTraceElement@16166} "org.openrewrite.maven.AbstractRewriteRunMojo.execute(AbstractRewriteRunMojo.java:53)"
53 = {StackTraceElement@16167} "org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)"
54 = {StackTraceElement@16168} "org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2(MojoExecutor.java:370)"
55 = {StackTraceElement@16169} "org.apache.maven.lifecycle.internal.MojoExecutor.doExecute(MojoExecutor.java:351)"
56 = {StackTraceElement@16170} "org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:215)"
57 = {StackTraceElement@16171} "org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:171)"
58 = {StackTraceElement@16172} "org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:163)"
59 = {StackTraceElement@16173} "org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)"
60 = {StackTraceElement@16174} "org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)"
61 = {StackTraceElement@16175} "org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)"
62 = {StackTraceElement@16176} "org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)"
63 = {StackTraceElement@16177} "org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:298)"
64 = {StackTraceElement@16178} "org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)"
65 = {StackTraceElement@16179} "org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)"
66 = {StackTraceElement@16180} "org.apache.maven.cli.MavenCli.execute(MavenCli.java:960)"
67 = {StackTraceElement@16181} "org.apache.maven.cli.MavenCli.doMain(MavenCli.java:293)"
68 = {StackTraceElement@16182} "org.apache.maven.cli.MavenCli.main(MavenCli.java:196)"
69 = {StackTraceElement@16183} "java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)"
70 = {StackTraceElement@16184} "java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)"
71 = {StackTraceElement@16185} "java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)"
72 = {StackTraceElement@16186} "java.base/java.lang.reflect.Method.invoke(Method.java:568)"
73 = {StackTraceElement@16187} "org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:282)"
74 = {StackTraceElement@16188} "org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:225)"
75 = {StackTraceElement@16189} "org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406)"
76 = {StackTraceElement@16190} "org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347)"
timtebeek commented 5 months ago

Thanks for the detailed report @SDamasani ; Sorry to see you didn't get a response earlier: I was traveling when this came in, and the parser label moved this to a different dashboard such that I failed to notice it when I got back.

Parsing Groovy and Jenkinsfiles in particular may have some rough edges to it still; we'd added support mostly for Gradle, but would like to cover Jenkinsfiles too. Perhaps a test similar to this one could help, but then stripped away as much as possible to only retain the snippet that the parser has an issue with. That way it's easier to follow up, ideally without a need to put Jenkins libs on the classpath.

SDamasani commented 5 months ago

@timtebeek Thank you taking time to look at the issue. this is the content for Jenkins file that is causing the issue . I unit tested with this content.

#!groovy
def getEndBuild() {
    { steps , config , domain ->
        new net.one.gtu.jenkins.helper.ConsoleLogger(steps).logInfo("Intentionally exiting the build early.  Please" +
                " disregard subsequent errors; they do not represent real concerns/issues");
        throw new org.jenkinsci.plugins.workflow.steps.FlowInterruptedException(hudson.model.Result.SUCCESS);

    }
}
def buildPipeline1() {
    pipelineRunner {
        yml = 'jenkins_old.yml'
        endBuild = getEndBuild();
    }
}
bshashank commented 1 month ago

I managed to hit this issue with the Jenkinsfile as well. Here's a simpler reproduction case:

#!groovy
def aSimpleFunction()
{
    { steps, domain, config ->
        steps.echo(
            "hello world",
            "${steps.env.BRANCH_NAME}"
        )
    }
}

Computing the cursor position when arguments are over multiple lines, and a $variable seems to cause the parser to freak-out.