search

openrewrite / rewrite-python

OpenRewrite recipes for Python.
Apache License 2.0
17 stars 5 forks source link

Python parsing is failing in Windows #61

Closed rpau closed 11 months ago

rpau commented 1 year ago

What version of OpenRewrite are you using?

I am using the latest in the Moderne CLI

How are you running OpenRewrite?

Using Moderne CLI and running the Python parser from a JVM process

What is the smallest, simplest way to reproduce the problem?

Parse any Python file in Windows. 

import os

class AClass:
    pass

def aFunction():
    pass

print("hello world")

If you belong to the Moderne organization you can see the entire stacktrace here:

https://github.com/moderneinc/moderne-cli/actions/runs/5323559209/jobs/9641587326#step:12:1

Error parsingerror processing compound statement of type PyClassImpl in:
--
class AClass:
--
java.lang.RuntimeException: error processing compound statement of type PyClassImpl in:
--
class AClass:
--
    at org.openrewrite.python.internal.PsiPythonMapper.mapStatement(PsiPythonMapper.java:173)
    at org.openrewrite.python.internal.PsiPythonMapper.mapBlock(PsiPythonMapper.java:1450)
    at org.openrewrite.python.internal.PsiPythonMapper.mapBlock(PsiPythonMapper.java:1406)
    at org.openrewrite.python.internal.PsiPythonMapper.mapFile(PsiPythonMapper.java:127)
    at org.openrewrite.python.internal.PsiPythonMapper.mapSource(PsiPythonMapper.java:111)
    at org.openrewrite.python.PythonParser.lambda$parseInputs$1(PythonParser.java:75)
    at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
    at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:1[79](https://github.com/moderneinc/moderne-cli/actions/runs/5323559209/jobs/9641587326#step:12:80))
    at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
    at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
    at io.moderne.serialization.TreeSerializer.write(Unknown Source)
    at io.moderne.serialization.TreeSerializer.write(Unknown Source)
    at io.moderne.modjava.commands.build.LSTUtils.writeLstFiles(LSTUtils.java:34)
    at io.moderne.modjava.commands.build.BuildPython.call(BuildPython.java:66)
    at io.moderne.modjava.commands.build.BuildPython.call(BuildPython.java:19)
    at picocli.CommandLine.executeUserObject(CommandLine.java:2041)
    at picocli.CommandLine.access$1500(CommandLine.java:148)
    at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2453)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2415)
    at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273)
    at picocli.CommandLine$RunLast.execute(CommandLine.java:2417)
    at picocli.CommandLine.execute(CommandLine.java:2170)
    at io.moderne.modjava.ModJava.main(ModJava.java:24)
Caused by: java.lang.IllegalStateException: did not stop (9) where expected (13);
In file example.py:
--
import os
        ^-------[ STOPPED HERE ]

class AClass:
    pass

def aFunction():
    pass

print("hello world")
--

In file example.py:
--
import os

class AClass:
  ^-------[ EXPECTED HERE ]
    pass

def aFunction():
    pass

print("hello world")
--

    at org.openrewrite.python.internal.PsiPaddingCursor.expectNext(PsiPaddingCursor.java:410)
    at org.openrewrite.python.internal.PsiPaddingCursor.consumeRemainingAndExpect(PsiPaddingCursor.java:315)
    at org.openrewrite.python.internal.PsiPythonMapper$BlockContext.nextStatementPrefix(PsiPythonMapper.java:[89](https://github.com/moderneinc/moderne-cli/actions/runs/5323559209/jobs/9641587326#step:12:90))
    at org.openrewrite.python.internal.PsiPythonMapper.mapClassDeclarationStatement(PsiPythonMapper.java:[120](https://github.com/moderneinc/moderne-cli/actions/runs/5323559209/jobs/9641587326#step:12:121)3)
    at org.openrewrite.python.internal.PsiPythonMapper.mapStatement(PsiPythonMapper.java:155)
    ... 27 more
timtebeek commented 11 months ago

A lot has changed with how we parse Python and run invoke the parser from the Moderne CLI. I propose we close this issue and revisit this when we next look into Python.