antlr / grammars-v4

Grammars written for ANTLR v4; expectation that the grammars are free of actions.
MIT License
10.15k stars 3.7k forks source link

Python3-py Grammar does not parse simple Python script #1248

Open andrfgs opened 6 years ago

andrfgs commented 6 years ago

I previously had closed this issue as I thought it was a tab encoding problem, but it is not

I have used Python3-py grammar and generated the Antlr files for python, using: antlr4 -Dlanguage=Python3 Python3.g4

When using the following test input program:

import numpy as np

def g():
    a = 2
    b = a * 2

    return b

print(g())

The parser outputs the following errors:

line 1:0 extraneous input 'import' expecting {STRING, NUMBER, 'lambda', 'not', 'None', 'True', 'False', 'await', NAME, '...', '(', '[', '+', '-', '~', '{'} line 1:13 mismatched input 'as' expecting {, NEWLINE}

Which is wrong as the above program is a valid Python3 program.

I used the following code for the parser:

import sys
from antlr4 import *
from antlrGenerated.Python3Lexer import Python3Lexer
from antlrGenerated.Python3Parser import Python3Parser

def main(argv):
    input = FileStream("C:\\Users\\and_s\\Desktop\\main.py")
    lexer = Python3Lexer(input)
    stream = CommonTokenStream(lexer)
    parser = Python3Parser(stream)
    tree = parser.eval_input()

if __name__ == '__main__':
    main(sys.argv)
andrfgs commented 6 years ago

Is this a known bug? Is there something wrong with the code? Does it require proper formating? I tried with both /n and /r/n endings and none work.