search

amykyta3 / speedy-antlr-tool

Generate an accelerator extension that makes your Antlr parser in Python super-fast!
BSD 3-Clause "New" or "Revised" License
29 stars 7 forks source link

tree.parser attribute is None #11

Open m-zakeri opened 2 years ago

m-zakeri commented 2 years ago

It seems that the tree.parser attribute does not set to its correct reference when using USE_CPP_BACKEND to create a parse tree. It returns None. However, this attribute is set to a valid reference of ANTLR parser when using Python backend It is required to initialize the TokenStreamRewriter class with parse tree tokens in the following code (line 4 raise an exception): 'NoneType' object has no attribute 'getInputStream'

file_stream = FileStream(java_file_path)
sa_javalabeled.USE_CPP_IMPLEMENTATION = config.USE_CPP_BACKEND
tree = sa_javalabeled.parse(file_stream, 'compilationUnit')
tokens = tree.parser.getInputStream()
rewriter = TokenStreamRewriter(tokens)
m-zakeri commented 2 years ago

I also watched that token_stream.fill(); is called in C++ do_parse function but it did not affects the returned PyObject.

m-zakeri commented 2 years ago

It seems that the tree.parser attribute does not set to its correct reference when using USE_CPP_BACKEND to create a parse tree. It returns None. However, this attribute is set to a valid reference of ANTLR parser when using Python backend It is required to initialize the TokenStreamRewriter class with parse tree tokens in the following code (line 4 raise an exception): 'NoneType' object has no attribute 'getInputStream'

file_stream = FileStream(java_file_path)
sa_javalabeled.USE_CPP_IMPLEMENTATION = config.USE_CPP_BACKEND
tree = sa_javalabeled.parse(file_stream, 'compilationUnit')
tokens = tree.parser.getInputStream()
rewriter = TokenStreamRewriter(tokens)

Can anyone help me implement the binding of the input stream tokens?