lark-parser / lark

Lark is a parsing toolkit for Python, built with a focus on ergonomics, performance and modularity.
MIT License
4.89k stars 414 forks source link

AssertionError on reconstruct with anonymous terminal? #254

Closed abathur closed 4 years ago

abathur commented 6 years ago

I was playing around with reconstruct and see an AssertionError with anonymous terminals (changing the item to a rule or de-anonymizing it both stop the error). These look like:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/lark/visitors.py", line 27, in _call_userfunc
    f = getattr(self, tree.data)
AttributeError: 'WriteTokensTransformer' object has no attribute 'a'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "recon.py", line 16, in <module>
    print(reconstructor.reconstruct(ast))
  File "/usr/local/lib/python3.6/dist-packages/lark/reconstruct.py", line 129, in reconstruct
    return ''.join(self._reconstruct(tree))
  File "/usr/local/lib/python3.6/dist-packages/lark/reconstruct.py", line 123, in _reconstruct
    for x in self._reconstruct(item):
  File "/usr/local/lib/python3.6/dist-packages/lark/reconstruct.py", line 120, in _reconstruct
    res = self.write_tokens.transform(unreduced_tree)
  File "/usr/local/lib/python3.6/dist-packages/lark/visitors.py", line 112, in transform
    return self._transform_tree(tree)
  File "/usr/local/lib/python3.6/dist-packages/lark/visitors.py", line 106, in _transform_tree
    return self._call_userfunc(tree)
  File "/usr/local/lib/python3.6/dist-packages/lark/visitors.py", line 29, in _call_userfunc
    return self.__default__(tree.data, children, tree.meta)
  File "/usr/local/lib/python3.6/dist-packages/lark/reconstruct.py", line 37, in __default__
    assert isinstance(t.pattern, PatternStr)
AssertionError

I boiled it down to a fairly terse case:

from lark import Lark
from lark.reconstruct import Reconstructor

grammar = """
start: a
a: /./ _B
_B: /./+
"""

parser = Lark(grammar, parser="lalr", debug=True)
reconstructor = Reconstructor(parser)

ast = parser.parse("abc")
print(ast.pretty())
print(reconstructor.reconstruct(ast))

If it's intentional, it may just need a nicer message?

erezsh commented 4 years ago

Sorry for not replying to this, I must have missed it. Anyway, this issue has been solved :)