search

lark-parser / lark

Lark is a parsing toolkit for Python, built with a focus on ergonomics, performance and modularity.
MIT License
4.64k stars 397 forks source link

Standalone parser is not deterministic #1341

Closed Kyuuhachi closed 3 months ago

Kyuuhachi commented 10 months ago

Describe the bug

I want to keep a standalone parser in my repo, to make it easy to use without any dependencies. But even rebuilding exactly the same grammar produces a different parse table, which makes git diff unhappy. I presume this is because set() is unordered.

To Reproduce

The simplest possible grammar: an empty start rule, start:. Then compile it twice and compare the output, diff =(python -m lark.tools.standalone a.g) =(python -m lark.tools.standalone a.g).

3157c3157
< ... 'parser': {'tokens': {0: 'start', 1: '$END'}, 'states': {0: {}, 1: {0: (0, 0), 1: (1, {'@': 0})}}, 'start_states': {'start': 1}, 'end_states': {'start': 0}}, ...
---
> ... 'parser': {'tokens': {0: 'start', 1: '$END'}, 'states': {0: {0: (0, 1), 1: (1, {'@': 0})}, 1: {}}, 'start_states': {'start': 0}, 'end_states': {'start': 1}}, ...

(I cut off some irrelevant parts)

MegaIng commented 10 months ago

Duplicate of https://github.com/lark-parser/lark/issues/1194

aspizu commented 8 months ago

what u doin here