Syntax diagram generation

[napulen]

Suggestion Provide a clear and concise description of what the problem is, what you want to happen.

How difficult would it be to add a generator of syntax diagrams for lark? Like this one: https://www.bottlecaps.de/rr/ui

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

I tried converting the lark syntax into the EBNF convention they use for those diagrams. Doing a string replace of : for ::= gets you a long way, but the aliases and a few operators create problems. I was just wondering whether the internal representations of lark already provide an easy way to do something like that.

Additional context Add any other context or screenshots about the feature request here.

[erezsh]

Have a look at examples/lark.lark, you can use it to parse Lark grammars and transform them into the desired syntax.

[j6k4m8]

I love this idea — would be super helpful toward debugging a syntax as you're writing it (and sharing it in a clear form with others). I need one of these, so if I manage to do the above, I'll share the code here :)

This library looks promising in terms of easy rendering.

[erezsh]

Cool. If it works I'll find somewhere in the codebase to add it.

[MegaIng]

I create a simple generator using railroad, which is a perfect fit for the lark Transformer way of work: https://github.com/MegaIng/lark-railroad. Results are useable, and include links from rules to each other.

[erezsh]

@MegaIng Nice! Too bad there's no support for templates or constant repetitions. I can plug it to the BNF result (it might even work without changing your code, since the BNF tree should be just a subset of the EBNF tree), but that might reduce readability.

[MegaIng]

I could represent templates, but I decided not to since I didn't want to think about how to best do that (Worst case: A string containing the template arguments and the template source).

Same goes for String ranges.

repetitions on the other hand are not supported at all (I could do weird generation akin to what is done during EBNF to BNF generation, but that might bloat the diagram.)

I am still not sure about the best way to represent %import, %ignore and %declare.

[MegaIng]

Ok, I added literal ranges and repetitions, as well as links to external regex visualizers (like regex101) and support for %import with links to other files.

[MegaIng]

@erezsh Would you want this as lark.tools.railroad or as an external package?

[erezsh]

@MegaIng How many times have you personally used it for practical purposes? (i.e. for genuine assistance in your other projects)

[RossPatterson]

I've been playing with @MegaIng's lark-railroad. It needed a little updating, due to "modern" Lark changes (e.g., Transformer.rule_params(), return Discard instead of raise Discard, etc.). But it's working fine. I'm using it as the basis for a help facility that includes a small Lark grammar in its output. Combined with my recent contribution to railroad-diagrams of the option of generating text-mode diagrams that display nicely on a terminal, it's proving useful.