EBNF Grammar Definition for Elvish

[awgh]

It would be extremely useful to have an EBNF grammar for elvish as a standalone file in the repo.

Alternatively, it would also be extremely useful to have the interpreter be able to generate its current EBNF grammar definition including builtins from the running interpreter.

With an EBNF spec of some kind, integration with third-party editors and IDEs becomes a lot easier. Having a grammar available inside the interpreter can also help implement auto-completion, which is another open issue.

Specifically, I'm interested in extending editor support to Sublime or VS code. The grammar is useful for syntax highlighting and auto-indentation, etc.

If the main dev has a grammar available (there are some EBNF-looking comments in parse.go), could they share the spec?

Thanks much!

[awgh]

If there is any kind of grammar spec available, I'm happy to convert it to the form I need.

Cheers!

[iwoloschin]

I wrote an Atom syntax highlighter a while back, it mostly worked, but is unsupported as I've moved to VSCode.

https://github.com/iwoloschin/language-elvish

Of course, the first time I fired up VSCode with an Elvish file it found a highlighter, apparently based off my work:

https://github.com/Champii/vscode-elvish

I don't believe either is really an EBNF grammar, but it might be helpful? If nothing else, there's at least some existing editor support if that's all you're looking for to start.

[awgh]

@iwoloschin That's great! I'll try that out for vscode. Thanks!

I do still think the EBNF would be nice to have for future portability and automation. People might want to write other parsers for it somewhere, and it's easier to generate those from a grammar. Not that urgent if VS code has syntax highlighting already though :)

[iwoloschin]

I suspect that the lack of a proper EBNF grammar is because no one has asked for one yet. If you're interested in one the best thing might be to just make a PR with one. Fortunately the language spec is relatively small (I wrote the Atom syntax highlighter in a weekend, and most of it was figuring out how to make a syntax highlighter).

[xiaq]

The parser's source code is annotated with some flavor of BNF https://github.com/elves/elvish/blob/master/parse/parse.go

That said, is a BNF syntax really useful for building editor integrations? Very few editors implement a full parser but rely on simple tokenization for syntax highlighting or code completion.

[xiaq]

The only editor I know of that implements a full parser is Atom, with its incremental parser called tree sitter. Other editors (I checked VS Code and Vim) use tokenization.

[awgh]

The VSCode syntax highlighter is working just fine for me! That was key, thank you! However there are two other reasons to have a published grammar for a language...

1) When language geeks like me find your project, it's easier to just read EBNF than try to pick through source and code and partial docs to piece it together.
2) It would be helpful for other implementations of the interpreter down the road, or other tools that might do transformations on Elvish or process elvish scripts.

What would be extremely nice would be not having a static grammar definition file, but a way of asking an instance of a parser to generate the EBNF file for you. This would also include all custom builtins or other things added by the application that might not be in mainline Elvish.

This would let people use their favorite parser generator (ie yacc/bison) to make things that parse or interact with Elvish scripts.

That said, the high priority for me was the syntax highlighting and that's great. This other stuff is a nice-to-have, it just occurred to me that you could probably get the parser to self-describe its own grammar, and that would be handy.