brownplt / pyret-lang

The Pyret language.
Other
1.07k stars 111 forks source link

Anchor nearley #1610

Closed manaspurohit closed 2 years ago

manaspurohit commented 3 years ago

Making an initial PR, i still need to do some cleanup here but the main feature is using nearley instead of jglr for parsing

the main file to review here is nearley-grammar-generator-(2). This file follows pretty similarly from parser-generator for jglr and will produce a grammar that works for nearley when provided a BNF.

Internally, it uses bnf-grammar.js which shows an example of the grammar produced from the grammar generator as well as being a translation to nearley of the BNF grammar in the parser generator on lines 17 to 90. Since the grammar generator itself uses nearley to produce grammars, the entire library is free from jglr. Some necessary functions / data structures (SrcLoc, Tokenizer, Token, etc) have been moved from jglr to nearley-helpers.js.

Finally nearley-tokenizer.js is more or less a copy of pyret-tokenizer with a few functions added to support nearley

pyret-grammar-(2) contains the result of uses the grammar generator on the pyret BNF and you can inspect the ASTs produced from nearley parsing by looking at nearley-grammar-test.js