more testing for PEG

[bakpakin]

I have just added a PEG implementation to Janet. It is similar in scope and interface to LPeg, although probably slower as it is implemented as a tree-walk interpreter over Janet data structures. The PEG needs more documentation and testing as well as an eventual compiler.

The PEG can be used with (peg/match pattern text [,start=0]).

An example from the test suite. This example is translated from the LPeg csv example.

(def csv
  '{:field (+
            (* `"` (<-s (at-least 0 (+ (- 1 `"`) (/ `""` `"`)))) `"`)
            (<- (at-least 0 (- 1 (set ",\n")))))
    :main (* :field (at-least 0 (* "," :field)) (+ "\n" -1))})

(peg/match csv "1,2,3") # -> @["1" "2" "3"]

[bakpakin]

PEGs now are compiled to a bytecode format that is faster. Pegs can also be pre-compiled and get better error messages now. I have been adding more tests to the test suite for testing captures and other features.

Use (peg/compile peg) to compile a Janet data structure to a peg.

NOTE: Some docs coming on PEGs, but since I'm still fiddling with the interface things aren't finalized yet.

[honix]

This is computer science for me, but, can PEG replace parse.c in the future?

[bakpakin]

With a few changes, it probably could, but I probably won't. Janet's grammar is certainly simple enough to be easily recognized by a PEG.

• parse.c already works
• parse.c can be suspended and started later (need for efficient repl)
• parse.c has good error reporting
• parse.c will be faster (it's pure C)
• parse.c enables manipulating and querying parser state externally, needed for for repl and error reporting/recovery ((parser/has-more), (parser/state), (parser/where))

The PEG is more for things like interfacing with external small languages, syntax highlighting, and replacing a regex engine or pattern matching library. I put peg in Janet because I wanted some pattern matching facilities and was inspired by Red/REBOL parse.

[honix]

I see. Red/REBOL parse is really cool indeed!

[bakpakin]

Initial PEG doc at https://github.com/janet-lang/janet/blob/master/doc/Peg.md. Interface should be stabalizing soon!

[bakpakin]

General Documentation now at https://janet-lang.org/introduction.html

I have moved the documentation into the janet-lang.org repository so that documentation could go on the website. Also, the Introduction has been broken up into multiple pages for better organization.

Peg docs at https://janet-lang.org/peg.html

[honix]

Now on mobiles overall site looks squeezed. :(

[bakpakin]

Ugh, you’re right. I was doing a lot of refactoring on the website, I will fix mobile soon.

[bakpakin]

Should look better now. For further issues and fixes with the website, post them in the website repo.

[bakpakin]

I'm closing this - any issues with the PEG should go in a new issue.