Add a syntax to support embed custom parser code during parsing (a proposal with demo code)

[fy0]

Sometimes, i want to use a hand writing parser for some exprerssion, for example:

e <- '{' myExpr '}'

myExpr <- {
   // some code to make brackets certainly matching, accept text like "{ {} }"
}

For that, i made a fork. Owe to pigeon's clear design, it's much easier than i thought it should be.

I use sign "*{}" to describle custom parser code block:

{
package main
}

e <- "begin" x "end" !.

myExpr <- '1' / '2' / '3'

x <-  *{
    start := p.pt

    arr := []rune{}
    for i := 0 ; i < 3; i+=1  {
        cur := p.pt.rn
        arr = append(arr, cur)
        p.read()
    }

    r, ok := p.parseExpr(p.rules["myExpr"].expr)

    p.failAt(true, start.position, "3 char + myExpr")
    return string(arr) + string(r.([]uint8)), ok, nil
}

test code:

package main

import "fmt"

func main() {
    fmt.Println(Parse("", []byte("beginxxx1end")))
}

> go run .
[[98 101 103 105 110] xxx1 [101 110 100] <nil>] <nil>

But if i want to control the parser by manual, expose "parser" object to action is necessary. I noticed parser object is willful hide for all code blocks. So i'm worried about i had break a guideline.

Have a look and have a nice day.

[breml]

Hi @fy0 Thanks for your pull request. I would like to better understand in what kind of situations you want to control the parser manually. Can you elaborate and provide some real world examples?

[fy0]

Hi @fy0 Thanks for your pull request. I would like to better understand in what kind of situations you want to control the parser manually. Can you elaborate and provide some real world examples?

For example, I want to embed another parser inside my peg parser.

{
package main
}

e <- "<%js" js_code "%>" !.

js_code <- *{
    // there is a js parser, eat unknow length text, return a object
}

If do this work with current pigeon, I need to describe all syntax by peg, or just match '%>' and pray for '%>' is not appears in js code part.

It's also useful to implement meta programming language feature. Another example, i want user can register their own operators. pigeon's #{ syntax is nearly works, but not good.