Can't get this grammar to work, failure on second writeln. Bug?

[enjoysmath]

import std.stdio; import pegged.grammar; import pegged.parser; import pegged.peg;

mixin(grammar(" Math: AbstractList <- List(Var / '...') List(A) <- A (:',' A) Var <- VarBase Sub? Sub <- '_' (Nat / VarBase) VarBase <~ [a-zA-Z] Set <- VarCap Sub? / StdSets Nat <- [0-9]+ VarCap <~ [A-Z] Word <~ ([a-z]+ / [A-Z] [a-z]+) Vocab <- Word (:' ' Word) StdSets <- 'RR' / 'CC' / 'ZZ' / 'NN' / 'QQ' "));

mixin(grammar(" Let: LetStmt <- Let_X_in_Y / Let_X_be_a_Y / Let_X_be_Y Let_X_in_Y <- 'Let ' :'(' Math.AbstractList ' in ' Math.Set :')' Let_X_be_a_Y <- 'Let ' :'(' Math.Var ' be a ' Math.Vocab :')' Let_X_be_Y <- 'Let ' :'(' Math.AbstractList ' be ' Math.Vocab :')' "));

int main(string[] argv) { writeln(Let("Let (a,b,c,... in G_1)")); writeln(Let("Let (K,H,G be groups)")); readln(); return 0; }

I've tried several ways of doing it and it keeps coming back with the same error:

Let [0, 22]["Let ", "a", "b", "c", "...", " in ", "G", "", "1"] +-Let.LetStmt [0, 22]["Let ", "a", "b", "c", "...", " in ", "G", "", "1"] +-Let.Let_X_inY [0, 22]["Let ", "a", "b", "c", "...", " in ", "G", "", "1"]

Let (failure) +-Let.LetStmt (failure) +-literal!(" be ") failure at line 0, col 14, after "(K,H,G be " expected " be ", but got "groups)"

The list is absorbing the ' be' as well as if spaces can be in the list, but I coded them not to be!

Thanks for your help.

[enjoysmath]

I verified that it either writeln will fail, but it's always the second one. So something is wrong with your library.

[veelo]

Confirmed.

The problem goes away when you turn off memoization on the composed grammar (the one(s) not directly called) like this (read on for another workaround):

import std.stdio;
import pegged.grammar;

mixin(grammar!(Memoization.no)("
    Math:
        AbstractList    <- List(Var / '...')
        List(A)         <- A (:',' A)*
        Var             <- VarBase Sub?
        Sub             <- '_' (Nat / VarBase)
        VarBase         <~ [a-zA-Z]
        Set             <- VarCap Sub? / StdSets
        Nat             <- [0-9]+
        VarCap          <~ [A-Z]
        Word            <~ ([a-z]+ / [A-Z] [a-z]+)
        Vocab           <- Word (:' ' Word)*
        StdSets         <- 'RR' / 'CC' / 'ZZ' / 'NN' / 'QQ'
"));

mixin(grammar("
    Let:
        LetStmt         <- Let_X_in_Y / Let_X_be_a_Y / Let_X_be_Y
        Let_X_in_Y      <- 'Let ' :'(' Math.AbstractList ' in ' Math.Set :')'
        Let_X_be_a_Y    <- 'Let ' :'(' Math.Var ' be a ' Math.Vocab :')'
        Let_X_be_Y      <- 'Let ' :'(' Math.AbstractList ' be ' Math.Vocab :')'
"));

void main(string[] argv)
{
    writeln(Let("Let (a,b,c,... in G_1)"));
    writeln(Let("Let (K,H,G be groups)"));
}

I think I know what the problem is. Every grammar has its own memoization table. The memoization table is cleared when a rule is called with a string, which usually means a new parse is started. But that happens only for the outer grammar, not for the ones called indirectly through composition.

So we need a way so clear all memoization tables in all grammars.

In the mean time, the workaround for memoizing parsers is to insert a dummy direct call to the other grammar like this:

import std.stdio;
import pegged.grammar;

mixin(grammar("
    Math:
        AbstractList    <- List(Var / '...')
        List(A)         <- A (:',' A)*
        Var             <- VarBase Sub?
        Sub             <- '_' (Nat / VarBase)
        VarBase         <~ [a-zA-Z]
        Set             <- VarCap Sub? / StdSets
        Nat             <- [0-9]+
        VarCap          <~ [A-Z]
        Word            <~ ([a-z]+ / [A-Z] [a-z]+)
        Vocab           <- Word (:' ' Word)*
        StdSets         <- 'RR' / 'CC' / 'ZZ' / 'NN' / 'QQ'
"));

mixin(grammar("
    Let:
        LetStmt         <- Let_X_in_Y / Let_X_be_a_Y / Let_X_be_Y
        Let_X_in_Y      <- 'Let ' :'(' Math.AbstractList ' in ' Math.Set :')'
        Let_X_be_a_Y    <- 'Let ' :'(' Math.Var ' be a ' Math.Vocab :')'
        Let_X_be_Y      <- 'Let ' :'(' Math.AbstractList ' be ' Math.Vocab :')'
"));

void main(string[] argv)
{
    writeln(Let("Let (a,b,c,... in G_1)"));
    auto dummy = Math("");
    writeln(Let("Let (K,H,G be groups)"));
}

Bastiaan.