toki-pona / linja-pona

ilo sitelen pi linja pona
MIT License
3 stars 0 forks source link

implement JISON parser #5

Open corporateanon opened 7 years ago

corporateanon commented 7 years ago

Let's start with the following (not working, but a good starting point):


/* lexical grammar */
%lex
%%

[aeiouptksmnlwj\-]+\b    return 'NIMI'
\s    return 'SPACE'
\n    return 'NEWLINE'
.+    return 'RAW'
<<EOF>>               return 'EOF'

/lex

%start text

%% /* language grammar */

text
  : tokens EOF
        {return $1;}
;

tokens
  : tokens t
        {return $1.concat([$2]);}
    | t
        {return [$1];}
    ;

t
  : SPACE
      {return {space: yytext}; }
  | NIMI
      {return {nimi: yytext}; }
  | NEWLINE
      {return {newline: yytext}; }
  | RAW
      {return {raw: yytext}; }
;
corporateanon commented 7 years ago

Also PEG.js which is more beautiful:

ItemsList
 = list:Item* {
   return list;
 }

Item
 = Space
 / Nimi
 / CartoucheExpression
 / OpenCartoucheExpression
 / Raw

Nimi 'nimi'
 = [aeiouptksmnlwj\-]+ {
    return ['nimi', text()];
}

Space 'space'
 = [ ]+ {
    return ['space', text()]
}

OpenCartoucheExpression
 = CartoucheStart items:ItemsList {
 return ['openCartouche', items]
 }

CartoucheExpression
 = CartoucheStart items:ItemsList CartoucheEnd {
 return ['cartouche', items]
 }

CartoucheStart '['
 = [\[]

CartoucheEnd ']'
 = [\]]

Raw
 = [^ ]+ { return ['raw', text()] }