antlr / antlr4-lab

A client/server for trying out and learning about ANTLR
MIT License
38 stars 11 forks source link

http://lab.antlr.org sample has broken #85

Closed man-chan closed 6 months ago

man-chan commented 6 months ago

Go to http://lab.antlr.org/, and hit the 'run' button to parse the sample grammar and expression, it shows errors below.

2:9 token recognition error at: '+'
2:10 extraneous input 'foo' expecting ';'

Sample expression:

f(x,y) {
    a = 3+foo;
    x and y;
}

Sample grammar:

parser grammar ExprParser;
options { tokenVocab=ExprLexer; }

program
    : stat EOF
    | def EOF
    ;

stat: ID '=' expr ';'
    | expr ';'
    ;

def : ID '(' ID (',' ID)* ')' '{' stat* '}' ;

expr: ID
    | INT
    | func
    | 'not' expr
    | expr 'and' expr
    | expr 'or' expr
    ;

func : ID '(' expr (',' expr)* ')' ;

Sample lexer

// DELETE THIS CONTENT IF YOU PUT COMBINED GRAMMAR IN Parser TAB
lexer grammar ExprLexer;

AND : 'and' ;
OR : 'or' ;
NOT : 'not' ;
EQ : '=' ;
COMMA : ',' ;
SEMI : ';' ;
LPAREN : '(' ;
RPAREN : ')' ;
LCURLY : '{' ;
RCURLY : '}' ;

INT : [0-9]+ ;
ID: [a-zA-Z_][a-zA-Z_0-9]* ;
WS: [ \t\n\r\f]+ -> skip ;
man-chan commented 6 months ago

I hacked it to work, but I'm not sure its validity.

Sample grammar

parser grammar ExprParser;
options { tokenVocab=ExprLexer; }

program
    : stat EOF
    | def EOF
    ;

stat: ID '=' expr ';'
    | expr ';'
    ;

def : ID '(' ID (',' ID)* ')' '{' stat* '}' ;

expr: ID
    | INT
    | func
    | 'not' expr
    | expr PLUS expr
    | expr MINUS expr
    | expr AND expr
    | expr OR expr
    ;

func : ID '(' expr (',' expr)* ')' ;

Sample lexer

// DELETE THIS CONTENT IF YOU PUT COMBINED GRAMMAR IN Parser TAB
lexer grammar ExprLexer;

PLUS : '+' ;
MINUS : '-' ;
AND : 'and' ;
OR : 'or' ;
NOT : 'not' ;
EQ : '=' ;
COMMA : ',' ;
SEMI : ';' ;
LPAREN : '(' ;
RPAREN : ')' ;
LCURLY : '{' ;
RCURLY : '}' ;

INT : [0-9]+ ;
ID: [a-zA-Z_][a-zA-Z_0-9]* ;
WS: [ \t\n\r\f]+ -> skip ;
parrt commented 6 months ago

I put an error in there on purpose so you could see that functionality :)

man-chan commented 6 months ago

Oh! My bad 🤣

BTW I want to say this for years. I really appreciate your selfless effort to put Antlr together 🫡. It is a great OSS project.

To the newbie like me, after some learning and hacks with GPT 😅, I get a simple end-to-end executable example working. Hope this helps anyone the first step in your hacking.

SimpleLang.g4:

grammar SimpleLang;

program
    : stat+ EOF
    ;

stat: assig SEMI
    | expr SEMI
    | def
    | SEMI
    ;

assig: ID EQ expr
    ;

def : ID LPAREN ID (COMMA ID)* RPAREN LCURLY stat* RCURLY ;

expr: ID
    | INT
    | 'not' expr
    | expr MULT expr
    | expr DIV expr
    | expr PLUS expr
    | expr MINUS expr
    | expr AND expr
    | expr OR expr
    | LPAREN expr RPAREN
    ;

func : ID LPAREN expr (COMMA expr)* RPAREN ;

PLUS : '+' ;
MINUS : '-' ;
MULT: '*' ;
DIV: '/' ;
AND : 'and' ;
OR : 'or' ;
NOT : 'not' ;
EQ : '=' ;
COMMA : ',' ;
SEMI : ';' ;
LPAREN : '(' ;
RPAREN : ')' ;
LCURLY : '{' ;
RCURLY : '}' ;

INT : [0-9]+ ;
ID: [a-zA-Z_][a-zA-Z_0-9]* ;
WS: [ \t\n\r\f]+ -> skip ;

Code generation:

$ antlr4 -Dlanguage=Python3 -listener SimpleLang.g4 -o gen/

simplelang.py (only '+' & '*' implemented)

from antlr4 import *
from gen.SimpleLangLexer import SimpleLangLexer
from gen.SimpleLangParser import SimpleLangParser
from gen.SimpleLangListener import SimpleLangListener

class EvalListener(SimpleLangListener):
    def __init__(self):
        self.vars = {}  # Dictionary to store variable values

    def exitAssig(self, ctx):
        name = ctx.ID().getText()  # The variable name
        value = self.evaluate(ctx.expr())
        self.vars[name] = value

    def exitExpr(self, ctx):
        if ctx.INT():
            self.vars[ctx.getText()] = int(ctx.getText())
        elif ctx.ID():
            self.vars[ctx.getText()] = self.vars.get(ctx.getText(), 0)
        elif ctx.PLUS():
            left = self.evaluate(ctx.expr(0))
            right = self.evaluate(ctx.expr(1))
            self.vars[ctx.getText()] = left + right
        elif ctx.MULT():
            left = self.evaluate(ctx.expr(0))
            right = self.evaluate(ctx.expr(1))
            self.vars[ctx.getText()] = left * right
        elif ctx.LPAREN():
            # Directly use the value computed for the expression inside parentheses
            self.vars[ctx.getText()] = self.evaluate(ctx.expr(0))

    def evaluate(self, expr):
        # Helper method to evaluate an expression node
        text = expr.getText()
        if text.isdigit():
            return int(text)
        elif text in self.vars:
            return self.vars[text]
        elif 'not' in text:
            return not self.evaluate(expr.expr(0))
        elif expr.PLUS():
            return self.evaluate(expr.expr(0)) + self.evaluate(expr.expr(1))
        elif expr.MULT():
            return self.evaluate(expr.expr(0)) * self.evaluate(expr.expr(1))
        elif expr.expr():
            return self.evaluate(expr.expr(0))
        return 0

def main():
    # Input string
    input = """
    f(x){ x*x; }
    x = 3; 
    y = 5; 
    z = (x + y) * 7; 
    z;
    """
    input_stream = InputStream(input)
    lexer = SimpleLangLexer(input_stream)
    stream = CommonTokenStream(lexer)
    parser = SimpleLangParser(stream)
    tree = parser.program()

    # Create the listener and walk through the parse tree
    listener = EvalListener()
    walker = ParseTreeWalker()
    walker.walk(listener, tree)

    # Output z
    print("The value of z is:", listener.vars['z'])

if __name__ == '__main__':
    main()

Output:

$ python simplelang.py
The value of z is: 56