Grammar railroad diagram

mingodad commented 1 year ago

Using some online tools like https://www.bottlecaps.de/rr/ui and https://www.bottlecaps.de/convert/ and a bit of manual fixes we can have a nice navigable railroad diagram.

Copy and paste the EBNF shown bellow on https://www.bottlecaps.de/rr/ui on the tab Edit Grammar the click on the tab View Diagram to see/download a navigable railroad diagram.

/* converted on Sun Nov 27, 2022, 10:23 (UTC+01) by bison-to-w3c v0.62 which is Copyright (c) 2011-2022 by Gunther Rademacher <grd@gmx.net> */

goal     ::= initialize optNewlines runeFile
initialize
         ::=
runeFile ::= statements
statements
         ::= statement*
statement
         ::= appendCode
           | assertStatement
           | assignmentStatement
           | callStatement
           | class
           | debugStatement
           | enum
           | externFunction
           | finalFunction
           | foreachStatement
           | forStatement
           | function
           | generateStatement
           | generatorStatement
           | ifStatement
           | import
           | prependCode
           | printlnStatement
           | printStatement
           | refStatement
           | relationStatement
           | returnStatement
           | struct
           | switchStatement
           | throwStatement
           | unitTestStatement
           | unrefStatement
           | whileStatement
           | yield
import   ::= ( ( KWIMPORT | KWIMPORTLIB | KWIMPORTRPC ) pathExpressionWithAlias | KWUSE IDENT ) newlines
class    ::= ( classHeader | exportClassHeader ) '(' oneOrMoreParameters ')' block
classHeader
         ::= KWCLASS IDENT optWidth
optWidth ::= ( ':' UINTTYPE )?
exportClassHeader
         ::= ( KWEXPORT | KWEXPORTLIB | KWRPC ) KWCLASS IDENT optWidth
struct   ::= structHeader '{' newlines ( structMember newlines )* '}' newlines
           | exportStructHeader block
structHeader
         ::= KWSTRUCT IDENT
structMember
         ::= optConst IDENT optTypeExpression optInitializer
optConst ::= KWCONST?
optInitializer
         ::= ( '=' expression )?
exportStructHeader
         ::= ( KWEXPORT | KWEXPORTLIB ) KWSTRUCT IDENT
appendCode
         ::= appendCodeHeader block
appendCodeHeader
         ::= KWAPPENDCODE pathExpression?
prependCode
         ::= prependCodeHeader block
prependCodeHeader
         ::= KWPREPENDCODE pathExpression?
block    ::= '{' newlines statements '}' optNewlines
function ::= ( functionHeader | exportFunctionHeader | rpcHeader ) '(' parameters ')' optFuncTypeExpression block
functionHeader
         ::= ( KWFUNC | KWITERATOR ) IDENT
           | KWOPERATOR operator
operator ::= '+'
           | '-'
           | '*'
           | '/'
           | '%'
           | KWAND
           | KWOR
           | KWXOR
           | '&'
           | '|'
           | '@'
           | '^'
           | KWSHL
           | KWSHR
           | KWROTL
           | KWROTR
           | KWADDTRUNC
           | KWSUBTRUNC
           | KWMULTRUNC
           | '~'
           | '<'
           | KWLE
           | '>'
           | KWGE
           | KWEQUAL
           | KWNOTEQUAL
           | '!'
           | '[' ']'
           | KWIN
exportFunctionHeader
         ::= ( KWEXPORT ( KWFUNC | KWITERATOR ) | KWEXPORTLIB KWFUNC ) IDENT
parameters
         ::= oneOrMoreParameters?
oneOrMoreParameters
         ::= parameter ( ',' optNewlines parameter )*
parameter
         ::= optVar ( IDENT | '<' IDENT '>' ) optTypeExpression
           | initializedParameter
optVar   ::= KWVAR?
initializedParameter
         ::= optVar ( IDENT | '<' IDENT '>' ) optTypeExpression '=' expression
externFunction
         ::= ( KWEXTERN STRING functionHeader | rpcHeader ) '(' parameters ')' optFuncTypeExpression newlines
rpcHeader
         ::= KWRPC IDENT
ifStatement
         ::= ifPart elseIfPart* optElsePart
ifPart   ::= ifStatementHeader expression block
ifStatementHeader
         ::= KWIF
elseIfPart
         ::= elseIfStatementHeader expression block
elseIfStatementHeader
         ::= KWELSE KWIF
optElsePart
         ::= elsePart?
elsePart ::= elseStatementHeader block
elseStatementHeader
         ::= KWELSE
switchStatement
         ::= switchStatementHeader expression switchBlock
switchStatementHeader
         ::= KWSWITCH
switchBlock
         ::= '{' newlines switchCase* optDefaultCase '}' optNewlines
switchCase
         ::= KWCASE expression ( ',' optNewlines expression )* block
optDefaultCase
         ::= ( defaultCaseHeader block )?
defaultCaseHeader
         ::= KWDEFAULT
whileStatement
         ::= optDoStatement whileStatementHeader expression ( newlines | block )
whileStatementHeader
         ::= KWWHILE
optDoStatement
         ::= doStatement?
doStatement
         ::= doStatementHeader block
doStatementHeader
         ::= KWDO
forStatement
         ::= forStatementHeader nonConstAssignmentExpression ',' optNewlines expression ',' optNewlines nonConstAssignmentExpression block
forStatementHeader
         ::= KWFOR
assignmentStatement
         ::= assignmentExpression newlines
assignmentExpression
         ::= KWCONST? nonConstAssignmentExpression
nonConstAssignmentExpression
         ::= accessExpression optTypeExpression assignmentOp expression
assignmentOp
         ::= '='
           | KWADDEQUALS
           | KWSUBEQUALS
           | KWMULEQUALS
           | KWDIVEQUALS
           | KWMODEQUALS
           | KWBITANDEQUALS
           | KWBITOREQUALS
           | KWBITXOREQUALS
           | KWANDEQUALS
           | KWOREQUALS
           | KWXOREQUALS
           | KWEXPEQUALS
           | KWSHLEQUALS
           | KWSHREQUALS
           | KWROTLEQUALS
           | KWROTREQUALS
           | KWADDTRUNCEQUALS
           | KWSUBTRUNCEQUALS
           | KWMULTRUNCEQUALS
optTypeExpression
         ::= ( ':' expression )?
optFuncTypeExpression
         ::= ( KWARROW expression )?
accessExpression
         ::= tokenExpression ( '(' callParameterList ')' | '.' IDENT | '[' expression ( ':' expression )? ']' )*
callStatement
         ::= accessExpression '(' callParameterList ')' newlines
callParameterList
         ::= ( callParameter ( ',' optNewlines callParameter )* optComma )?
callParameter
         ::= ( IDENT '=' )? expression
optComma ::= ','?
printStatement
         ::= KWPRINT expressionList newlines
printlnStatement
         ::= KWPRINTLN expressionList newlines
throwStatement
         ::= KWTHROW expressionList newlines
assertStatement
         ::= KWASSERT expressionList newlines
returnStatement
         ::= KWRETURN expression? newlines
generatorStatement
         ::= generatorHeader '(' parameters ')' block
generatorHeader
         ::= KWGENERATOR IDENT
generateStatement
         ::= KWGENERATE pathExpression '(' expressionList ')' newlines
relationStatement
         ::= KWRELATION pathExpression pathExpression optLabel pathExpression optLabel optCascade optExpressionList newlines
optLabel ::= ( ':' STRING )?
optCascade
         ::= KWCASCADE?
optExpressionList
         ::= ( '(' expressionList ')' )?
yield    ::= KWYIELD expression newlines
unitTestStatement
         ::= unitTestHeader block
unitTestHeader
         ::= KWUNITTEST IDENT
debugStatement
         ::= debugHeader block
debugHeader
         ::= KWDEBUG
enum     ::= enumHeader '{' newlines entry+ '}' newlines
enumHeader
         ::= KWENUM IDENT
entry    ::= IDENT ( '=' INTEGER )? newlines
foreachStatement
         ::= forStatementHeader IDENT KWIN expression block
finalFunction
         ::= finalHeader '(' parameter ')' block
finalHeader
         ::= KWFINAL
refStatement
         ::= KWREF expression newlines
unrefStatement
         ::= KWUNREF expression newlines
expressionList
         ::= oneOrMoreExpressions?
oneOrMoreExpressions
         ::= expression ( ',' optNewlines expression )*
expression
         ::= dotDotDotExpression
dotDotDotExpression
         ::= selectExpression ( KWDOTDOTDOT selectExpression )?
selectExpression
         ::= orExpression ( '?' orExpression ':' orExpression )?
orExpression
         ::= xorExpression ( KWOR xorExpression )*
xorExpression
         ::= andExpression ( KWXOR andExpression )*
andExpression
         ::= inExpression ( KWAND inExpression )*
inExpression
         ::= modExpression ( KWIN modExpression )?
modExpression
         ::= relationExpression ( KWMOD bitorExpression )?
relationExpression
         ::= bitorExpression ( ( '<' | KWLE | '>' | KWGE | KWEQUAL | KWNOTEQUAL ) bitorExpression )?
bitorExpression
         ::= bitxorExpression ( '|' bitxorExpression )*
bitxorExpression
         ::= bitandExpression ( '@' bitandExpression )*
bitandExpression
         ::= shiftExpression ( '&' shiftExpression )*
shiftExpression
         ::= addExpression ( ( KWSHL | KWSHR | KWROTL | KWROTR ) addExpression )?
addExpression
         ::= KWSUBTRUNC? mulExpression ( ( '+' | '-' | KWADDTRUNC | KWSUBTRUNC ) mulExpression )*
mulExpression
         ::= prefixExpression ( ( '*' | '/' | '%' | KWMULTRUNC ) prefixExpression )*
prefixExpression
         ::= postfixExpression ( '^' postfixExpression )*
           | ( '!' | '~' | '-' | ( '<' | KWCASTTRUNC ) prefixExpression '>' ) prefixExpression
postfixExpression
         ::= accessExpression
           | '&' pathExpression '(' expressionList ')'
tokenExpression
         ::= IDENT
           | STRING
           | INTEGER
           | FLOAT
           | RANDUINT
           | BOOL
           | '[' oneOrMoreExpressions ']'
           | ( KWSECRET | KWREVEAL | KWARRAYOF | KWTYPEOF | KWUNSIGNED | KWSIGNED | KWWIDTHOF | KWISNULL )? '(' expression ')'
           | tupleExpression
           | KWNULL
           | typeLiteral
typeLiteral
         ::= UINTTYPE
           | INTTYPE
           | KWSTRING
           | KWBOOL
           | KWF32
           | KWF64
pathExpression
         ::= IDENT ( '.' IDENT )*
pathExpressionWithAlias
         ::= pathExpression ( KWAS IDENT )?
tupleExpression
         ::= '(' expression ( ( ',' optNewlines expression )+ optComma | ',' ) ')'
optNewlines
         ::= '\n'*
newlines ::= ( '\n' | ';' )+

//
// Tokens
//

// <INITIAL>\("[^"]+"\)\s+{.+?return \([^;]+\); }  -> \2 ::= \1

KWAPPENDCODE ::= "appendcode"
KWARRAYOF ::= "arrayof"
KWAS ::= "as"
KWASSERT ::= "assert"
KWBOOL ::= "bool"
KWCASCADE ::= "cascade"
KWCASE ::= "case"
KWCLASS ::= "class"
KWCONST ::= "const"
KWDEBUG ::= "debug"
KWDEFAULT ::= "default"
KWDO ::= "do"
KWELSE ::= "else"
KWENUM ::= "enum"
KWEXPORT ::= "export"
KWEXPORTLIB ::= "exportlib"
KWRPC ::= "rpc"
KWEXTERN ::= "extern"
BOOL ::= "false"
KWFINAL ::= "final"
KWFOR ::= "for"
KWFUNC ::= "func"
KWGENERATE ::= "generate"
KWGENERATOR ::= "generator"
KWIF ::= "if"
KWIMPORT ::= "import"
KWIMPORTLIB ::= "importlib"
KWIMPORTRPC ::= "importrpc"
KWIN ::= "in"
KWISNULL ::= "isnull"
KWITERATOR ::= "iterator"
KWMOD ::= "mod"
KWNULL ::= "null"
KWOPERATOR ::= "operator"
KWPREPENDCODE ::= "prependcode"
KWPRINT ::= "print"
KWPRINTLN ::= "println"
KWREF ::= "ref"
KWRELATION ::= "relation"
KWRETURN ::= "return"
KWREVEAL ::= "reveal"
KWSECRET ::= "secret"
KWSIGNED ::= "signed"
KWSTRING ::= "string"
KWSTRUCT ::= ("struct"|"message")
KWSWITCH ::= "switch"
KWTHROW ::= "throw"
BOOL ::= "true"
KWTYPEOF ::= "typeof"
KWUNITTEST ::= "unittest"
KWUNREF ::= "unref"
KWUNSIGNED ::= "unsigned"
KWUSE ::= "use"
KWVAR ::= "var"
KWWHILE ::= "while"
KWWIDTHOF ::= "widthof"
KWYIELD ::= "yield"

KWADDEQUALS ::= "+="
KWADDTRUNCEQUALS ::= "!+="
KWADDTRUNC ::= "!+"
KWANDEQUALS ::= "&&="
KWBITANDEQUALS ::= "&="
KWAND ::= "&&"
KWARROW ::= "->"
KWCASTTRUNC ::= "!<"
KWDIVEQUALS ::= "/="
KWDOTDOTDOT ::= "..."
KWEQUAL ::= "=="
KWEXPEQUALS ::= "^="
KWGE ::= ">="
KWLE ::= "<="
KWMODEQUALS ::= "%="
KWMULEQUALS ::= "*="
KWMULTRUNCEQUALS ::= "!*="
KWMULTRUNC ::= "!*"
KWNOTEQUAL ::= "!="
KWBITOREQUALS ::= "|="
KWOREQUALS ::= "||="
KWOR ::= "||"
KWROTLEQUALS ::= "<<<="
KWROTL ::= "<<<"
KWROTREQUALS ::= ">>>="
KWROTR ::= ">>>"
KWSHLEQUALS ::= "<<="
KWSHL ::= "<<"
KWSHREQUALS ::= ">>="
KWSHR ::= ">>"
KWSUBEQUALS ::= "-="
KWSUBTRUNCEQUALS ::= "!-="
KWSUBTRUNC ::= "!-"
KWBITXOREQUALS ::= "@="
KWXOREQUALS ::= "@@="
KWXOR ::= "@@"
KWF32 ::= "f32"
KWF64 ::= "f64"

waywardgeek commented 1 year ago

I let the doc/rune_reference.md file get out of date, so I regenerated the BNF like you did, and ran it through the web sites you linked to. I'll push the new docs today to github. The BNF now is able to be used to generate the railroad diagrams.

mingodad commented 1 year ago

It seems that it got a bit messed up here https://github.com/google/rune/blob/main/doc/rune_reference.md#grammar , see //!!!! marks bellow:

statements ::= | statements statement statement ::=  appendCode | assertStatement //!!!! here seems to be missing a newline
    | assignmentStatement | callStatement | class | debugStatement | enum | externFunction | finalFunction
    | foreachStatement | forStatement | function | generateStatement | generatorStatement | ifStatement
    | import | prependCode | printlnStatement | printStatement | refStatement | relationStatement
    | returnStatement | struct | switchStatement | throwStatement | unitTestStatement | unrefStatement
    | whileStatement | yield

accessExpression ::=  tokenExpression
    | accessExpression '(' callParameterList ')'
    | accessExpression '.' IDENT
    | accessExpression '[' expression ']'
    | accessExpression '[' expression ' ::= ' expression ']' //!!!! here should be ':' instead of ' ::= '

selectExpression ::=  orExpression
    | orExpression '?' orExpression ' ::= ' orExpression //!!!! here should be ':' instead of ' ::= '

mingodad commented 1 year ago

I've just added https://mingodad.github.io/parsertl-playground/playground/ an Yacc/Lex compatible online editor/tester (select Rune parser from Examples then click Parse to see a parser tree for the content in Input source editor).

I hope it can help debug/develop/test/document this project grammar !

Any feedback is welcome !

mingodad commented 5 months ago

I'm getting the error shown bellow when trying to run the hello.rn as shown in the readme.

./rune -g hello.rn 
hello.ll:9:45: error: expected ')' at end of argument list
declare i32 @_setjmp(%struct.__jmp_buf_tag* noundef)
                                            ^
1 error generated.

mingodad commented 5 months ago

The problem mentioned above was that I have clang-6 as default installed but rune is built with clang-15 and was not checking that it's executing the correct version of clang/llvm, maybe would be nice to check/save the clang/llvm version/location.

google / rune

Grammar railroad diagram #2