mna / pigeon

Command pigeon generates parsers in Go from a PEG grammar.
BSD 3-Clause "New" or "Revised" License
822 stars 66 forks source link

trouble with \n and EOF #104

Closed gaissmai closed 1 year ago

gaissmai commented 1 year ago

HI, sorry but I know no other channel for pigeon. Please help me with a stupid problem. I stripped the grammar down as tiny as possible to hightlight my problem:

File  <- Line*  EOF
Line  <- [^\n]* EOL
EOL   <- '\n'
EOF   <- !.

If I feed the parser with text without a newline at the end, I get an expected parsing error:

$ echo -n "asdf" | ./main
ParseErr: STDIN:1:5 (4): no match found, expected: "\n" or [^\n]

but if I modify the EOL rule to:

File  <- Line*  EOF
Line  <- [^\n]* EOL
EOL   <- '\n' / EOF 
EOF   <- !.

the parser is stuck in a recursive endless loop. With Debug and MaxExpression options i can see the following:

$ echo "asdf" | ./main
ParseErr: STDIN:2:1 (5): rule EOL: max number of expresssions parsed

and in the Debug Output I find the next rune as EOF (U+FFFD '�', with width 0)

Any hint appreciated how can I solve this problem.

Best Regards Charly

gaissmai commented 1 year ago

solved, sorry for the line noise, see the + instead of * at the Line rule:

File  <- Line*  EOF
Line  <- [^\n]+ EOL
EOL   <- '\n' / EOF 
EOF   <- !.
gaissmai commented 1 year ago

closing this issue with a working grammar for a line scanner:

File  <- Line* EOF
Line  <- EOL / (Char+  EOL?)

Char  <- [^\n\r]
EOL   <- '\n' / "\r\n"
EOF   <- !.