error recovery is poorly documented

[chrismacklin]

Documentation for options for parse error handling and recovery are nonexistent. When a parse error occurs, what happens? What options are available for handling this error? How can I provide custom behavior for the exception that is raised when a parse error occurs?

[artempyanykh]

Faced a similar problem. I'm trying to use FsLexYacc in my project, and although I can draw some parallels with OCaml's tooling or lex/yacc this helps only to some extent and I still can't figure out how to do high fidelity error handling.

Of course, a Menhir-like manual on error recovery would be great, but even an real-world example parser (non-toy with real error handling) would be of great help: examples I have found in the repo and the docs are not complex enough and/or don't have any error recovery. So, any links would be appreciated.

[daz10000]

I would second this. When I’ve tried to do it, I guessed the conventions from other implementations . error case in the match ? Underscore like you would in F# ? My next step was to try to read the F# source code as an intro :). Even one example would be transformative

Darren

Am 2/11/22 um 2:49 AM schrieb Artem Pyanykh @.***>:

 Faced a similar problem. I'm trying to use FsLexYacc in my project, and although I can draw some parallels with OCaml's tooling or lex/yacc this helps only to some extent and I still can't figure out how to do high fidelity error handling.

Of course, a Menhir-like manual on error recovery would be great, but even an real-world example parser (non-toy with real error handling) would be of great help: examples I have found in the repo and the docs are not complex enough and/or don't have any error recovery. So, any links would be appreciated.

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you are subscribed to this thread.

[robreno]

First off, I love F# and FsLexYacc. I am working on creating a query engine (lexer/parser) for a custom query language for a 2091 page text I have reverse indexed. I have successfully created a very simply query language (lexer/parser) that works really well.

I too have encountered this problem and trying to solve it right now. I downloaded the solution/projects and have them compiling locally (getting some interesting warnings that might be bugs, like inconsistent parameter names stuff. Will document more fully and when I figure out who to ask if this is a real issue will do so). I am too afraid to do it on Git right now as I am really new to using Git and don't want to offend anyone or do something stupid. But I entered a Term with the wrong syntax (paraid instead of parid) and threw an exception. Stepping into the QueryParser.fsi file I set a breakpoint at :

// Implementation file for parser generated by fsyacc
parseError = (fun (ctxt:FSharp.Text.Parsing.ParseErrorContext<_>) -> 
                              match parse_error_rich with 
                              | Some f -> f ctxt
                              | None -> parse_error ctxt.Message);

The ctxt.Message is "syntax error", but of course this does not propagate error message "syntax error" to the exception handler in the F# code I am calling the parser from. Or at least I don't see it or know how to get it, but that would be very useful as a syntax error is good to know in a query so I can handle it correctly in my application. I really want to figure out how to propagate a more verbose error message like "syntax error" in such a case.

Here is all the error context I can get in my code:

System.Exception
  HResult=0x80131500
  Message=parse error
  Source=FsLexYacc.Runtime

This exception was originally thrown at this call stack:
    [External Code]
    UBViews.Query.Parser.engine<a>(Microsoft.FSharp.Core.FSharpFunc<FSharp.Text.Lexing.LexBuffer<a>, UBViews.Query.Parser.token>, FSharp.Text.Lexing.LexBuffer<a>, int) in QueryParser.fs
    UBViews.Query.Parser.start<a>(Microsoft.FSharp.Core.FSharpFunc<FSharp.Text.Lexing.LexBuffer<a>, UBViews.Query.Parser.token>, FSharp.Text.Lexing.LexBuffer<a>) in QueryParser.fs
    UBViews.Query.Local.SimpleParse.parseQuery(string) in SimpleParse.fs

I found the following in the FsLexYacc.Runtime:

#if INTERNALIZED_FSLEXYACC_RUNTIME
module internal ParseHelpers = 
#else
module ParseHelpers = 
#endif
    let parse_error (_s:string) = ()
    let parse_error_rich = (None : (ParseErrorContext<_> -> unit) option)

I would like to know how to get access to the ParseHelpers which may allow access to parse_error_rich which may carry the correct error message.

[fxf06]

Got similar problem on how error is handled. Though is supposed to behaves like Ocamlyacc, it is hard to handle the "error" token right. Regarding the error message, the parse_error_rich function is defined to None in the source. I did this is my code:

let my_parse_error (ctx:ParseErrorContext<_>) =
    let debugText = ref ""
    let (startPos, endPos) = ctx.ParseState.ResultRange
    debugText.Value <- debugText.Value + $"{startPos.FileName}({startPos.Line},{startPos.Column}): Parse error: token: '{ctx.CurrentToken}'\n"
    let sp =
        {
            CommentStart = -1;
            CommentEnd   = -1;
            CommentCol   = -1;
            SourceStart  = startPos.AbsoluteOffset;
            SourceEnd    = endPos.AbsoluteOffset;
            SourceCol    = startPos.Column;
            LineStart    = startPos.Line;
            LineEnd      = endPos.Line
        }
    debugText.Value <- debugText.Value + $"Input:\n{sp.fragment()}\n"
    debugText.Value <- debugText.Value + $"ReduceTokens        : {ctx.ReduceTokens}\n"
    debugText.Value <- debugText.Value + $"ReducibleProductions: {ctx.ReducibleProductions}\n"
    debugText.Value <- debugText.Value + $"ShiftTokens         : {ctx.ShiftTokens}\n"
    debugText.Value <- debugText.Value + $"StateStack          : {ctx.StateStack}\n"
    ParserTools.SetParseErrorContext debugText.Value

let parse_error_rich = Some my_parse_error

which is:

1. define your parser error function
2. set parser_error_rich to Some "your_function"

I used a reference in a other module to get the data once the parser engine returns.

What is also unfortunate is that Fsyacc skips anything below the last "%%", which is normally code that would be simply added at end of the generated F# code. Then, we could have access also to everything defined like tokens values, rules values, etc...

[mcgowanr-mtw]

Could we get an update on this?

The fsyacc page does give us RaiseError as part of the parseState and then says "You can call RaiseError if you like." Is that what I'm supposed to use for parser errors? If so, how would I do that?

[daz10000]

Put me in the loves fsyacc category and I maintain a huge DNA compiler using it.. That said, I never worked out how to do error handling :( My vague recollection of the journey was

• for fsyacc docs... see ocaml docs
• ocaml docs.. see yacc docs
• tries the error token - strange errors. Maybe it's the _ pattern... ? stranger errors..
• looked at the F# compiler source 😢 - too complex

I'd weep for even 5 mins of typed simple examples showing how to intercept lack rules where none of the expected tokens match..

To keep this more concrete, I want to be add something like an | error -> { throw an expected X, Y,Z message here} to a block like this., Apologies if it all works now, but I got nowhere when I last gave up,

Exp:
    | IntLiteral                    { $1 }
    | StringLiteral                 { $1 }
    | FloatLiteral                  { $1 }
    | VARIABLE                      { (tokenToVariable $1 NotYetTyped) }
    | LPAREN Exp RPAREN             { $2 }
    | HYPHEN Exp %prec UMINUS       { (negate $2) }
    | Exp STAR Exp                  { (createBinaryOp Multiply $1 $3) }
    | Exp SLASH Exp                 { (createBinaryOp Divide $1 $3) }
    | Exp PLUS Exp                  { (createBinaryOp Add $1 $3) }
    | Exp HYPHEN Exp                { (createBinaryOp Subtract $1 $3) }
  (* want to add  a handler here for anything else*)

Searching in 2023 gets me this question on stack overflow which is the second hit (first is how to calculate line numbers after wrapping whole parser in a try/with block). There must be something more sophisticated? (if not, that's probably good to know too).