how to get partial results on error?

[akonsu]

On a parse error, Parser.results is undefined. Is there a way to get partial results for the part of the input that has been matched?

[akonsu]

I am considering using nearley in a code editor, and as the user types, I need to check for syntax errors. The current expression will always generate a parse error, until the user completes the expression. I suppose I am asking whether there is a way to at least recognize this specific case...

[akonsu]

BTW, I found a couple of papers on error recovery in Earley:

• S. O. Anderson and R. C. Backhouse. 1981. Locally Least-Cost Error Recovery in Earley's Algorithm. ACM Trans. Program. Lang. Syst. 3, 3 (July 1981), 318-347.
• Gordon Lyon. 1974. Syntax-directed least-errors analysis for context-free languages: a practical approach. Commun. ACM 17, 1 (January 1974), 3-14.

I do not know if you have seen these. The second one is available for free on the web. To get the first one, one would need to pay to ACM : )

[tjvr]

You've mentioned a couple of problems here:

• Partial parses, where the input up to the cursor is valid syntax, just incomplete;
• Error recovery, where the parser can "skip over" syntax errors and continue parsing.

The former is certainly possible, but it requires a lot of effort and good knowledge of how the Earley algorithm works! You can inspect Nearley's parse table to work out what the current state of parsing is. This is more practical than some parsers (e.g. PLY), since Earley is a chart parser; all the context you need for working out where the parsing got up to is still available. However, it's still fairly complicated to interpret.

FWIW, I wrote tosh, an Earley-based code editor including a particular sort of autocomplete (I use a funky trick to take into account the entire line, both before and after the cursor). This is how I know it's possible! I've been planning to package this work into a more general-purpose thing, "nearley-codemirror", for a while, as part of tosh2; but who knows when that will happen. :)

[kasbah]

The way I did this for Electro Grammar is to set keepHistory to true and roll it back when it fails. I am even working on recovering any ignored input in another branch.

[akonsu]

Thanks! This helps.

Electro Grammar code is quite easy to read, thanks!

But I need help with tosh2. @tjvr could you explain your code a bit? I see Highlighter and Completer (https://github.com/tjvr/tosh2/blob/master/editor/mode.js) How do they work? Also, how to run tosh2?

[tjvr]

The way I did this for Electro Grammar is to set keepHistory to true and roll it back when it fails

FWIW, a couple of comments:

• For future-proofing, you should use Grammar.fromCompiled, and not access ParserRules / ParserStart yourself
• If you're only ever using save(), you shouldn't need keepHistory :-)

---

But I need help with tosh2. @tjvr could you explain your code a bit? I see Highlighter and Completer (https://github.com/tjvr/tosh2/blob/master/editor/mode.js)

Nothing to see here, I'm afraid; that's very much work-in-progress. I mentioned Tosh only to show that what you want to do is theoretically possible.

[kasbah]

Thanks for the tips @tjvr, I would have never guessed that keepHistory wasn't required for save() on my own. I was going to say we should probably fix the save() documentation but I actually can't find it anywhere :open_mouth:! I think I just copied the usage from the unit tests.

[akonsu]

ok.. I looked at the Parser.table, it is more or less clear to me, except for the wants and scannable properties of columns. Could you clarify?

[tjvr]

I was going to say we should probably fix the save() documentation but I actually can't find it anywhere

Yeah, it's funky new stuff we haven't got round to documenting yet :-)

[tjvr]

ok.. I looked at the Parser.table, it is more or less clear to me, except for the wants and scannable properties of columns. Could you clarify?

wants and scannable probably aren't useful to you.

You probably want to iterate over the list of states on the final Column. The rule and reference properties tell you what grammar productions are being considered, and how far through the rule you are (State::toString() can be useful for debugging). You might use the left and right properties to access earlier States in the parse, which seems to be what I use for my Highlighter.

That's all I can give for now, sorry :)

[PhillippOhlandt]

Has there been any updates on the autocomplete front? Doesn't look like so and all issues I've found (like this one) are years old. The recommendations given aren't really useful for me at the moment. The list of states are a wild mess of data that are not easy to make sense of.

At this point, my plan is to use whatever incomplete rule I am in right now and do the manual part of deciding what inputs are accepted next, as opposed to have nearley tell me that. But I don't even see the current (partially) matched rule in the parser state.

ping @tjvr