C++ runtime: For UNWANTED_TOKEN_EXCEPTION, expected token is printed instead of seen token.

[nyh]

The ScyllaDB project uses Antlr3 with the C++ runtime. In https://github.com/scylladb/scylladb/issues/1703 it was noted that ugly error messages are reported. One of the strange error messages is:

line 1:71 extraneous input 'STORAGE' expecting <invalid>

This is ScyllaDB's own (broken in its own way) error message printer, but when I try to use displayRecognitionError() from /usr/include/antlr3exception.inl, it's even worse - only this wrong "Expected" is printed. A snippted from that function:

        case UNWANTED_TOKEN_EXCEPTION:
                ...
                                str_stream << " : Extraneous input - expected "
                                                   << tokenNames[ BaseType::m_expecting] << " ...\n";

So in UNWANTED_TOKEN_EXCEPTION this function prints, instead of the seen (and unwanted) token, the "expected" token, which isn't available in the case of this exception, which is why it's printed as "<invalid">. What needs to be printed in this case is the seen token, not the expected token.

[ibre5041]

Try to use runtime from this branch. https://github.com/ibre5041/antlr3/commits/master So time ago I fixed a some bugs in C++ runtime.

Also try to use this fixed version of antlr tool. It generates much faster C++ code(faster than Java) https://github.com/tora-tool/tora/tree/master/extlibs/parsing.cpp

Long story short, Java string templates generate conditions like: if (a == 'c' && x.predicate())

While C/C++ parser code has condition order reversed: if (x.predicate() && a=='c')