rubberduck-vba / Rubberduck

Every programmer needs a rubberduck. COM add-in for the VBA & VB6 IDE (VBE).
https://rubberduckvba.com
GNU General Public License v3.0
1.92k stars 301 forks source link

SLL parser fails on module with complex precompiler directives? #4204

Open comintern opened 6 years ago

comintern commented 6 years ago

I was looking for a beast of a module to use for isolating inspection issues and downloaded the spreadsheet from #2476. By complete coincidence, it contained the module (or at least the function) that @FreeMan was having issues with this morning in chat. I logged the following when it parsed:

2018-07-17 20:12:37.8699;WARN-2.2.6772.33906;Rubberduck.Parsing.VBA.VBAModuleParser;SLL mode failed while parsing the code pane version of module JsonConverter at symbol End Function at L1066C1. Retrying using LL.;
2018-07-17 20:12:37.8699;DEBUG-2.2.6772.33906;Rubberduck.Parsing.VBA.VBAModuleParser;SLL mode exception;Rubberduck.Parsing.Symbols.ParsingExceptions.MainParseSyntaxErrorException: mismatched input 'End Function' expecting {ABS, ANY, ARRAY, CBOOL, CBYTE, CCUR, CDATE, CDBL, CDEC, CINT, CLNG, CLNGLNG, CLNGPTR, CSNG, CSTR, CURRENCY, CVAR, CVERR, DEBUG, DOEVENTS, FIX, INPUTB, INT, LBOUND, LEN, LENB, LONGLONG, LONGPTR, MIDB, PSET, SGN, UBOUND, ACCESS, ADDRESSOF, ALIAS, AND, ATTRIBUTE, APPEND, AS, BEGINPROPERTY, BEGIN, BINARY, BOOLEAN, BYVAL, BYREF, BYTE, CLASS, CLOSE, DATABASE, DATE, DOUBLE, ELSE, ELSEIF, END_IF, ENDPROPERTY, END, EQV, ERROR, FALSE, GET, IMP, IN, INPUT, IS, INTEGER, LOCK, LONG, LIB, LIKE, LINE_INPUT, LOCK_READ, LOCK_WRITE, LOCK_READ_WRITE, ME, MID, MOD, NAME, NEW, NOT, NOTHING, NULL, OBJECT, ON_ERROR, OPEN, OPTIONAL, OR, OUTPUT, PARAMARRAY, PRESERVE, PRINT, PTRSAFE, PUT, RANDOM, RANDOMIZE, READ, READ_WRITE, REM, RESET, SEEK, SHARED, SINGLE, SPC, STEP, STRING, TAB, TEXT, THEN, TO, TRUE, TYPEOF, UNLOCK, UNTIL, VARIANT, VERSION, WIDTH, WITHEVENTS, WRITE, XOR, '[', OCTLITERAL, HEXLITERAL, FLOATLITERAL, INTEGERLITERAL, WS, IDENTIFIER, LINE_CONTINUATION} ---> Antlr4.Runtime.InputMismatchException: Exception of type 'Antlr4.Runtime.InputMismatchException' was thrown.
   at Antlr4.Runtime.DefaultErrorStrategy.Sync(Parser recognizer)
   at Rubberduck.Parsing.Grammar.VBAParser.ifStmt() in C:\Rubberduck\Rubberduck.Parsing\obj\Debug\VBAParser.cs:line 9074
   --- End of inner exception stack trace ---
   at Rubberduck.Parsing.Symbols.ParsingExceptions.MainParseExceptionErrorListener.SyntaxError(IRecognizer recognizer, IToken offendingSymbol, Int32 line, Int32 charPositionInLine, String msg, RecognitionException e) in C:\Rubberduck\Rubberduck.Parsing\Symbols\ParsingExceptions\MainParseExceptionErrorListener.cs:line 15
   at Antlr4.Runtime.ProxyErrorListener`1.SyntaxError(IRecognizer recognizer, Symbol offendingSymbol, Int32 line, Int32 charPositionInLine, String msg, RecognitionException e)
   at Antlr4.Runtime.Parser.NotifyErrorListeners(IToken offendingToken, String msg, RecognitionException e)
   at Rubberduck.Parsing.Grammar.VBAParser.ifStmt() in C:\Rubberduck\Rubberduck.Parsing\obj\Debug\VBAParser.cs:line 9158
   at Rubberduck.Parsing.Grammar.VBAParser.mainBlockStmt() in C:\Rubberduck\Rubberduck.Parsing\obj\Debug\VBAParser.cs:line 2900
   at Rubberduck.Parsing.Grammar.VBAParser.blockStmt() in C:\Rubberduck\Rubberduck.Parsing\obj\Debug\VBAParser.cs:line 2654
   at Rubberduck.Parsing.Grammar.VBAParser.block() in C:\Rubberduck\Rubberduck.Parsing\obj\Debug\VBAParser.cs:line 2495
   at Rubberduck.Parsing.Grammar.VBAParser.functionStmt() in C:\Rubberduck\Rubberduck.Parsing\obj\Debug\VBAParser.cs:line 8807
   at Rubberduck.Parsing.Grammar.VBAParser.moduleBodyElement() in C:\Rubberduck\Rubberduck.Parsing\obj\Debug\VBAParser.cs:line 2403
   at Rubberduck.Parsing.Grammar.VBAParser.moduleBody() in C:\Rubberduck\Rubberduck.Parsing\obj\Debug\VBAParser.cs:line 2334
   at Rubberduck.Parsing.Grammar.VBAParser.module() in C:\Rubberduck\Rubberduck.Parsing\obj\Debug\VBAParser.cs:line 444
   at Rubberduck.Parsing.Grammar.VBAParser.startRule() in C:\Rubberduck\Rubberduck.Parsing\obj\Debug\VBAParser.cs:line 332
   at Rubberduck.Parsing.VBA.VBAModuleParser.Parse(String moduleName, CommonTokenStream moduleTokens, IParseTreeListener[] listeners, BaseErrorListener errorListener) in C:\Rubberduck\Rubberduck.Parsing\VBA\VBAModuleParser.cs:line 25
Token: End Function at L1066C1
Component: JsonConverter (code pane version)
ParseType: Main parse

The full module is attached below (note that it is stored as .txt) to make sure it doesn't get "fixed" with a pull on either repo:

WARNING: If you download the workbook to test it in context, open it in without macros and comment out the auto open.

JsonConverter.bas

What makes this particularly interesting is how the precompiler directives are arranged at the the start of the module, because they confuse the hell out of the VBE. It highlights them as syntax errors, but it compiles cleanly:

wtfvbe

I think this raises the possibility that RD's parser is similarly confused, so this should be investigated.

daFreeMan commented 6 years ago

I can confirm that the "inappropriate platform" code is highlighted in red in my environment, as well.

Win10, Office2016 - the #If Win64 code looks fine, then #Else code is in red.

(I'd post an actual sample, but since we've all "upgraded" to Win10/O2016, there was no real need to keep it.)

bclothier commented 6 years ago

Just for record, as mentioned via chat, the red highlighting is expected behavior.

On VBA6 and prior hosts, the Declare statements with PtrSafe will light up red because it's unknown keyword. On 32-bit VBA7 hosts, both type of Declare statements are valid. On 64-bit VBA7 hosts, the Declare statements without PtrSafe will light up red because 64-bit VBA requires PtrSafe keyword to be present.

With correct compilation arguments, those won't be issues with compiling the codebase since the red section will also be dead.

MDoerner commented 6 years ago

Adjusted the title to indicate that only the SLL parser fails but not the LL one. Thus, this only impacts the parsing performance.