search

rubberduck-vba / Rubberduck

Every programmer needs a rubberduck. COM add-in for the VBA & VB6 IDE (VBE).
https://rubberduckvba.com
GNU General Public License v3.0
1.91k stars 299 forks source link

Access program with over a hundred forms. : System.OutOfMemoryException #4094

Closed thetopic closed 5 years ago

thetopic commented 6 years ago

Good Morning,

When I start processing at first it works, the processor is used at more than 75% (I have 8 cores) and writing to the log is done regularly (TRACE level enabled 1 second for a form) but memory consumption increases little by little (I have 16GB), but once the memory consumed by Access reaches 1GB the processor consumption stagnates at 13% and writing to the log is very slow (several minutes for a form) and I have an exception :

2018-06-15 09:43:48.2631;ERROR-2.2.6738.40126;Rubberduck.Parsing.VBA.ComponentParseTask; Unexpected exception thrown in thread 10 while parsing module Form_XXXXXXXXXXXXXX, ParseTaskID 651d8108-9641-4f3d-af93-9067ed1ad480.;System.OutOfMemoryException: Une exception de type 'System.OutOfMemoryException' a été levée.
   à System.Collections.Generic.Dictionary`2.Resize(Int32 newSize, Boolean forceNewHashCodes)
   à System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
   à Antlr4.Runtime.Atn.ATNConfigSet.Add(ATNConfig e, PredictionContextCache contextCache)
   à Antlr4.Runtime.Atn.ParserATNSimulator.Closure(ATNConfig config, ATNConfigSet configs, ATNConfigSet intermediate, HashSet`1 closureBusy, Boolean collectPredicates, Boolean hasMoreContexts, PredictionContextCache contextCache, Int32 depth, Boolean treatEofAsEpsilon)
   à Antlr4.Runtime.Atn.ParserATNSimulator.Closure(ATNConfig config, ATNConfigSet configs, ATNConfigSet intermediate, HashSet`1 closureBusy, Boolean collectPredicates, Boolean hasMoreContexts, PredictionContextCache contextCache, Int32 depth, Boolean treatEofAsEpsilon)
   à Antlr4.Runtime.Atn.ParserATNSimulator.Closure(ATNConfig config, ATNConfigSet configs, ATNConfigSet intermediate, HashSet`1 closureBusy, Boolean collectPredicates, Boolean hasMoreContexts, PredictionContextCache contextCache, Int32 depth, Boolean treatEofAsEpsilon)
   à Antlr4.Runtime.Atn.ParserATNSimulator.Closure(ATNConfig config, ATNConfigSet configs, ATNConfigSet intermediate, HashSet`1 closureBusy, Boolean collectPredicates, Boolean hasMoreContexts, PredictionContextCache contextCache, Int32 depth, Boolean treatEofAsEpsilon)
   à Antlr4.Runtime.Atn.ParserATNSimulator.Closure(ATNConfigSet sourceConfigs, ATNConfigSet configs, Boolean collectPredicates, Boolean hasMoreContext, PredictionContextCache contextCache, Boolean treatEofAsEpsilon)
   à Antlr4.Runtime.Atn.ParserATNSimulator.ComputeTargetState(DFA dfa, DFAState s, ParserRuleContext remainingGlobalContext, Int32 t, Boolean useContext, PredictionContextCache contextCache)
   à Antlr4.Runtime.Atn.ParserATNSimulator.ComputeReachSet(DFA dfa, SimulatorState previous, Int32 t, PredictionContextCache contextCache)
   à Antlr4.Runtime.Atn.ParserATNSimulator.ExecATN(DFA dfa, ITokenStream input, Int32 startIndex, SimulatorState initialState)
   à Antlr4.Runtime.Atn.ParserATNSimulator.ExecDFA(DFA dfa, ITokenStream input, Int32 startIndex, SimulatorState state)
   à Antlr4.Runtime.Atn.ParserATNSimulator.AdaptivePredict(ITokenStream input, Int32 decision, ParserRuleContext outerContext, Boolean useContext)
   à Antlr4.Runtime.Atn.ParserATNSimulator.ExecDFA(DFA dfa, ITokenStream input, Int32 startIndex, SimulatorState state)
   à Antlr4.Runtime.Atn.ParserATNSimulator.AdaptivePredict(ITokenStream input, Int32 decision, ParserRuleContext outerContext, Boolean useContext)
   à Antlr4.Runtime.Atn.ParserATNSimulator.AdaptivePredict(ITokenStream input, Int32 decision, ParserRuleContext outerContext)
   à Rubberduck.Parsing.Grammar.VBAParser.whiteSpace() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 21401
   à Rubberduck.Parsing.Grammar.VBAParser.endOfLine() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 20347
   à Rubberduck.Parsing.Grammar.VBAParser.endOfStatement() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 20445
   à Rubberduck.Parsing.Grammar.VBAParser.block() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 2101
   à Rubberduck.Parsing.Grammar.VBAParser.withStmt() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 14089
   à Rubberduck.Parsing.Grammar.VBAParser.mainBlockStmt() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 2628
   à Rubberduck.Parsing.Grammar.VBAParser.blockStmt() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 2259
   à Rubberduck.Parsing.Grammar.VBAParser.block() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 2100
   à Rubberduck.Parsing.Grammar.VBAParser.ifStmt() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 8667
   à Rubberduck.Parsing.Grammar.VBAParser.mainBlockStmt() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 2502
   à Rubberduck.Parsing.Grammar.VBAParser.blockStmt() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 2259
   à Rubberduck.Parsing.Grammar.VBAParser.block() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 2100
   à Rubberduck.Parsing.Grammar.VBAParser.caseClause() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 12186
   à Rubberduck.Parsing.Grammar.VBAParser.selectCaseStmt() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 11983
   à Rubberduck.Parsing.Grammar.VBAParser.mainBlockStmt() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 2593
   à Rubberduck.Parsing.Grammar.VBAParser.blockStmt() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 2259
   à Rubberduck.Parsing.Grammar.VBAParser.block() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 2100
   à Rubberduck.Parsing.Grammar.VBAParser.subStmt() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 12669
   à Rubberduck.Parsing.Grammar.VBAParser.moduleBodyElement() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 2036
   à Rubberduck.Parsing.Grammar.VBAParser.moduleBody() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 1939
   à Rubberduck.Parsing.Grammar.VBAParser.module() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 443
   à Rubberduck.Parsing.Grammar.VBAParser.startRule() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 331
   à Rubberduck.Parsing.VBA.VBAModuleParser.Parse(String moduleName, CommonTokenStream moduleTokens, IParseTreeListener[] listeners, BaseErrorListener errorListener) dans C:\projects\rubberduck\Rubberduck.Parsing\VBA\VBAModuleParser.cs:ligne 37
   à Rubberduck.Parsing.VBA.AttributeParser.Parse(QualifiedModuleName module, CancellationToken cancellationToken) dans C:\projects\rubberduck\Rubberduck.Parsing\VBA\AttributeParser.cs:ligne 85
   à Rubberduck.Parsing.VBA.ComponentParseTask.RunAttributesPass(CancellationToken cancellationToken) dans C:\projects\rubberduck\Rubberduck.Parsing\VBA\ComponentParseTask.cs:ligne 151
   à Rubberduck.Parsing.VBA.ComponentParseTask.Start(CancellationToken cancellationToken) dans C:\projects\rubberduck\Rubberduck.Parsing\VBA\ComponentParseTask.cs:ligne 56
2018-06-15 09:43:48.2631;DEBUG-2.2.6738.40126;Rubberduck.Parsing.VBA.RubberduckParserState;Module 'Form_XXXXXX' state is changing to 'Error' (thread 10);

Othe thing, I don't know if this is normal but in the log when TRACE is enabled I have hundreds of errors of this type:

2018-06-15 09:24:01.4201;DEBUG-2.2.6738.40126;Rubberduck.Parsing.VBA.VBAModuleParser;SLL mode exception;Rubberduck.Parsing.Symbols.ParsingExceptions.MainParseSyntaxErrorException: mismatched input '[' expecting {<EOF>, ':', REM, NEWLINE, ''', WS, LINE_CONTINUATION} ---> Antlr4.Runtime.InputMismatchException: Une exception de type 'Antlr4.Runtime.InputMismatchException' a été levée.
   à Antlr4.Runtime.DefaultErrorStrategy.Sync(Parser recognizer)
   à Rubberduck.Parsing.Grammar.VBAParser.endOfStatement() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 20428
   --- Fin de la trace de la pile d'exception interne ---
   à Rubberduck.Parsing.Symbols.ParsingExceptions.MainParseExceptionErrorListener.SyntaxError(IRecognizer recognizer, IToken offendingSymbol, Int32 line, Int32 charPositionInLine, String msg, RecognitionException e) dans C:\projects\rubberduck\Rubberduck.Parsing\Symbols\ParsingExceptions\MainParseExceptionErrorListener.cs:ligne 15
   à Antlr4.Runtime.ProxyErrorListener`1.SyntaxError(IRecognizer recognizer, Symbol offendingSymbol, Int32 line, Int32 charPositionInLine, String msg, RecognitionException e)
   à Antlr4.Runtime.Parser.NotifyErrorListeners(IToken offendingToken, String msg, RecognitionException e)
   à Antlr4.Runtime.DefaultErrorStrategy.NotifyErrorListeners(Parser recognizer, String message, RecognitionException e)
   à Antlr4.Runtime.DefaultErrorStrategy.ReportInputMismatch(Parser recognizer, InputMismatchException e)
   à Antlr4.Runtime.DefaultErrorStrategy.ReportError(Parser recognizer, RecognitionException e)
   à Rubberduck.Parsing.Grammar.VBAParser.endOfStatement() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 20515
   à Rubberduck.Parsing.Grammar.VBAParser.block() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 2101
   à Rubberduck.Parsing.Grammar.VBAParser.subStmt() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 12669
   à Rubberduck.Parsing.Grammar.VBAParser.moduleBodyElement() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 2036
   à Rubberduck.Parsing.Grammar.VBAParser.moduleBody() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 1939
   à Rubberduck.Parsing.Grammar.VBAParser.module() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 443
   à Rubberduck.Parsing.Grammar.VBAParser.startRule() dans C:\projects\rubberduck\Rubberduck.Parsing\obj\Release\VBAParser.cs:ligne 331
   à Rubberduck.Parsing.VBA.VBAModuleParser.Parse(String moduleName, CommonTokenStream moduleTokens, IParseTreeListener[] listeners, BaseErrorListener errorListener) dans C:\projects\rubberduck\Rubberduck.Parsing\VBA\VBAModuleParser.cs:ligne 25
Token: [ at L117C30
Component: Form_XXXXXXXX (code pane version)
ParseType: Main parse

It matches this type of code: Rst![Field_To_Retrieve]

Version 2.2.6738.40126 Système d'exploitation : Microsoft Windows NT 6.1.7601 Service Pack 1, x64 Produit hôte : Microsoft Office XP x86 Version hôte : 10.0.2627 Exécutable hôte : MSACCESS.EXE

retailcoder commented 6 years ago

SLL mode failures mean that the parser's faster mode has failed and the parsing of the module is being re-attempted with the slower LL mode. rst![field name] syntax is shorthand for the longer but more explicit rst.Fields("field name").Value syntax, which shouldn't fail to parse with SLL. A lot of care was taken in writing the grammar in such a way that SLL mode would be used as much as possible, however it's not always possible... the bang operator is one of these cases.

We do need quite a bit of memory, since we've prioritized performance over memory savings, so a lot of caching is going on... we do have ideas to improve the memory footprint without hurting performance too much, but the changes required are massive and we're just not there yet.

I've got Rubberduck working fine against a 90-module Excel project, consuming roughly 400MB RAM, but that's on 64-bit Win7 in Office 2010 x86... 100 Access forms might be stretching it, depending on the code-behind and number of controls in each one.

That said, nice to see ducky "working" in OfficeXP! Does it work decently in a new project?

retailcoder commented 6 years ago

Linking #3347

thetopic commented 6 years ago

For information :

MDoerner commented 6 years ago

Just for reference why an out of memory error occurs for much less memory consumption than the memory you have installed. Since the Office running is a 32 bit application, it will never see more than a little short of 2 GB of memory.

More precisely, it will not be able to use more memory for two reasons. First, the developers of Office apparently did not set the LARGEADDRESSAWARE flag. (Otherwise, the limit would be 4 GB on a 64bit system.) Second, we do not go through the hoops to map address spaces to other memory regions and swap the address spaces as needed, which is rather tedious and error prone.

bclothier commented 5 years ago

Closing this in favor of tracking the issue previously referenced (#3347 )