mike-lischke
commented
6 years ago Could you please strip down the log to the minimal possible size and summarize what exactly are we talking about here? What's doing wrong access here and where?
Open adarre opened 6 years ago
mike-lischke
commented
6 years ago Could you please strip down the log to the minimal possible size and summarize what exactly are we talking about here? What's doing wrong access here and where?
adarre
commented
6 years ago If I remember correctly the main problem was that there was use on a stack variable after it went out of scope. These functions are where it occurred.
5: RuleContext ParserRuleContext::addChild(RuleContext ruleInvocation) { children.push_back(ruleInvocation); return ruleInvocation; }
6: void Parser::unrollRecursionContexts(ParserRuleContext parentctx) { _precedenceStack.pop_back(); _ctx->stop = _input->LT(-1); ParserRuleContext retctx = _ctx; // save current ctx (return value)
// unroll so ctx is as it was before call to recursive method if (_parseListeners.size() > 0) { while (_ctx != parentctx) { triggerExitRuleEvent(); _ctx = dynamic_cast<ParserRuleContext *>(_ctx->parent); } } else { _ctx = parentctx; }
// hook into tree retctx->parent = parentctx;
if (_buildParseTrees && parentctx != nullptr) { // add return ctx into invoking rule's tree parentctx->addChild(retctx); } }
I believe somehow child went out of scope during the recursion process.
jasonar81
commented
4 years ago We fixed this on our side a while ago, but I upgraded ANTLR and was looking at our patches. We fixed this by changing all vectors of ParseTree* to deques. This changes a bunch of function signatures.
mike-lischke
commented
4 years ago How can this change help with this problem @jasonar81? The deque is only a container with slighty different access characteristics (insertion on both ends is O(1), no continous space to keep content).
jasonar81
commented
4 years ago Apologies. This was to solve a different problem we didn't open a ticket for. Where the vector resizing was causing a problem... a problem the deque doesn't have. Going back through ticket comments, this particular issue seems to have been caused be the some details of the combination of unrollRecursionContexts() and the finally lambdas. We solved this by rewriting it to get rid of lambdas.
Having said that, I have not seen this issue happen again since upgrading to 4.8 which I have not re-patched with that change of ours.
In a grammar with the following lexer token: EQUALS : '=' ; The input '==' causes ASAN to report a stack use after scope in antlr code. Here is the backtrace for error that was reported
==2730==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7fffdc931400 at pc 0x00000049eb99 bp 0x7fffdc9313b0 sp 0x7fffdc9313a8 READ of size 8 at 0x7fffdc931400 thread T29
0 0x49eb98 in unsigned long const& std::max(unsigned long const&, unsigned long const&) (/Workspace/dev/xgsrc/rolehostd/build/rolehostd+0x49eb98)
Address 0x7fffdc931400 is located in stack of thread T29 SUMMARY: AddressSanitizer: stack-use-after-scope (/Workspace/dev/xgsrc/rolehostd/build/rolehostd+0x49eb98) in unsigned long const& std::max(unsigned long const&, unsigned long const&)
Shadow bytes around the buggy address:
0x10007b91e230: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10007b91e240: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10007b91e250: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10007b91e260: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10007b91e270: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x10007b91e280:[f8]00 00 00 f1 f1 f1 f1 00 f2 f2 f2 f3 f3 f3 f3
0x10007b91e290: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10007b91e2a0: f1 f1 f1 f1 00 f2 f2 f2 f3 f3 f3 f3 00 00 00 00
0x10007b91e2b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10007b91e2c0: 00 00 00 00 f1 f1 f1 f1 00 f2 f2 f2 f3 f3 f3 f3
0x10007b91e2d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Thread T29 created by T12 here:
0 0x7ffff6e5ba30 in __interceptor_pthread_create /Workspace/dev/xgsrc/toolchain/gcc/gcc-7.1.0/libsanitizer/asan/asan_interceptors.cc:243
Thread T12 created by T0 here:
0 0x7ffff6e5ba30 in __interceptor_pthread_create /Workspace/dev/xgsrc/toolchain/gcc/gcc-7.1.0/libsanitizer/asan/asan_interceptors.cc:243
And here are the functions for stack frames 5-7:
5: RuleContext ParserRuleContext::addChild(RuleContext ruleInvocation) { children.push_back(ruleInvocation); return ruleInvocation; }
6: void Parser::unrollRecursionContexts(ParserRuleContext parentctx) { _precedenceStack.pop_back(); _ctx->stop = _input->LT(-1); ParserRuleContext retctx = _ctx; // save current ctx (return value)
// unroll so ctx is as it was before call to recursive method if (_parseListeners.size() > 0) { while (_ctx != parentctx) { triggerExitRuleEvent(); _ctx = dynamic_cast<ParserRuleContext *>(_ctx->parent); } } else { _ctx = parentctx; }
// hook into tree retctx->parent = parentctx;
if (_buildParseTrees && parentctx != nullptr) { // add return ctx into invoking rule's tree parentctx->addChild(retctx); } }
7: commandParser::FilterExpressionContext commandParser::filterExpression(int precedence) { ParserRuleContext parentContext = _ctx; size_t parentState = getState(); commandParser::FilterExpressionContext _localctx = _tracker.createInstance(_ctx, parentState);
commandParser::FilterExpressionContext previousContext = _localctx;
size_t startState = 68;
enterRecursionRule(_localctx, 68, commandParser::RuleFilterExpression, precedence);
auto onExit = finally([=] { unrollRecursionContexts(parentContext); }); try { size_t alt; enterOuterAlt(_localctx, 1); setState(533); _errHandler->sync(this); switch (getInterpreter()->adaptivePredict(_input, 77, _ctx)) {
case 1: {
_localctx = _tracker.createInstance(_localctx);
_ctx = _localctx;
previousContext = _localctx;
} catch (RecognitionException &e) { _errHandler->reportError(this, e); _localctx->exception = std::current_exception(); _errHandler->recover(this, _localctx->exception); } return _localctx; }