search

antlr / antlr4

ANTLR (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing, or translating structured text or binary files.
http://antlr.org
BSD 3-Clause "New" or "Revised" License
17.23k stars 3.29k forks source link

[Cpp] crash in multi-threading #2004

Open chund opened 7 years ago

chund commented 7 years ago

I see ANTLR 4.7 CPP target crash, when running the parser in multiple threads. It is probably in warm-up phase. I am using a test scenario with concurrency::parallel_for() to parse multiple files in parallel. (4 threads on 4 core CPU)

I guess the critical piece of code is this, but I have no clue what is wrong and how to fix:

DefaultErrorStrategy::sync(Parser *recognizer) {
...
  auto nextTokens = recognizer->getATN().nextTokens(s);

The call stacks looks like this:

Thread 1 (which crashed): antlr4-runtime.dll!std::allocator::destroy(antlr4::misc::Interval _Ptr) Line 607 antlr4-runtime.dll!std::allocator_traits<std::allocator >::destroy(std::allocator & _Al, antlr4::misc::Interval _Ptr) Line 731 antlr4-runtime.dll!std::_Wrap_alloc<std::allocator >::destroy(antlr4::misc::Interval _Ptr) Line 879 antlr4-runtime.dll!std::_Destroy_range<std::_Wrap_alloc<std::allocator > >(antlr4::misc::Interval _First, antlr4::misc::Interval _Last, std::_Wrap_alloc<std::allocator > & _Al, std::_Nonscalar_ptr_iterator_tag __formal) Line 82 antlr4-runtime.dll!std::_Destroy_range<std::_Wrap_alloc<std::allocator > >(antlr4::misc::Interval _First, antlr4::misc::Interval _Last, std::_Wrap_alloc<std::allocator > & _Al) Line 96 antlr4-runtime.dll!std::vector<antlr4::misc::Interval,std::allocator >::_Destroy(antlr4::misc::Interval _First, antlr4::misc::Interval _Last) Line 1567 antlr4-runtime.dll!std::vector<antlr4::misc::Interval,std::allocator >::operator=(const std::vector<antlr4::misc::Interval,std::allocator > & _Right) Line 968 antlr4-runtime.dll!antlr4::misc::IntervalSet::operator=(const antlr4::misc::IntervalSet & __that) antlr4-runtime.dll!antlr4::atn::ATN::nextTokens(antlr4::atn::ATNState s) Line 92 C++ Symbols loaded. antlr4-runtime.dll!antlr4::DefaultErrorStrategy::sync(antlr4::Parser * recognizer) Line 103 myParser.dll ....

Thread 2:
antlr4-runtime.dll!antlr4::misc::IntervalSet::addAll(const antlr4::misc::IntervalSet & set) Line 138
antlr4-runtime.dll!antlr4::misc::IntervalSet::IntervalSet(const antlr4::misc::IntervalSet & set) Line 38
antlr4-runtime.dll!antlr4::DefaultErrorStrategy::sync(antlr4::Parser * recognizer) Line 103
myParser.dll ....

Thread 3:
antlr4-runtime.dll!antlr4::misc::IntervalSet::addAll(const antlr4::misc::IntervalSet & set) Line 138
antlr4-runtime.dll!antlr4::misc::IntervalSet::IntervalSet(const antlr4::misc::IntervalSet & set) Line 38
antlr4-runtime.dll!antlr4::DefaultErrorStrategy::sync(antlr4::Parser * recognizer) Line 103
myParser.dll ....

Thread 4:
antlr4-runtime.dll!antlr4::misc::IntervalSet::addAll(const antlr4::misc::IntervalSet & set) Line 138
antlr4-runtime.dll!antlr4::misc::IntervalSet::IntervalSet(const antlr4::misc::IntervalSet & set) Line 38
antlr4-runtime.dll!antlr4::DefaultErrorStrategy::sync(antlr4::Parser * recognizer) Line 103
chund commented 7 years ago

alternatively (multi-threading errors are not that deterministic) I also see this debug error message "vector iterator not incrementable" given by the std::vector iterator

with this call stack: msvcr120d.dll!_CrtDbgBreak() Line 87 msvcr120d.dll!_VCrtDbgReportW(int nRptType, void returnAddress, const wchar_t szFile, int nLine, const wchar_t szModule, const wchar_t szFormat, char arglist) Line 506 msvcr120d.dll!_CrtDbgReportWV(int nRptType, void returnAddress, const wchar_t szFile, int nLine, const wchar_t szModule, const wchar_t szFormat, char arglist) Line 262 msvcr120d.dll!_CrtDbgReportW(int nRptType, const wchar_t szFile, int nLine, const wchar_t szModule, const wchar_t szFormat, ...) Line 279 msvcp120d.dll!std::_Debug_message(const wchar_t message, const wchar_t file, unsigned int line) Line 13 antlr4-runtime.dll!std::_Vector_const_iterator<std::_Vector_val<std::_Simple_types > >::operator++() Line 101 antlr4-runtime.dll!antlr4::misc::IntervalSet::addAll(const antlr4::misc::IntervalSet & set) Line 138 antlr4-runtime.dll!antlr4::misc::IntervalSet::IntervalSet(const antlr4::misc::IntervalSet & set) Line 38 antlr4-runtime.dll!antlr4::DefaultErrorStrategy::sync(antlr4::Parser recognizer) Line 103