Open llvmbot opened 6 years ago
Any news on this, or with which clang versions it happens?
I see the code in KDevelop calls the crashing path when CINDEX_VERSION_MINOR<100, but maybe it should also check the major version (and possibly make this a runtime check as far as that's possible)?
35485 is actually a different bug, this one here happens even with UserFilesAreVolatile set to true.
Actually, the crash also happens without our call to clang_Cursor_getParsedComment, just more rarely. Trace below.
Thread 260 "Queue(0x5555562" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fff467fc700 (LWP 20315)]
clang::comments::(anonymous namespace)::findCCommentEnd (BufferEnd=
Just reverted the workarounds for this libclang bug in KDevelop and reproduced the crash on current KDevelop master branch and almost current (2022-12-10) LLVM main branch. Opened the ktexteditor project in KDevelop, opened a file that was substantially modified between two far apart revisions A
and B
. Run for i in {1..40}; do git checkout A; sleep 1; git checkout B; sleep 1; done
in KDevelop's Terminal tool view. The fresh backtrace thus produced:
#0 ??() at /usr/lib/libc.so.6
#1 raise() at /usr/lib/libc.so.6
#2 abort() at /usr/lib/libc.so.6
#3 ??() at /usr/lib/libc.so.6
#4 __assert_fail() at /usr/lib/libc.so.6
#5 clang::comments::Lexer::lex() at llvm-project/clang/lib/AST/CommentLexer.cpp:754
#6 clang::comments::Parser::consumeToken() at llvm-project/clang/include/clang/AST/CommentParser.h:62
#7 clang::RawComment::parse() at llvm-project/clang/include/clang/AST/ASTContext.h:695
#8 clang::ASTContext::getCommentForDecl() at llvm-project/clang/lib/AST/ASTContext.cpp:662
#9 clang_Cursor_getParsedComment() at llvm-project/clang/tools/libclang/CXComment.cpp:36
#10 (anonymous namespace)::Visitor::setDeclData<(CXCursorKind)21>() at kdevelop/plugins/clang/duchain/builder.cpp:1020
#11 (anonymous namespace)::Visitor::setDeclData<(CXCursorKind)21>() at kdevelop/plugins/clang/duchain/builder.cpp:1139
#12 (anonymous namespace)::Visitor::setDeclData<(CXCursorKind)21>() at kdevelop/plugins/clang/duchain/builder.cpp:1212
#13 (anonymous namespace)::Visitor::createDeclarationCommon<(CXCursorKind)21, KDevelop::ClassFunctionDeclaration>() at kdevelop/plugins/clang/duchain/builder.cpp:431
#14 (anonymous namespace)::Visitor::createDeclaration<(CXCursorKind)21, KDevelop::ClassFunctionDeclaration>() at kdevelop/plugins/clang/duchain/builder.cpp:456
#15 (anonymous namespace)::Visitor::buildDeclaration<(CXCursorKind)21, KDevelop::ClassFunctionDeclaration, true>() at kdevelop/plugins/clang/duchain/builder.cpp:1285
#16 (anonymous namespace)::Visitor::dispatchCursor<(CXCursorKind)21, (Decision)0, (Decision)1>() at kdevelop/plugins/clang/duchain/builder.cpp:1007
#17 (anonymous namespace)::Visitor::dispatchCursor<(CXCursorKind)21, (Decision)0, (Decision)2>() at kdevelop/plugins/clang/duchain/builder.cpp:983
#18 (anonymous namespace)::Visitor::dispatchCursor<(CXCursorKind)21>() at kdevelop/plugins/clang/duchain/builder.cpp:970
#19 (anonymous namespace)::visitCursor() at kdevelop/plugins/clang/duchain/builder.cpp:1679
#20 clang::cxcursor::CursorVisitor::Visit() at llvm-project/clang/tools/libclang/CIndex.cpp:221
#21 clang::cxcursor::CursorVisitor::handleDeclForVisitation() at llvm-project/clang/tools/libclang/CIndex.cpp:683
#22 clang::cxcursor::CursorVisitor::VisitDeclContext() at llvm-project/clang/tools/libclang/CIndex.cpp:644
#23 clang::cxcursor::CursorVisitor::VisitChildren() at llvm-project/clang/tools/libclang/CIndex.cpp:510
#24 clang_visitChildren() at llvm-project/clang/tools/libclang/CIndex.cpp:4,772
#25 (anonymous namespace)::Visitor::buildDeclaration<(CXCursorKind)4, KDevelop::ClassDeclaration, true>() at kdevelop/plugins/clang/duchain/builder.cpp:1288
#26 (anonymous namespace)::Visitor::dispatchCursor<(CXCursorKind)4, (Decision)1, (Decision)0>() at kdevelop/plugins/clang/duchain/builder.cpp:1007
#27 (anonymous namespace)::Visitor::dispatchCursor<(CXCursorKind)4, (Decision)1, (Decision)2>() at kdevelop/plugins/clang/duchain/builder.cpp:982
#28 (anonymous namespace)::Visitor::dispatchCursor<(CXCursorKind)4>() at kdevelop/plugins/clang/duchain/builder.cpp:971
#29 (anonymous namespace)::visitCursor() at kdevelop/plugins/clang/duchain/builder.cpp:1671
#30 clang::cxcursor::CursorVisitor::Visit() at llvm-project/clang/tools/libclang/CIndex.cpp:221
#31 clang::cxcursor::CursorVisitor::handleDeclForVisitation() at llvm-project/clang/tools/libclang/CIndex.cpp:683
#32 clang::cxcursor::CursorVisitor::VisitDeclContext() at llvm-project/clang/tools/libclang/CIndex.cpp:644
#33 clang::cxcursor::CursorVisitor::VisitChildren() at llvm-project/clang/tools/libclang/CIndex.cpp:544
#34 clang_visitChildren() at llvm-project/clang/tools/libclang/CIndex.cpp:4,772
#35 (anonymous namespace)::Visitor::Visitor() at kdevelop/plugins/clang/duchain/builder.cpp:1590
#36 Builder::visit() at kdevelop/plugins/clang/duchain/builder.cpp:1737
#37 ClangHelpers::buildDUChain(void*, QMultiHash<void*, Import> const&, ParseSession const&, QFlags<KDevelop::TopDUContext::Feature>, QHash<void*, KDevelop::ReferencedTopDUContext>&, QHash<KDevelop::IndexedString, KDevelop::ModificationRevision> const&, KDevelop::IndexedString const&, ClangIndex*, std::function<bool ()> const&)() at kdevelop/plugins/clang/duchain/clanghelpers.cpp:209
#38 ClangHelpers::buildDUChain(void*, QMultiHash<void*, Import> const&, ParseSession const&, QFlags<KDevelop::TopDUContext::Feature>, QHash<void*, KDevelop::ReferencedTopDUContext>&, QHash<KDevelop::IndexedString, KDevelop::ModificationRevision> const&, KDevelop::IndexedString const&, ClangIndex*, std::function<bool ()> const&)() at kdevelop/plugins/clang/duchain/clanghelpers.cpp:121
#39 ClangParseJob::run() at kdevelop/plugins/clang/clangparsejob.cpp:322
#40 ThreadWeaver::IdDecorator::run(QSharedPointer<ThreadWeaver::JobInterface>, ThreadWeaver::Thread*)() at /usr/lib/libKF5ThreadWeaver.so.5
#41 ThreadWeaver::Executor::run(QSharedPointer<ThreadWeaver::JobInterface> const&, ThreadWeaver::Thread*)() at /usr/lib/libKF5ThreadWeaver.so.5
#42 ThreadWeaver::Job::execute(QSharedPointer<ThreadWeaver::JobInterface> const&, ThreadWeaver::Thread*)() at /usr/lib/libKF5ThreadWeaver.so.5
#43 ThreadWeaver::Thread::run()() at /usr/lib/libKF5ThreadWeaver.so.5
#44 ??() at /usr/lib/libQt5Core.so.5
#45 ??() at /usr/lib/libc.so.6
#46 ??() at /usr/lib/libc.so.6
Extended Description
In KDevelop, we use libclang to parse C++ code and then use a visitor to walk over the result and extract information. During that, calling clang_Cursor_getParsedComment sometimes crashes with traces like below.
The crash happens when the contents of the analyzed file are changed while the parse job is running. The reason is, I think, that RawComment::getRawTextSlow gets the comment's text from the SourceManager using the cursors it thinks the comment is at, but SourceManager has discarded the original buffer since, making it read from the new, changed file on disk. It then retrieves some text that is not actually a comment (e.g. doesn't end with */ or so), for which the comment lexer is not designed, and crashes.
This bug is easy to reproduce here when e.g. switching git branches while parsing. Below is the trace, as well as the start and end pointers of the buffer: In one version of the file, a character was added to the affected comment, which results in the BufferEnd pointer pointing at the closing / character instead of behind it, causing the lexer to walk beyond the end location and crash.
I'm not sure how to fix this. It would be simple to make the comment lexer survive this situation, but that doesn't really fix the problem (instead, we'll then sometimes have half the source code returned when retrieving a comment). One would need to ensure that the same buffer is used to read the comments from as the AST was generated for.
Ideas?
Crash trace:
(gdb) bt
0 0x00007fff94d9c9c0 in clang::comments::Lexer::lex(clang::comments::Token&) (this=0x7fff88886940, T=...) at ../tools/clang/lib/AST/CommentLexer.cpp:803
1 0x00007fff94da246c in clang::comments::Parser::consumeToken() (this=) at ../tools/clang/include/clang/AST/CommentParser.h:63
2 0x00007fff94da246c in clang::comments::Parser::parseParagraphOrBlockCommand() (this=0x7fff88886a70) at ../tools/clang/lib/AST/CommentParser.cpp:636
3 0x00007fff94da3b1d in clang::comments::Parser::parseBlockContent() (this=this@entry=0x7fff88886a70) at ../tools/clang/lib/AST/CommentParser.cpp:736
4 0x00007fff94da3bcc in clang::comments::Parser::parseFullComment() (this=this@entry=0x7fff88886a70) at ../tools/clang/lib/AST/CommentParser.cpp:766
5 0x00007fff94ea4ec0 in clang::RawComment::parse(clang::ASTContext const&, clang::Preprocessor const, clang::Decl const) const (this=, Context=..., PP=PP@entry=0x0, D=0x7fff1ef5bd50) at ../tools/clang/lib/AST/RawCommentList.cpp:221
6 0x00007fff94d04d95 in clang::ASTContext::getCommentForDecl(clang::Decl const, clang::Preprocessor const) const (this=0x7fff4cd135e0, D=, D@entry=0x7fff1ef5bd50, PP=PP@entry=0x0) at ../tools/clang/lib/AST/ASTContext.cpp:542
7 0x00007fff951d089a in clang_Cursor_getParsedComment(CXCursor) (C=...) at ../tools/clang/tools/libclang/CXComment.cpp:37
8 0x00007fff9546e4f3 in (anonymous namespace)::Visitor::setDeclData<(CXCursorKind)3>(CXCursor, KDevelop::Declaration*, bool) (cursor=..., decl=decl@entry=0x7fff7de33d40, this=, setComment=true) at ../plugins/clang/duchain/builder.cpp:949
9 0x00007fff9548aa07 in (anonymous namespace)::Visitor::createDeclarationCommon<(CXCursorKind)7, KDevelop::Declaration>(CXCursor, KDevelop::Identifier const&) (this=0x7fff88889640, cursor=..., id=...) at ../plugins/clang/duchain/builder.cpp:428
10 0x00007fff9548e691 in (anonymous namespace)::Visitor::createDeclaration<(CXCursorKind)5, KDevelop::Declaration> (context=0x7fff7e0729f0, id=..., cursor=..., this=0x7fff88889640) at ../plugins/clang/duchain/builder.cpp:439
11 0x00007fff9548e691 in (anonymous namespace)::Visitor::buildDeclaration<(CXCursorKind)5, KDevelop::Declaration, true> (cursor=..., this=0x7fff88889640) at ../plugins/clang/duchain/builder.cpp:1178
12 0x00007fff9548e691 in (anonymous namespace)::Visitor::dispatchCursor<(CXCursorKind)5, (Decision)1, (Decision)0> (parent=..., cursor=..., this=0x7fff88889640) at ../plugins/clang/duchain/builder.cpp:939
13 0x00007fff9548e691 in (anonymous namespace)::Visitor::dispatchCursor<(CXCursorKind)5, (Decision)1, (Decision)2> (parent=..., cursor=..., this=0x7fff88889640) at ../plugins/clang/duchain/builder.cpp:913
14 0x00007fff9548e691 in (anonymous namespace)::Visitor::dispatchCursor<(CXCursorKind)5> (parent=..., cursor=..., this=0x7fff88889640) at ../plugins/clang/duchain/builder.cpp:901
15 0x00007fff9548e691 in (anonymous namespace)::visitCursor(CXCursor, CXCursor, CXClientData) (cursor=..., parent=..., data=0x7fff88889640) at ../plugins/clang/duchain/builder.cpp:1512
16 0x00007fff951bc050 in clang::cxcursor::CursorVisitor::Visit(CXCursor, bool) (this=this@entry=0x7fff88887f80, Cursor=..., CheckedRegionOfInterest=CheckedRegionOfInterest@entry=true) at ../tools/clang/tools/libclang/CIndex.cpp:210
17 0x00007fff951bf88d in clang::cxcursor::CursorVisitor::handleDeclForVisitation(clang::Decl const*) (this=this@entry=0x7fff88887f80, D=0x7fff1ef5bd50) at ../tools/clang/tools/libclang/CIndex.cpp:667
18 0x00007fff951bfa7b in clang::cxcursor::CursorVisitor::VisitDeclContext(clang::DeclContext*) (this=0x7fff88887f80, DC=0x7fff1ef5bc88) at ../tools/clang/tools/libclang/CIndex.cpp:628
19 0x00007fff951bba69 in clang::cxcursor::CursorVisitor::VisitChildren(CXCursor) (this=this@entry=0x7fff88887f80, Cursor=...) at ../tools/clang/tools/libclang/CIndex.cpp:499
20 0x00007fff951c4557 in clang_visitChildren(CXCursor, CXCursorVisitor, CXClientData) (parent=..., visitor=visitor@entry=0x7fff9548ca50 <(anonymous namespace)::visitCursor(CXCursor, CXCursor, CXClientData)>, client_data=client_data@entry=0x7fff88889640) at ../tools/clang/tools/libclang/CIndex.cpp:4276
21 0x00007fff95478e17 in (anonymous namespace)::Visitor::buildDeclaration<(CXCursorKind)22, KDevelop::Declaration, true>(CXCursor) (this=this@entry=0x7fff88889640, cursor=...) at ../plugins/clang/duchain/builder.cpp:1181
22 0x00007fff9548d5b6 in (anonymous namespace)::Visitor::dispatchCursor<(CXCursorKind)22> (cursor=..., this=0x7fff88889640, parent=...) at ../plugins/clang/duchain/builder.cpp:939
23 0x00007fff9548d5b6 in (anonymous namespace)::visitCursor(CXCursor, CXCursor, CXClientData) (cursor=..., parent=..., data=0x7fff88889640) at ../plugins/clang/duchain/builder.cpp:1520
24 0x00007fff951bc050 in clang::cxcursor::CursorVisitor::Visit(CXCursor, bool) (this=this@entry=0x7fff88889400, Cursor=..., CheckedRegionOfInterest=CheckedRegionOfInterest@entry=true) at ../tools/clang/tools/libclang/CIndex.cpp:210
25 0x00007fff951bf88d in clang::cxcursor::CursorVisitor::handleDeclForVisitation(clang::Decl const*) (this=this@entry=0x7fff88889400, D=0x7fff1ef5bc60) at ../tools/clang/tools/libclang/CIndex.cpp:667
26 0x00007fff951bfa7b in clang::cxcursor::CursorVisitor::VisitDeclContext(clang::DeclContext*) (this=this@entry=0x7fff88889400, DC=0x7fff1dcf6788) at ../tools/clang/tools/libclang/CIndex.cpp:628
27 0x00007fff951bbd5b in clang::cxcursor::CursorVisitor::VisitChildren(CXCursor) (this=this@entry=0x7fff88889400, Cursor=...) at ../tools/clang/tools/libclang/CIndex.cpp:533
28 0x00007fff951c4557 in clang_visitChildren(CXCursor, CXCursorVisitor, CXClientData) (parent=..., visitor=visitor@entry=0x7fff9548ca50 <(anonymous namespace)::visitCursor(CXCursor, CXCursor, CXClientData)>, client_data=client_data@entry=0x7fff88889640) at ../tools/clang/tools/libclang/CIndex.cpp:4276
29 0x00007fff9547593c in (anonymous namespace)::Visitor::Visitor (update=, includes=..., file=, tu=, this=0x7fff88889640) at ../plugins/clang/duchain/builder.cpp:1435
30 0x00007fff9547593c in Builder::visit(CXTranslationUnitImpl, void, QHash<void*, KDevelop::ReferencedTopDUContext> const&, bool) (tu=, file=, includes=..., update=) at ../plugins/clang/duchain/builder.cpp:1575
31 0x00007fff9549c5b5 in ClangHelpers::buildDUChain(void, QMultiHash<void, Import> const&, ParseSession const&, KDevelop::TopDUContext::Features, QHash<void, KDevelop::ReferencedTopDUContext>&, ClangIndex, std::function<bool ()> const&) (file=, imports=..., session=..., features=features@entry=KDevelop::TopDUContext::AllDeclarationsContextsAndUses, includedFiles=..., index=0x55555a8e8b00, abortFunction=...) at ../plugins/clang/duchain/clanghelpers.cpp:189
32 0x00007fff9549bee0 in ClangHelpers::buildDUChain(void, QMultiHash<void, Import> const&, ParseSession const&, KDevelop::TopDUContext::Features, QHash<void, KDevelop::ReferencedTopDUContext>&, ClangIndex, std::function<bool ()> const&) (file=, imports=..., session=..., features=KDevelop::TopDUContext::AllDeclarationsContextsAndUses, includedFiles=..., index=0x55555a8e8b00, abortFunction=...) at ../plugins/clang/duchain/clanghelpers.cpp:121
33 0x00007fff95700ae5 in ClangParseJob::run(QSharedPointer, ThreadWeaver::Thread*) (this=0x55555c23ffa0) at ../plugins/clang/clangparsejob.cpp:323
34 0x00007fffed11c087 in ThreadWeaver::IdDecorator::run(QSharedPointer, ThreadWeaver::Thread*) () at /usr/lib/libKF5ThreadWeaver.so.5
35 0x00007fffed11c678 in ThreadWeaver::Executor::run(QSharedPointer const&, ThreadWeaver::Thread*) () at /usr/lib/libKF5ThreadWeaver.so.5
36 0x00007fffed11b670 in ThreadWeaver::Job::execute(QSharedPointer const&, ThreadWeaver::Thread*) () at /usr/lib/libKF5ThreadWeaver.so.5
37 0x00007fffed11ada5 in ThreadWeaver::Thread::run() () at /usr/lib/libKF5ThreadWeaver.so.5
38 0x00007ffff1052fcb in () at /usr/lib/libQt5Core.so.5
39 0x00007fffe9c0208a in start_thread () at /usr/lib/libpthread.so.0
40 0x00007ffff03fc24f in clone () at /usr/lib/libc.so.6
Extra info:
(gdb) p BufferStart $3 = 0x7fff7db0f180 "/Positions for tool views in Sublime UI. To combine them, use\n Positions class (QFlags based).*/\n enum Position\n {\n Left = 1 /*< left tool view position /,\n Right = 2 /"... (gdb) p BufferEnd $4 = 0x7fff7db0f1e4 "/\n enum Position\n {\n Left = 1 /< left tool view position */,\n Right = 2 /*< right tool view position /,\n Top = 4 /< top tool view position */,\n Bottom = 8"... (gdb) p EndWhitespace $5 = 0x7fff80000000 <error: Cannot access memory at address 0x7fff80000000>