search

Quuxplusone / LLVMBugzillaTest

0 stars 0 forks source link

clang::Stmt::getID assertion failure on string literal longer than ~4096 characters #48895

Open Quuxplusone opened 3 years ago

Quuxplusone commented 3 years ago
Bugzilla Link PR49926
Status NEW
Importance P normal
Reported by Matt McCutchen (matt@correctcomputation.com)
Reported on 2021-04-11 11:46:07 -0700
Last modified on 2021-04-11 11:46:07 -0700
Version unspecified
Hardware PC Linux
CC htmldeveloper@gmail.com, llvm-bugs@lists.llvm.org, neeilans@live.com, richard-llvm@metafoo.co.uk
Fixed by commit(s)
Attachments LargeStringLiteralGetId.cpp (1178 bytes, text/x-c++src)
Blocks
Blocked by
See also 
Created attachment 24745
Unit test demonstrating the problem

We get an assertion failure from clang::Stmt::getID for a Stmt representing a
string literal longer than about ~4096 characters.  If the string literal is
somewhat shorter (say 4000 characters), then there is no assertion failure.

It looks like the cause is that the Stmt is allocated using a BumpPtrAllocator
(ASTContext::BumpAlloc) and it exceeds the default slab size of 4096 bytes, so
it goes into a custom-size slab.  This causes
BumpPtrAllocatorImpl::identifyObject to return an ID that is -1 minus the
offset of the object among all the custom-size slabs.
BumpPtrAllocatorImpl::identifyKnownAlignedObject doesn't handle this rare case
properly: it assumes the ID is divisible by the alignment, which is true when
using a default-size slab but not using a custom-size slab.  I guess no one
thought about this case.  Having identifyKnownAlignedObject subtract (alignment
- 1) when the ID is negative would likely fix the problem.

Perhaps this bug should be filed against BumpPtrAllocatorImpl, but I thought I
would start with Clang because the getID methods appear to have been the
original motivating use case for
BumpPtrAllocatorImpl::identifyKnownAlignedObject and they are where the end-to-
end behavior is unquestionably wrong, while I might be missing some design
consideration at the BumpPtrAllocatorImpl level.

I wrote a Clang unit test (attached) as a way to demonstrate the problem; I
don't claim that it complies with any of the coding standards.  To run the
test, save the file at clang/unittests/AST/LargeStringLiteralGetId.cpp, add it
to the list in clang/unittests/AST/CMakeLists.txt, and run "ninja check-clang
unit" or the equivalent in your environment.  I tested it on the current LLVM
monorepo main branch (ea8dd3ee2eb457a8c3975e1f64caa7a58169e02e).

The stack trace (to make it easier to find this bug report in a search):

FAIL: Clang-Unit :: AST/./ASTTests/LargeStringLiteralGetId.SingleTest (119 of
13978)
******************** TEST 'Clang-Unit ::
AST/./ASTTests/LargeStringLiteralGetId.SingleTest' FAILED ********************
Note: Google Test filter = LargeStringLiteralGetId.SingleTest
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from LargeStringLiteralGetId
[ RUN      ] LargeStringLiteralGetId.SingleTest
ASTTests: /home/matt/llvm-main-
test.wt/llvm/include/llvm/Support/Allocator.h:272: int64_t
llvm::BumpPtrAllocatorImpl<AllocatorT, SlabSize, SizeThreshold,
GrowthDelay>::identifyKnownAlignedObject(const void*) [with T = clang::Stmt;
AllocatorT = llvm::MallocAllocator; long unsigned int SlabSize = 4096; long
unsigned int SizeThreshold = 4096; long unsigned int GrowthDelay = 128; int64_t
= long int]: Assertion `Out % alignof(T) == 0 && "Wrong alignment information"'
failed.
 #0 0x00005632d1990a93 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /home/matt/llvm-main-test.wt/llvm/lib/Support/Unix/Signals.inc:565:22
 #1 0x00005632d1990b4a PrintStackTraceSignalHandler(void*) /home/matt/llvm-main-test.wt/llvm/lib/Support/Unix/Signals.inc:632:1
 #2 0x00005632d198e3af llvm::sys::RunSignalHandlers() /home/matt/llvm-main-test.wt/llvm/lib/Support/Signals.cpp:71:20
 #3 0x00005632d198e81a SignalHandler(int) /home/matt/llvm-main-test.wt/llvm/lib/Support/Unix/Signals.inc:397:31
 #4 0x00007fa621e053c0 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x153c0)
 #5 0x00007fa6218ab18b raise /build/glibc-eX1tMB/glibc-2.31/signal/../sysdeps/unix/sysv/linux/raise.c:51:1
 #6 0x00007fa62188a859 abort /build/glibc-eX1tMB/glibc-2.31/stdlib/abort.c:81:7
 #7 0x00007fa62188a729 get_sysdep_segment_value /build/glibc-eX1tMB/glibc-2.31/intl/loadmsgcat.c:509:8
 #8 0x00007fa62188a729 _nl_load_domain /build/glibc-eX1tMB/glibc-2.31/intl/loadmsgcat.c:970:34
 #9 0x00007fa62189bf36 (/lib/x86_64-linux-gnu/libc.so.6+0x36f36)
#10 0x00005632d1ec0ec0 clang::Stmt::getID(clang::ASTContext const&) const
/home/matt/llvm-main-test.wt/clang/lib/AST/Stmt.cpp:362:1
#11 0x00005632d17fea60
clang::RecursiveASTVisitor<GetIdVisitor>::getStmtChildren(clang::Stmt*)
/home/matt/llvm-main-test.wt/clang/include/clang/AST/RecursiveASTVisitor.h:341:66
#12 0x00005632d17fea60
clang::RecursiveASTVisitor<GetIdVisitor>::TraverseStringLiteral(clang::StringLiteral*,
llvm::SmallVectorImpl<llvm::PointerIntPair<clang::Stmt*, 1u, bool,
llvm::PointerLikeTypeTraits<clang::Stmt*>,
llvm::PointerIntPairInfo<clang::Stmt*, 1u,
llvm::PointerLikeTypeTraits<clang::Stmt*> > > >*) /home/matt/llvm-main-
test.wt/clang/include/clang/AST/RecursiveASTVisitor.h:2771:1
#13 0x00005632d17f5e25
clang::RecursiveASTVisitor<GetIdVisitor>::dataTraverseNode(clang::Stmt*,
llvm::SmallVectorImpl<llvm::PointerIntPair<clang::Stmt*, 1u, bool,
llvm::PointerLikeTypeTraits<clang::Stmt*>,
llvm::PointerIntPairInfo<clang::Stmt*, 1u,
llvm::PointerLikeTypeTraits<clang::Stmt*> > > >*) /home/matt/llvm-main-
test.wt/build/tools/clang/include/clang/AST/StmtNodes.inc:1375:1
#14 0x00005632d17f60c3
clang::RecursiveASTVisitor<GetIdVisitor>::TraverseStmt(clang::Stmt*,
llvm::SmallVectorImpl<llvm::PointerIntPair<clang::Stmt*, 1u, bool,
llvm::PointerLikeTypeTraits<clang::Stmt*>,
llvm::PointerIntPairInfo<clang::Stmt*, 1u,
llvm::PointerLikeTypeTraits<clang::Stmt*> > > >*) /home/matt/llvm-main-
test.wt/clang/include/clang/AST/RecursiveASTVisitor.h:605:7
#15 0x00005632d18116c0
clang::RecursiveASTVisitor<GetIdVisitor>::TraverseVarHelper(clang::VarDecl*)
/home/matt/llvm-main-test.wt/clang/include/clang/AST/RecursiveASTVisitor.h:2130:1
#16 0x00005632d181a13d
clang::RecursiveASTVisitor<GetIdVisitor>::TraverseVarDecl(clang::VarDecl*)
/home/matt/llvm-main-test.wt/clang/include/clang/AST/RecursiveASTVisitor.h:2132:1
#17 0x00005632d17f53bc
clang::RecursiveASTVisitor<GetIdVisitor>::TraverseDecl(clang::Decl*)
/home/matt/llvm-main-test.wt/build/tools/clang/include/clang/AST/DeclNodes.inc:453:1
#18 0x00005632d17f5503
clang::RecursiveASTVisitor<GetIdVisitor>::TraverseDeclContextHelper(clang::DeclContext*)
/home/matt/llvm-main-test.wt/clang/include/clang/AST/RecursiveASTVisitor.h:1389:7
#19 0x00005632d1817c8e
clang::RecursiveASTVisitor<GetIdVisitor>::TraverseTranslationUnitDecl(clang::TranslationUnitDecl*)
/home/matt/llvm-main-test.wt/clang/include/clang/AST/RecursiveASTVisitor.h:1490:1
#20 0x00005632d1817d4a MyConsumer::HandleTranslationUnit(clang::ASTContext&)
/home/matt/llvm-main-test.wt/clang/unittests/AST/LargeStringLiteralGetId.cpp:22:3
#21 0x00005632d2483592 clang::ParseAST(clang::Sema&, bool, bool)
/home/matt/llvm-main-test.wt/clang/lib/Parse/ParseAST.cpp:178:11
#22 0x00005632d21dc96d clang::ASTFrontendAction::ExecuteAction()
/home/matt/llvm-main-test.wt/clang/lib/Frontend/FrontendAction.cpp:1058:1
#23 0x00005632d21e0d9c clang::FrontendAction::Execute() /home/matt/llvm-main-
test.wt/clang/lib/Frontend/FrontendAction.cpp:949:21
#24 0x00005632d2143d26 llvm::Error::setChecked(bool) /home/matt/llvm-main-
test.wt/llvm/include/llvm/Support/Error.h:305:22
#25 0x00005632d2143d26 llvm::Error::operator bool() /home/matt/llvm-main-
test.wt/llvm/include/llvm/Support/Error.h:236:15
#26 0x00005632d2143d26
clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) /home/matt/llvm-
main-test.wt/clang/lib/Frontend/CompilerInstance.cpp:958:42
#27 0x00005632d2480a17
clang::tooling::FrontendActionFactory::runInvocation(std::shared_ptr<clang::CompilerInvocation>,
clang::FileManager*, std::shared_ptr<clang::PCHContainerOperations>,
clang::DiagnosticConsumer*) /home/matt/llvm-main-
test.wt/clang/lib/Tooling/Tooling.cpp:396:46
#28 0x00005632d24790cc clang::tooling::ToolInvocation::runInvocation(char
const*, clang::driver::Compilation*,
std::shared_ptr<clang::CompilerInvocation>,
std::shared_ptr<clang::PCHContainerOperations>) /home/matt/llvm-main-
test.wt/clang/lib/Tooling/Tooling.cpp:371:31
#29 0x00005632d247c803 clang::tooling::ToolInvocation::run() /home/matt/llvm-
main-test.wt/clang/lib/Tooling/Tooling.cpp:356:23
#30 0x00005632d247d128
clang::tooling::runToolOnCodeWithArgs(std::unique_ptr<clang::FrontendAction,
std::default_delete<clang::FrontendAction> >, llvm::Twine const&,
llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>,
std::vector<std::__cxx11::basic_string<char, std::char_traits<char>,
std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > > > const&, llvm::Twine const&,
llvm::Twine const&, std::shared_ptr<clang::PCHContainerOperations>)
/home/matt/llvm-main-test.wt/clang/lib/Tooling/Tooling.cpp:197:24
#31 0x00005632d247d68b
clang::tooling::runToolOnCodeWithArgs(std::unique_ptr<clang::FrontendAction,
std::default_delete<clang::FrontendAction> >, llvm::Twine const&,
std::vector<std::__cxx11::basic_string<char, std::char_traits<char>,
std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > > > const&, llvm::Twine const&,
llvm::Twine const&, std::shared_ptr<clang::PCHContainerOperations>,
std::vector<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>,
std::allocator<char> >, std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > >,
std::allocator<std::pair<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >,
std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >
> > > const&) /home/matt/llvm-main-test.wt/clang/lib/Tooling/Tooling.cpp:223:31
#32 0x00005632d247d967
clang::tooling::runToolOnCode(std::unique_ptr<clang::FrontendAction,
std::default_delete<clang::FrontendAction> >, llvm::Twine const&, llvm::Twine
const&, std::shared_ptr<clang::PCHContainerOperations>) /home/matt/llvm-main-
test.wt/clang/lib/Tooling/Tooling.cpp:159:31
#33 0x00005632d17f204c std::unique_ptr<clang::FrontendAction,
std::default_delete<clang::FrontendAction> >::~unique_ptr()
/usr/include/c++/9/bits/unique_ptr.h:291:12
#34 0x00005632d17f204c LargeStringLiteralGetId_SingleTest_Test::TestBody()
/home/matt/llvm-main-test.wt/clang/unittests/AST/LargeStringLiteralGetId.cpp:43:41
#35 0x00005632d19b3fa5 void
testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test,
void>(testing::Test*, void (testing::Test::*)(), char const*) /home/matt/llvm-
main-test.wt/llvm/utils/unittest/googletest/src/gtest.cc:2404:1
#36 0x00005632d19b9658 void
testing::internal::HandleExceptionsInMethodIfSupported<testing::Test,
void>(testing::Test*, void (testing::Test::*)(), char const*) /home/matt/llvm-
main-test.wt/llvm/utils/unittest/googletest/src/gtest.cc:2455:75
#37 0x00005632d19aa395 testing::Test::Run() /home/matt/llvm-main-
test.wt/llvm/utils/unittest/googletest/src/gtest.cc:2474:50
#38 0x00005632d19ac96e testing::TestInfo::Run() /home/matt/llvm-main-
test.wt/llvm/utils/unittest/googletest/src/gtest.cc:2656:14
#39 0x00005632d19aca11 testing::TestCase::Run() /home/matt/llvm-main-
test.wt/llvm/utils/unittest/googletest/src/gtest.cc:2773:3
#40 0x00005632d19af403 testing::internal::UnitTestImpl::RunAllTests()
/home/matt/llvm-main-test.wt/llvm/utils/unittest/googletest/src/gtest.cc:4647:9
#41 0x00005632d19b4684 bool
testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,
bool>(testing::internal::UnitTestImpl*, bool
(testing::internal::UnitTestImpl::*)(), char const*) /home/matt/llvm-main-
test.wt/llvm/utils/unittest/googletest/src/gtest.cc:2404:1
#42 0x00005632d19b977b bool
testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,
bool>(testing::internal::UnitTestImpl*, bool
(testing::internal::UnitTestImpl::*)(), char const*) /home/matt/llvm-main-
test.wt/llvm/utils/unittest/googletest/src/gtest.cc:2455:75
#43 0x00005632d19aa41b testing::UnitTest::Run() /home/matt/llvm-main-
test.wt/llvm/utils/unittest/googletest/src/gtest.cc:4260:64
#44 0x00005632d199269c main /home/matt/llvm-main-
test.wt/llvm/utils/unittest/UnitTestMain/TestMain.cpp:51:1
#45 0x00007fa62188c0b3 __libc_start_main /build/glibc-eX1tMB/glibc-
2.31/csu/../csu/libc-start.c:342:3
#46 0x00005632d14af93e _start (/home/matt/llvm-main-
test.wt/build/tools/clang/unittests/AST/./ASTTests+0x10b393e)

********************
********************
Failed Tests (1):
  Clang-Unit :: AST/./ASTTests/LargeStringLiteralGetId.SingleTest

Testing Time: 22.60s
  Passed: 13977
  Failed:     1
Quuxplusone commented 3 years ago

Attached LargeStringLiteralGetId.cpp (1178 bytes, text/x-c++src): Unit test demonstrating the problem