search

llvm / llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
http://llvm.org
Other
27.15k stars 11.12k forks source link

clang-tidy asserts and fails to process valid preprocessed source when in C++20 mode only due to ExpandModularHeadersPPCallbacks #47910

Open 41fa9ab2-4e9a-46d3-8102-9366090d3552 opened 3 years ago

41fa9ab2-4e9a-46d3-8102-9366090d3552 commented 3 years ago
Bugzilla Link 48566
Version unspecified
OS All

Extended Description

Using the latest commit as of writing (https://github.com/llvm/llvm-project/commit/3fa2d37eb3f8acddcfde749ca822f2cc7d900cbb), clang-tidy fails to process the following preprocessed source:

    # 1 "main.cpp"
    # 1 "usr/include/sys/cdefs.h" 1
    # 1 "usr/include/android/api-level.h" 1
    # 47 "usr/include/android/api-level.h"
    extern "C" {
    # 170 "usr/include/android/api-level.h"
    #if 1
    # 1 "usr/include/bits/get_device_api_level_inlines.h" 1
    # 30 "usr/include/bits/get_device_api_level_inlines.h"
    # 32 "usr/include/bits/get_device_api_level_inlines.h"
    # 33 "usr/include/bits/get_device_api_level_inlines.h"
    # 34 "usr/include/bits/get_device_api_level_inlines.h"
    # 51 "usr/include/bits/get_device_api_level_inlines.h"
    #endif
    # 177 "usr/include/android/api-level.h" 2
    # 191 "usr/include/android/api-level.h"
    }
    # 363 "usr/include/sys/cdefs.h" 2
    # 1 "main.cpp" 2

The following command line was used:

    clang-tidy main.cpp -- -std=c++20

In Release, clang-tidy outputs this error:

    1 error generated.
    Error while processing main.cpp.
    main.cpp:19:16: error: invalid line marker flag '2': cannot pop empty include stack [clang-diagnostic-error]
    # 1 "main.cpp" 2
                   ^
    Found compiler error(s).

In Debug, clang-tidy crashes with this assert:

    clang-tidy: /tmp/llvm-project/clang/lib/Basic/SourceManager.cpp:210: void clang::LineTableInfo::AddLineNote(clang::FileID, unsigned int, unsigned int, int, unsigned int, clang::SrcMgr::CharacteristicKind): Assertion `(Entries.empty() || Entries.back().FileOffset < Offset) && "Adding line entries out of order!"' failed.
    PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.
    Stack dump:
    0.      Program arguments: ./clang-tidy /tmp/main.cpp -- -std=c++20
    1.      main.cpp:0:3: current parser token '1'
     #&#8203;0 0x000055cffd08ed58 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /tmp/llvm-project/llvm/lib/Support/Unix/Signals.inc:563:0
     #&#8203;1 0x000055cffd08ee0f PrintStackTraceSignalHandler(void*) /tmp/llvm-project/llvm/lib/Support/Unix/Signals.inc:630:0
     #&#8203;2 0x000055cffd08caf3 llvm::sys::RunSignalHandlers() /tmp/llvm-project/llvm/lib/Support/Signals.cpp:71:0
     #&#8203;3 0x000055cffd08e6da SignalHandler(int) /tmp/llvm-project/llvm/lib/Support/Unix/Signals.inc:405:0
     #&#8203;4 0x00007fb179480980 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x12980)
     #&#8203;5 0x00007fb178335fb7 gsignal /build/glibc-S7xCS9/glibc-2.27/signal/../sysdeps/unix/sysv/linux/raise.c:51:0
     #&#8203;6 0x00007fb178337921 abort /build/glibc-S7xCS9/glibc-2.27/stdlib/abort.c:81:0
     #&#8203;7 0x00007fb17832748a __assert_fail_base /build/glibc-S7xCS9/glibc-2.27/assert/assert.c:89:0
     #&#8203;8 0x00007fb178327502 (/lib/x86_64-linux-gnu/libc.so.6+0x30502)
     #&#8203;9 0x000055cffcb5f09d clang::LineTableInfo::AddLineNote(clang::FileID, unsigned int, unsigned int, int, unsigned int, clang::SrcMgr::CharacteristicKind) /tmp/llvm-project/clang/lib/Basic/SourceManager.cpp:209:0
    llvm/llvm-project#382 0x000055cffcb5f46d clang::SourceManager::AddLineNote(clang::SourceLocation, unsigned int, int, bool, bool, clang::SrcMgr::CharacteristicKind) /tmp/llvm-project/clang/lib/Basic/SourceManager.cpp:290:0
    llvm/llvm-project#383 0x000055cffca8a137 clang::Preprocessor::HandleDigitDirective(clang::Token&) /tmp/llvm-project/clang/lib/Lex/PPDirectives.cpp:1363:0
    llvm/llvm-project#384 0x000055cffca88b1a clang::Preprocessor::HandleDirective(clang::Token&) /tmp/llvm-project/clang/lib/Lex/PPDirectives.cpp:977:0
    llvm/llvm-project#385 0x000055cffca4c554 clang::Lexer::LexTokenInternal(clang::Token&, bool) /tmp/llvm-project/clang/lib/Lex/Lexer.cpp:3975:0
    llvm/llvm-project#386 0x000055cffca498d8 clang::Lexer::Lex(clang::Token&) /tmp/llvm-project/clang/lib/Lex/Lexer.cpp:3188:0
    llvm/llvm-project#387 0x000055cffcacc1fa clang::Preprocessor::Lex(clang::Token&) /tmp/llvm-project/clang/lib/Lex/Preprocessor.cpp:898:0
    llvm/llvm-project#388 0x000055cffaded894 clang::Parser::ConsumeToken() /tmp/llvm-project/clang/include/clang/Parse/Parser.h:483:0
    llvm/llvm-project#389 0x000055cffade2f55 clang::Parser::Initialize() /tmp/llvm-project/clang/lib/Parse/Parser.cpp:562:0
    llvm/llvm-project#390 0x000055cffaddedb6 clang::ParseAST(clang::Sema&, bool, bool) /tmp/llvm-project/clang/lib/Parse/ParseAST.cpp:156:0
    llvm/llvm-project#391 0x000055cffab19495 clang::ASTFrontendAction::ExecuteAction() /tmp/llvm-project/clang/lib/Frontend/FrontendAction.cpp:1056:0
    llvm/llvm-project#392 0x000055cffab18d5e clang::FrontendAction::Execute() /tmp/llvm-project/clang/lib/Frontend/FrontendAction.cpp:953:0
    llvm/llvm-project#393 0x000055cffaa7ab38 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) /tmp/llvm-project/clang/lib/Frontend/CompilerInstance.cpp:991:0
    llvm/llvm-project#394 0x000055cff9ed747d clang::tooling::FrontendActionFactory::runInvocation(std::shared_ptr<clang::CompilerInvocation>, clang::FileManager*, std::shared_ptr<clang::PCHContainerOperations>, clang::DiagnosticConsumer*) /tmp/llvm-project/clang/lib/Tooling/Tooling.cpp:396:0
    llvm/llvm-project#395 0x000055cff9e5f9a3 runInvocation /tmp/llvm-project/clang-tools-extra/clang-tidy/ClangTidy.cpp:556:0
    llvm/llvm-project#396 0x000055cff9e5f9a3 clang::tidy::runClangTidy(clang::tidy::ClangTidyContext&, clang::tooling::CompilationDatabase const&, llvm::ArrayRef<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, llvm::IntrusiveRefCntPtr<llvm::vfs::OverlayFileSystem>, bool, llvm::StringRef)::ActionFactory::runInvocation(std::shared_ptr<clang::CompilerInvocation>, clang::FileManager*, std::shared_ptr<clang::PCHContainerOperations>, clang::DiagnosticConsumer*) (./clang-tidy+0x11e19a3)
    llvm/llvm-project#397 0x000055cff9ed72ad clang::tooling::ToolInvocation::runInvocation(char const*, clang::driver::Compilation*, std::shared_ptr<clang::CompilerInvocation>, std::shared_ptr<clang::PCHContainerOperations>) /tmp/llvm-project/clang/lib/Tooling/Tooling.cpp:372:0
    llvm/llvm-project#398 0x000055cff9ed70c4 clang::tooling::ToolInvocation::run() /tmp/llvm-project/clang/lib/Tooling/Tooling.cpp:357:0
    llvm/llvm-project#399 0x000055cff9ed884d clang::tooling::ClangTool::run(clang::tooling::ToolAction*) /tmp/llvm-project/clang/lib/Tooling/Tooling.cpp:549:0
    llvm/llvm-project#400 0x000055cff9e5fda1 clang::tidy::runClangTidy(clang::tidy::ClangTidyContext&, clang::tooling::CompilationDatabase const&, llvm::ArrayRef<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, llvm::IntrusiveRefCntPtr<llvm::vfs::OverlayFileSystem>, bool, llvm::StringRef) /tmp/llvm-project/clang-tools-extra/clang-tidy/ClangTidy.cpp:577:0
    llvm/llvm-project#401 0x000055cff927ce27 clang::tidy::clangTidyMain(int, char const**) /tmp/llvm-project/clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp:486:0
    llvm/llvm-project#402 0x000055cff927b1ea main /tmp/llvm-project/clang-tools-extra/clang-tidy/tool/ClangTidyToolMain.cpp:21:0
    llvm/llvm-project#403 0x00007fb178318bf7 __libc_start_main /build/glibc-S7xCS9/glibc-2.27/csu/../csu/libc-start.c:344:0
    llvm/llvm-project#404 0x000055cff927b0ea _start (./clang-tidy+0x5fd0ea)
    Aborted

This only occurs with -std=c++20. It does not occur with -std=c++17.

If I disable the use of ExpandModularHeadersPPCallbacks here (https://github.com/llvm/llvm-project/blob/3fa2d37eb3f8acddcfde749ca822f2cc7d900cbb/clang-tools-extra/clang-tidy/ClangTidy.cpp#L414-L419) then clang-tidy succeeds even in C++20 mode.

avudnez commented 5 months ago

I am hitting this too (17.0.1), after #62405.

This is making analyzing Unreal Engine (for example) source code even more difficult, and I will need to resort to ugly hacks to work around this. Maybe @PiotrZSL can provide some information?

llvmbot commented 5 months ago

@llvm/issue-subscribers-c-20

Author: None (41fa9ab2-4e9a-46d3-8102-9366090d3552)

| | | | --- | --- | | Bugzilla Link | [48566](https://llvm.org/bz48566) | | Version | unspecified | | OS | All | ## Extended Description Using the latest commit as of writing (https://github.com/llvm/llvm-project/commit/3fa2d37eb3f8acddcfde749ca822f2cc7d900cbb), clang-tidy fails to process the following preprocessed source: ```cpp # 1 "main.cpp" # 1 "usr/include/sys/cdefs.h" 1 # 1 "usr/include/android/api-level.h" 1 # 47 "usr/include/android/api-level.h" extern "C" { # 170 "usr/include/android/api-level.h" #if 1 # 1 "usr/include/bits/get_device_api_level_inlines.h" 1 # 30 "usr/include/bits/get_device_api_level_inlines.h" # 32 "usr/include/bits/get_device_api_level_inlines.h" # 33 "usr/include/bits/get_device_api_level_inlines.h" # 34 "usr/include/bits/get_device_api_level_inlines.h" # 51 "usr/include/bits/get_device_api_level_inlines.h" #endif # 177 "usr/include/android/api-level.h" 2 # 191 "usr/include/android/api-level.h" } # 363 "usr/include/sys/cdefs.h" 2 # 1 "main.cpp" 2 ``` The following command line was used: ``` clang-tidy main.cpp -- -std=c++20 ``` In Release, clang-tidy outputs this error: ``` 1 error generated. Error while processing main.cpp. main.cpp:19:16: error: invalid line marker flag '2': cannot pop empty include stack [clang-diagnostic-error] # 1 "main.cpp" 2 ^ Found compiler error(s). ``` In Debug, clang-tidy crashes with this assert: ``` clang-tidy: /tmp/llvm-project/clang/lib/Basic/SourceManager.cpp:210: void clang::LineTableInfo::AddLineNote(clang::FileID, unsigned int, unsigned int, int, unsigned int, clang::SrcMgr::CharacteristicKind): Assertion `(Entries.empty() || Entries.back().FileOffset < Offset) && "Adding line entries out of order!"' failed. PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace. Stack dump: 0. Program arguments: ./clang-tidy /tmp/main.cpp -- -std=c++20 1. main.cpp:0:3: current parser token '1' #&#8203;0 0x000055cffd08ed58 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /tmp/llvm-project/llvm/lib/Support/Unix/Signals.inc:563:0 #&#8203;1 0x000055cffd08ee0f PrintStackTraceSignalHandler(void*) /tmp/llvm-project/llvm/lib/Support/Unix/Signals.inc:630:0 #&#8203;2 0x000055cffd08caf3 llvm::sys::RunSignalHandlers() /tmp/llvm-project/llvm/lib/Support/Signals.cpp:71:0 #&#8203;3 0x000055cffd08e6da SignalHandler(int) /tmp/llvm-project/llvm/lib/Support/Unix/Signals.inc:405:0 #&#8203;4 0x00007fb179480980 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x12980) #&#8203;5 0x00007fb178335fb7 gsignal /build/glibc-S7xCS9/glibc-2.27/signal/../sysdeps/unix/sysv/linux/raise.c:51:0 #&#8203;6 0x00007fb178337921 abort /build/glibc-S7xCS9/glibc-2.27/stdlib/abort.c:81:0 #&#8203;7 0x00007fb17832748a __assert_fail_base /build/glibc-S7xCS9/glibc-2.27/assert/assert.c:89:0 #&#8203;8 0x00007fb178327502 (/lib/x86_64-linux-gnu/libc.so.6+0x30502) #&#8203;9 0x000055cffcb5f09d clang::LineTableInfo::AddLineNote(clang::FileID, unsigned int, unsigned int, int, unsigned int, clang::SrcMgr::CharacteristicKind) /tmp/llvm-project/clang/lib/Basic/SourceManager.cpp:209:0 llvm/llvm-project#382 0x000055cffcb5f46d clang::SourceManager::AddLineNote(clang::SourceLocation, unsigned int, int, bool, bool, clang::SrcMgr::CharacteristicKind) /tmp/llvm-project/clang/lib/Basic/SourceManager.cpp:290:0 llvm/llvm-project#383 0x000055cffca8a137 clang::Preprocessor::HandleDigitDirective(clang::Token&) /tmp/llvm-project/clang/lib/Lex/PPDirectives.cpp:1363:0 llvm/llvm-project#384 0x000055cffca88b1a clang::Preprocessor::HandleDirective(clang::Token&) /tmp/llvm-project/clang/lib/Lex/PPDirectives.cpp:977:0 llvm/llvm-project#385 0x000055cffca4c554 clang::Lexer::LexTokenInternal(clang::Token&, bool) /tmp/llvm-project/clang/lib/Lex/Lexer.cpp:3975:0 llvm/llvm-project#386 0x000055cffca498d8 clang::Lexer::Lex(clang::Token&) /tmp/llvm-project/clang/lib/Lex/Lexer.cpp:3188:0 llvm/llvm-project#387 0x000055cffcacc1fa clang::Preprocessor::Lex(clang::Token&) /tmp/llvm-project/clang/lib/Lex/Preprocessor.cpp:898:0 llvm/llvm-project#388 0x000055cffaded894 clang::Parser::ConsumeToken() /tmp/llvm-project/clang/include/clang/Parse/Parser.h:483:0 llvm/llvm-project#389 0x000055cffade2f55 clang::Parser::Initialize() /tmp/llvm-project/clang/lib/Parse/Parser.cpp:562:0 llvm/llvm-project#390 0x000055cffaddedb6 clang::ParseAST(clang::Sema&, bool, bool) /tmp/llvm-project/clang/lib/Parse/ParseAST.cpp:156:0 llvm/llvm-project#391 0x000055cffab19495 clang::ASTFrontendAction::ExecuteAction() /tmp/llvm-project/clang/lib/Frontend/FrontendAction.cpp:1056:0 llvm/llvm-project#392 0x000055cffab18d5e clang::FrontendAction::Execute() /tmp/llvm-project/clang/lib/Frontend/FrontendAction.cpp:953:0 llvm/llvm-project#393 0x000055cffaa7ab38 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) /tmp/llvm-project/clang/lib/Frontend/CompilerInstance.cpp:991:0 llvm/llvm-project#394 0x000055cff9ed747d clang::tooling::FrontendActionFactory::runInvocation(std::shared_ptr<clang::CompilerInvocation>, clang::FileManager*, std::shared_ptr<clang::PCHContainerOperations>, clang::DiagnosticConsumer*) /tmp/llvm-project/clang/lib/Tooling/Tooling.cpp:396:0 llvm/llvm-project#395 0x000055cff9e5f9a3 runInvocation /tmp/llvm-project/clang-tools-extra/clang-tidy/ClangTidy.cpp:556:0 llvm/llvm-project#396 0x000055cff9e5f9a3 clang::tidy::runClangTidy(clang::tidy::ClangTidyContext&, clang::tooling::CompilationDatabase const&, llvm::ArrayRef<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, llvm::IntrusiveRefCntPtr<llvm::vfs::OverlayFileSystem>, bool, llvm::StringRef)::ActionFactory::runInvocation(std::shared_ptr<clang::CompilerInvocation>, clang::FileManager*, std::shared_ptr<clang::PCHContainerOperations>, clang::DiagnosticConsumer*) (./clang-tidy+0x11e19a3) llvm/llvm-project#397 0x000055cff9ed72ad clang::tooling::ToolInvocation::runInvocation(char const*, clang::driver::Compilation*, std::shared_ptr<clang::CompilerInvocation>, std::shared_ptr<clang::PCHContainerOperations>) /tmp/llvm-project/clang/lib/Tooling/Tooling.cpp:372:0 llvm/llvm-project#398 0x000055cff9ed70c4 clang::tooling::ToolInvocation::run() /tmp/llvm-project/clang/lib/Tooling/Tooling.cpp:357:0 llvm/llvm-project#399 0x000055cff9ed884d clang::tooling::ClangTool::run(clang::tooling::ToolAction*) /tmp/llvm-project/clang/lib/Tooling/Tooling.cpp:549:0 llvm/llvm-project#400 0x000055cff9e5fda1 clang::tidy::runClangTidy(clang::tidy::ClangTidyContext&, clang::tooling::CompilationDatabase const&, llvm::ArrayRef<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, llvm::IntrusiveRefCntPtr<llvm::vfs::OverlayFileSystem>, bool, llvm::StringRef) /tmp/llvm-project/clang-tools-extra/clang-tidy/ClangTidy.cpp:577:0 llvm/llvm-project#401 0x000055cff927ce27 clang::tidy::clangTidyMain(int, char const**) /tmp/llvm-project/clang-tools-extra/clang-tidy/tool/ClangTidyMain.cpp:486:0 llvm/llvm-project#402 0x000055cff927b1ea main /tmp/llvm-project/clang-tools-extra/clang-tidy/tool/ClangTidyToolMain.cpp:21:0 llvm/llvm-project#403 0x00007fb178318bf7 __libc_start_main /build/glibc-S7xCS9/glibc-2.27/csu/../csu/libc-start.c:344:0 llvm/llvm-project#404 0x000055cff927b0ea _start (./clang-tidy+0x5fd0ea) Aborted ``` This only occurs with -std=c++20. It does not occur with -std=c++17. If I disable the use of `ExpandModularHeadersPPCallbacks` here (https://github.com/llvm/llvm-project/blob/3fa2d37eb3f8acddcfde749ca822f2cc7d900cbb/clang-tools-extra/clang-tidy/ClangTidy.cpp#L414-L419) then clang-tidy succeeds even in C++20 mode.