error MSB6006: "clang-cl.exe" exited with code -1073740940 (heap corruption)

[YukinoHayakawa]

I'm kinda reluctant to say that I'm facing a new problem now. But the compiler is very nice otherwise so I don't want to abandon it. The problem is the compiler crashes badly in some cases, especially when the input file is large. Unfortunately, the minimal example that is able to reproduce the problem involves google test headers and compiler explorer doesn't have that on cppx branch. My compiler was built from e71f5467. But I believe this is not the first time I've encountered similar problems.

#include <gtest/gtest.h>
// #include <fmt/printf.h>

#include <experimental/meta>
#include <experimental/compiler>

using namespace std::experimental;

struct ComponentA
{
    int a;
    char b;
    float c;
    char d[16];
};

template <typename C>
std::string layout_string_yaml()
{
    // fmt::memory_buffer buffer;
    consteval
    {
        constexpr auto clazz = reflexpr(C);
        // constexpr auto buf = reflexpr(buffer);
        -> fragment {
            // format_to(idexpr(%{buf}), "---\n");
            printf("name: %s\n", meta::name_of(%{clazz}));
            printf("size: %llu\n", sizeof(C));
            printf("fields:\n");
        };
        for(meta::info member : meta::data_member_range(clazz))
        {
            -> fragment {
                printf("  -\n");
                printf("    name: %s\n", meta::name_of(%{member}));
                printf("    type: %s\n", meta::name_of(meta::type_of(%{member})));
                printf("    offset: %llu\n", ((::size_t)&reinterpret_cast<char const volatile&>((((typename(%{clazz})*)0)->idexpr(%{member})))));
                // hopefully this can be fixed one day
                // printf("    offset: %llu\n", offsetof(typename(%{clazz}), idexpr(%{member})));
                printf("    size: %llu\n", sizeof(C::unqualid(%{member})));
            };
        }
    }
    // return buffer.data();
    return {};
}

std::string layout_cmp = R"(---
name: ComponentA
size: 28
fields:
  -
    name: a
    type: int
    offset: 0
    size: 4
  -
    name: b
    type: char
    offset: 4
    size: 1
  -
    name: c
    type: float
    offset: 8
    size: 4
  -
    name: d
    type: char [16]
    offset: 12
    size: 16
)";

TEST(Driver, ComponentLayout)
{
    EXPECT_EQ(layout_string_yaml<ComponentA>(), layout_cmp);
}

Output:

E:\Dev\cppx\build\RelWithDebInfo\bin\clang-cl.exe  /c /I"E:\Dev\Tools\vcpkg\installed\x64-windows-static\include" /Z7 /W3 /WX /diagnostics:column /Od /D _UNICODE /D UNICODE /EHsc /MTd /GS /arch:AVX2 /fp:precise /permissive- /GR /std:c++latest /Fo"x64\Debug\\" /Gd /TP -Xclang -freflection -m64  -Xclang -std=c++20 -Wno-c++98-compat -Wno-pragma-pack -Wno-macro-redefined Component2.cpp
error MSB6006: "clang-cl.exe" exited with code -1073740940.

I recognize that there are many parameters possibly irrelevant to this problem, but these are parameters I'm currently using for my projects and I hope the compiler could deal with them. The error code is actually 0xC0000374 STATUS_HEAP_CORRUPTION. The craziest thing is that it compiles fine with preprocessed source... I initially believed this might be some race condition with multithreaded compilation because there is a very very small chance on my machine that it doesn't crash. But I checked the parameters which do not seem to have /MP option.

I attached a debugger to the compiler and here is what I've got:

I've saved a memory dump from the debug session but I worry that it's not a good idea to post it here. Maybe I can share it via some other channel if it would be helpful.

[YukinoHayakawa]

Hi. Today I've done some tests with the compiler options trying to figure out some way to avoid the crash. I've found that if I remove both /Z7 /MTd, the behavior changes from most likely to crash to most likely not to crash. Like one crash in 10 invocations. So obviously the problem is still there and the stacktrace indicates it's something with the meta code injection. At the site of yesterday's crash, I tried to view the memory casted to the supposed type (shown below). It seems to me that the structure is intact (hope this is not misleading). Maybe I can try to compile a debug version of the compiler and hope that some assertions could catch the problem earlier.

[YukinoHayakawa]

I've built a debug version of the compiler and I ran it with the original command line with /Z7 /MTd. Lucily enough, it did move the error to an earlier phase as shown below:

I belive this is something to do with the debug information but I basically don't have enough experience with the codebase to deal with it. So again I've saved a memory dump in case it would be helpful. Besides, I'm not quite sure that would each run of the compiler result in crash in the same place. Two runs of the debug version all crash at Loc=0x000001c5 though and it's something about injection. I'm very confused why this only happens when I include large headers like gtest.

[YukinoHayakawa]

Well, turns out that the problem here is rather simple and I overlooked it. cast<RecordDecl>(D) fails when D is a CXXMetaprogramDecl simply because it doesn't inherit from RecordDecl anyway. An unchecked cast in the release version would finally result in heap corruption later. Now the problem is how to fix that :)

I'm still trying to make a minimal reproducible sample but my attempts show that the code path does not necessarily reach that point of crash. That really confuses me...

[YukinoHayakawa]

Finally, I was able to make a minimal program that reproduces the crash.

#include <experimental/meta>
#include <experimental/compiler>

using namespace std::experimental;

#include <cstdio>

struct ComponentA
{
};

template <typename C>
void layout_string_yaml()
{
    consteval
    {
        constexpr auto clazz = reflexpr(C);
        -> fragment {
            printf("name: %s\n", meta::name_of(%{clazz}));
        };
    }
}

class B
{
    void foo();
};

void B::foo() // must 
{
    layout_string_yaml<ComponentA>();
}

The key is the code generation must go down the EmitGlobalFunctionDefinition path, i.e. B::foo() implemented outside the class definition. It doesn't matter that layout_string_yaml is templated or not.

The compiler must be debug compiled and run with /Z7 option. It is now clear that why it crashes with a chance in the release version. It's because the invalid casting corrupts the heap. But it's the heap after all so it does not necessarily cause a crash.

E:\Dev\cppx\build\Debug\bin\clang-cl.exe /Z7 /EHsc -Xclang -freflection -m64 -Xclang -std=c++20 c4.cpp
Assertion failed: isa<X>(Val) && "cast<Ty>() argument of incompatible type!", file E:\Dev\cppx\llvm-project\llvm\include\llvm/Support/Casting.h, line 262
PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.      Program arguments: E:\Dev\cppx\build\Debug\bin\clang-cl.exe -cc1 -triple x86_64-pc-windows-msvc19.27.29111 -emit-obj -mrelax-all -mincremental-linker-compatible --mrelax-relocations -disable-free -main-file-name c4.cpp -mrelocation-model pic -pic-level 2 -mframe-pointer=none -relaxed-aliasing -fmath-errno -fno-rounding-math -mconstructor-aliases -munwind-tables -target-cpu x86-64 -mllvm -x86-asm-syntax=intel -tune-cpu generic -D_MT -flto-visibility-public-std --dependent-lib=libcmt --dependent-lib=oldnames -stack-protector 2 -fcxx-exceptions -fexceptions -fexternc-nounwind -fms-volatile -fdiagnostics-format msvc -gno-column-info -gcodeview -debug-info-kind=limited -resource-dir E:\Dev\cppx\build\Debug\lib\clang\12.0.0 -internal-isystem E:\Dev\cppx\build\Debug\lib\clang\12.0.0\include -internal-isystem C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.27.29110\include -internal-isystem C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.27.29110\atlmfc\include -internal-isystem C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\ucrt -internal-isystem C:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared -internal-isystem C:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um -internal-isystem C:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt -fdeprecated-macro -fdebug-compilation-dir *omitted* -ferror-limit 19 -fmessage-length=156 -fno-use-cxa-atexit -fms-extensions -fms-compatibility -fms-compatibility-version=19.27.29111 -std=c++14 -fdelayed-template-parsing -fcolor-diagnostics -freflection -std=c++20 -faddrsig -o C:\Users\YUKINO~1\AppData\Local\Temp\c4-db505f.obj -x c++ c4.cpp
1.      <eof> parser at end of file
2.      Per-file LLVM IR generation
3.      c4.cpp:13:6: Generating code for declaration 'layout_string_yaml'
 #0 0x00007ff770f6006c HandleAbort E:\Dev\cppx\llvm-project\llvm\lib\Support\Windows\Signals.inc:408:0
 #1 0x00007ffaf9e4c3e1 (C:\WINDOWS\SYSTEM32\ucrtbased.dll+0x6c3e1)
 #2 0x00007ffaf9e4e039 (C:\WINDOWS\SYSTEM32\ucrtbased.dll+0x6e039)
 #3 0x00007ffaf9e53c65 (C:\WINDOWS\SYSTEM32\ucrtbased.dll+0x73c65)
 #4 0x00007ffaf9e537d7 (C:\WINDOWS\SYSTEM32\ucrtbased.dll+0x737d7)
 #5 0x00007ffaf9e51841 (C:\WINDOWS\SYSTEM32\ucrtbased.dll+0x71841)
 #6 0x00007ffaf9e541cf (C:\WINDOWS\SYSTEM32\ucrtbased.dll+0x741cf)
 #7 0x00007ff7718f41b5 llvm::cast<class clang::RecordDecl, class clang::Decl const>(class clang::Decl const &) E:\Dev\cppx\llvm-project\llvm\include\llvm\Support\Casting.h:262:0
 #8 0x00007ff7718e4cde clang::CodeGen::CodeGenFunction::EmitDecl(class clang::Decl const &) E:\Dev\cppx\llvm-project\clang\lib\CodeGen\CGDecl.cpp:112:0
 #9 0x00007ff771e49bc4 clang::CodeGen::CodeGenFunction::EmitDeclStmt(class clang::DeclStmt const &) E:\Dev\cppx\llvm-project\clang\lib\CodeGen\CGStmt.cpp:1281:0
#10 0x00007ff771e469c8 clang::CodeGen::CodeGenFunction::EmitSimpleStmt(class clang::Stmt const *) E:\Dev\cppx\llvm-project\clang\lib\CodeGen\CGStmt.cpp:387:0
#11 0x00007ff771e45905 clang::CodeGen::CodeGenFunction::EmitStmt(class clang::Stmt const *, class llvm::ArrayRef<class clang::Attr const *>) E:\Dev\cppx\llvm-project\clang\lib\CodeGen\CGStmt.cpp:54:0
#12 0x00007ff771e470e0 clang::CodeGen::CodeGenFunction::EmitCompoundStmtWithoutScope(class clang::CompoundStmt const &, bool, class clang::CodeGen::AggValueSlot) E:\Dev\cppx\llvm-project\clang\lib\CodeGen\CGStmt.cpp:464:0
#13 0x00007ff771a36d82 clang::CodeGen::CodeGenFunction::EmitFunctionBody(class clang::Stmt const *) E:\Dev\cppx\llvm-project\clang\lib\CodeGen\CodeGenFunction.cpp:1147:0
#14 0x00007ff771a2fd7c clang::CodeGen::CodeGenFunction::GenerateCode(class clang::GlobalDecl, class llvm::Function *, class clang::CodeGen::CGFunctionInfo const &) E:\Dev\cppx\llvm-project\clang\lib\CodeGen\CodeGenFunction.cpp:1313:0
#15 0x00007ff771620cd6 clang::CodeGen::CodeGenModule::EmitGlobalFunctionDefinition(class clang::GlobalDecl, class llvm::GlobalValue *) E:\Dev\cppx\llvm-project\clang\lib\CodeGen\CodeGenModule.cpp:4565:0
#16 0x00007ff7716209d1 clang::CodeGen::CodeGenModule::EmitGlobalDefinition(class clang::GlobalDecl, class llvm::GlobalValue *) E:\Dev\cppx\llvm-project\clang\lib\CodeGen\CodeGenModule.cpp:2925:0
#17 0x00007ff771624eb9 clang::CodeGen::CodeGenModule::EmitDeferred(void) E:\Dev\cppx\llvm-project\clang\lib\CodeGen\CodeGenModule.cpp:2248:0
#18 0x00007ff77160cf0a clang::CodeGen::CodeGenModule::Release(void) E:\Dev\cppx\llvm-project\clang\lib\CodeGen\CodeGenModule.cpp:406:0
#19 0x00007ff778ab14bd `anonymous namespace'::CodeGeneratorImpl::HandleTranslationUnit E:\Dev\cppx\llvm-project\clang\lib\CodeGen\ModuleBuilder.cpp:271:0
#20 0x00007ff778aaacb0 clang::BackendConsumer::HandleTranslationUnit(class clang::ASTContext &) E:\Dev\cppx\llvm-project\clang\lib\CodeGen\CodeGenAction.cpp:291:0
#21 0x00007ff7757de266 clang::ParseAST(class clang::Sema &, bool, bool) E:\Dev\cppx\llvm-project\clang\lib\Parse\ParseAST.cpp:178:0
#22 0x00007ff772312a0b clang::ASTFrontendAction::ExecuteAction(void) E:\Dev\cppx\llvm-project\clang\lib\Frontend\FrontendAction.cpp:1059:0
#23 0x00007ff778a9f26c clang::CodeGenAction::ExecuteAction(void) E:\Dev\cppx\llvm-project\clang\lib\CodeGen\CodeGenAction.cpp:1189:0
#24 0x00007ff77231239c clang::FrontendAction::Execute(void) E:\Dev\cppx\llvm-project\clang\lib\Frontend\FrontendAction.cpp:954:0
#25 0x00007ff772288036 clang::CompilerInstance::ExecuteAction(class clang::FrontendAction &) E:\Dev\cppx\llvm-project\clang\lib\Frontend\CompilerInstance.cpp:984:0
#26 0x00007ff77251cf4d clang::ExecuteCompilerInvocation(class clang::CompilerInstance *) E:\Dev\cppx\llvm-project\clang\lib\FrontendTool\ExecuteCompilerInvocation.cpp:278:0
#27 0x00007ff76e8e02a3 cc1_main(class llvm::ArrayRef<char const *>, char const *, void *) E:\Dev\cppx\llvm-project\clang\tools\driver\cc1_main.cpp:240:0
#28 0x00007ff76e8cbaa5 ExecuteCC1Tool E:\Dev\cppx\llvm-project\clang\tools\driver\driver.cpp:330:0
#29 0x00007ff76e8cc3c0 main E:\Dev\cppx\llvm-project\clang\tools\driver\driver.cpp:407:0
#30 0x00007ff7783fa699 invoke_main D:\agent\_work\9\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:79:0
#31 0x00007ff7783fa53e __scrt_common_main_seh D:\agent\_work\9\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288:0
#32 0x00007ff7783fa3fe __scrt_common_main D:\agent\_work\9\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:331:0
#33 0x00007ff7783fa729 mainCRTStartup D:\agent\_work\9\s\src\vctools\crt\vcstartup\src\startup\exe_main.cpp:17:0
#34 0x00007ffb53837bd4 (C:\WINDOWS\System32\KERNEL32.DLL+0x17bd4)
#35 0x00007ffb5572ce51 (C:\WINDOWS\SYSTEM32\ntdll.dll+0x6ce51)
clang-cl: error: clang frontend command failed due to signal (use -v to see invocation)
clang version 12.0.0 (https://github.com/lock3/meta.git e71f5467e7082d676d8c146d3f2f11bb08cfc823)
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: E:\Dev\cppx\build\Debug\bin
clang-cl: note: diagnostic msg:
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang-cl: note: diagnostic msg: C:\Users\YUKINO~1\AppData\Local\Temp\c4-d9279e.cpp
clang-cl: note: diagnostic msg: C:\Users\YUKINO~1\AppData\Local\Temp\c4-d9279e.sh
clang-cl: note: diagnostic msg:

********************