Open SasisaDev opened 1 year ago
@llvm/issue-subscribers-c-20
@llvm/issue-subscribers-clang-modules
One more example of this issue.
for me crash it's caused by assert(hasASTFileSupport() && "This action does not have AST file support!");
Looks like it would took you a while guys, to fix. Thanks for working on making clang, goodluck!
If i can help in any way, write me anytime.
I can't reproduce this locally. I am not sure if this is Windows related or you use the incorrect command line. Would you like to provide the full command line reproducer?
In your crash stack trace, it looks like you're compiling WorldObject.cppm instead of Consumer.cppm.
@ChuanqiXu9 I attached files of my project in .zip archive. Yes, i misunderstood what causes this issue at first, it's reason is more, than just a templated function export, it turned out. In archieve i sent all files that should be compiled, i guess they should be enough. If they're not enough, contact me, i will make minimal reproducer based on my actual code and will send it to you with more details.
Also i edited original comment, maybe it would be helpful.
@ChuanqiXu9 I attached files of my project in .zip archive. Yes, i misunderstood what causes this issue at first, it's reason is more, than just a templated function export, it turned out. In archieve i sent all files that should be compiled, i guess they should be enough. If they're not enough, contact me, i will make minimal reproducer based on my actual code and will send it to you with more details.
Also i edited original comment, maybe it would be helpful.
I downloaded the codes in the zip and it is a VS project. I can't run it. It would be best to have a minimal reproducer and the corresponding set of flags.
I downloaded the codes in the zip and it is a VS project. I can't run it. It would be best to have a minimal reproducer and the corresponding set of flags.
Sorry for inconvenience! Here is my minimal reproducer. It's the most isolated i was able to actually make. Be aware, that LLVM linker driver i used in make file is window-only. Also, it crashes at 3rd line from the end in make file, and if it doesn't crash after you execute all commands, than it doesn't crash on your PC/Platform/Architecture. It crashes for me on Window 10 & 11, tested on 3 PCs.
Sorry, it has too many files that I can't run it. Could you try to reproduce them further? A pretty good example could be found at https://github.com/llvm/llvm-project/issues/60036
Sorry, it has too many files that I can't run it. Could you try to reproduce them further? A pretty good example could be found at https://github.com/llvm/llvm-project/issues/60036
Sorry, you see, i can't minimize it much further without it starting to compile successfully. To have a crash you need: 2 such templated functions and their appropriate calls(atleast 3 in sum), or you need on such templated function, 2 calls to it and at least 1 Concept. I decided to give you 3 modules, each of which contain key elements for this crash, removing at least one of them stops the crash from appearance.
module "Module" contains ModuleRegister<>() function, and call for it
module "Object" contains Module and static ModuleRegister<>() call, also NObject and ObjectRegister<>() function with it's call.
module "World" contains World object, and ObjectRegister<>() call, at this point crash happens. I may change Module module with Concept in Object module, would crash too
Sorry that i added a lot of useless files tho, i removed them. But i can't remove anything else, because then crash won't happen.
I included file with all commands, that i use to compile. On my PC and few other PCs, these commands cause crash.
@ChuanqiXu9 Hello, sorry for ping, how's this issue going? Do you need anything else maybe? Was you able to run reproducer?
I didn't have progress on this. It'll still be helpful to reduce it further. BTW, I guess you can try to remove the -fmodules
flag and use the newer clang to have a try.
@SasisaDev I think your example can be much simpler if squashed it into two files and remove extra stuff:
export module Module;
export template <class ObjectType> bool ModuleRegister() { return true; };
export struct ModuleEntry {
static const bool bRegistered;
};
const bool ModuleEntry::bRegistered = ModuleRegister<ModuleEntry>();
export module Object;
import Module;
export template <class ObjectType> bool ObjectRegister() { return true; }
export struct NObject {
static const bool bRegistered;
};
export struct ObjectModuleEntry {
static const bool bRegistered;
};
// This function is also required for crash
const bool NObject::bRegistered = ObjectRegister<NObject>();
// One another function, that helps clang crash
const bool ObjectModuleEntry::bRegistered = ModuleRegister<ObjectModuleEntry>();
Does that still fail if you compile in your machine? Or does it only fail if split into multiple parts per module and/or including the "world" module?
We might be able to reduce it further: does it need both NObject
and ObjectModuleEntry
to fail?
I have a similar issue (same exception code and stack dump), but it is harder for me to reproduce it because it only fails running the build in a GitHub workflow (my code works fine in my PC). It would be helpful if we can tweak your example to reproduce with minimal effort for devs. Bonus points if it can eventually become a unit test in llvm, I guess.
@SasisaDev I think your example can be much simpler if squashed it into two files and remove extra stuff:
export module Module; export template <class ObjectType> bool ModuleRegister() { return true; }; export struct ModuleEntry { static const bool bRegistered; }; const bool ModuleEntry::bRegistered = ModuleRegister<ModuleEntry>();
export module Object; import Module; export template <class ObjectType> bool ObjectRegister() { return true; } export struct NObject { static const bool bRegistered; }; export struct ObjectModuleEntry { static const bool bRegistered; }; // This function is also required for crash const bool NObject::bRegistered = ObjectRegister<NObject>(); // One another function, that helps clang crash const bool ObjectModuleEntry::bRegistered = ModuleRegister<ObjectModuleEntry>();
Does that still fail if you compile in your machine? Or does it only fail if split into multiple parts per module and/or including the "world" module?
We might be able to reduce it further: does it need both
NObject
andObjectModuleEntry
to fail?I have a similar issue (same exception code and stack dump), but it is harder for me to reproduce it because it only fails running the build in a GitHub workflow (my code works fine in my PC). It would be helpful if we can tweak your example to reproduce with minimal effort for devs. Bonus points if it can eventually become a unit test in llvm, I guess.
The reproducer works fine for me.
@SasisaDev I think your example can be much simpler if squashed it into two files and remove extra stuff:
export module Module; export template <class ObjectType> bool ModuleRegister() { return true; }; export struct ModuleEntry { static const bool bRegistered; }; const bool ModuleEntry::bRegistered = ModuleRegister<ModuleEntry>();
export module Object; import Module; export template <class ObjectType> bool ObjectRegister() { return true; } export struct NObject { static const bool bRegistered; }; export struct ObjectModuleEntry { static const bool bRegistered; }; // This function is also required for crash const bool NObject::bRegistered = ObjectRegister<NObject>(); // One another function, that helps clang crash const bool ObjectModuleEntry::bRegistered = ModuleRegister<ObjectModuleEntry>();
Does that still fail if you compile in your machine? Or does it only fail if split into multiple parts per module and/or including the "world" module?
We might be able to reduce it further: does it need both
NObject
andObjectModuleEntry
to fail?I have a similar issue (same exception code and stack dump), but it is harder for me to reproduce it because it only fails running the build in a GitHub workflow (my code works fine in my PC). It would be helpful if we can tweak your example to reproduce with minimal effort for devs. Bonus points if it can eventually become a unit test in llvm, I guess.
Your minimal reproducer doesn't crash for me, yet that one example that i sent before still does crash. I have no idea how and why this happens, but i really tried tweaking it further and further, yet latest my reproducer is the minimal i could achieve, if i do anything else to it, it stops from crashing. I'm really sorry, if my reproducer is really hard to work with, but i was not able to reduce it further and still get crashes.
Somehow I got a crash in my Windows machine in my own project and I decided to try yours again. I noticed I get a crash only once every ~10 times.
After a lot of trial-and-error, I was able to import the project into this repo and shrink its size further. I stopped at this tag because it now fails in two different ways (at random, around 50~50 chance). One is the same crash. Other is this message:
E:\Downloads\MinimalReproducerV2\dump>clang++ -std=c++20 -fprebuilt-module-path=. --precompile -o .\World.pcm .\World.cppm
.\World.cppm:5:38: error: declaration of 'ModuleRegister' must be imported from module 'Module:Reflector' before it is required
export const bool NWorldRegistered = ModuleRegister<long>();
^
E:\Downloads\MinimalReproducerV2\dump\Module-Reflector.cppm:3:41: note: declaration here is not visible
export template <class ObjectType> bool ModuleRegister() { return true; };
^
1 error generated.
This is at random. I run build.bat
multiple times and I get different results. The bat
deletes all output files then compiles with minimal flags.
As @SasisaDev mentions, this bug is really hard to reproduce in a smaller setup. Removing code often "fixes" the issue. I was able to remove a bunch of custom types, leaving only the ModuleRegister
template function in a module part named Module:Reflector
, then instantiating it in Module
consumers (Object
and World
).
From what I noticed in my attempts and the error message I may get when building, I suspect there's something off about symbol exposure. World.cppm
does not import Module
, but it imports Object
which imports Module
without exporting it. I think using those symbols without importing should not be possible. And importing Module
in World
is one way of fixing the example. Which makes sense, given the error message I sometimes get.
Removing somewhat unrelated symbols, like the Object:Reflector
stabilizes the error - I get the error above, but I never get the crash anymore. I don't know much of clang internals, but it suggests to me the symbol table size might have a relation to this bug somehow.
@ChuanqiXu9 - I saw your test case, but it was using my first attempt with only two modules. I'm trying to change it to reflect what I was able to reproduce in my machine. Unfortunately, I'm not able to build LLVM in that Windows PC, but I was able to write a patch: https://reviews.llvm.org/D154801 - the World
module unit and the :Reflector
module part units are important to reproduce the issue, as I found out after some trial-and-error.
I hope that helps.
Now, THIS is something I hope can help. I was able to build clang with debug symbols. Running the build.bat
from the example (more specifically, the version I have in this GH repo), it fails with the following stacktrace.
Nonetheless, I noticed this time it fails when compiling Object.cppm
. The "release" version generates the Object.pcm
file and crashes when compiling World.cppm
(which imports Object
).
Assertion failed: ((getOffset()+Offset) & MacroIDBit) == 0 && "offset overflow", file F:\Sandbox\LLVM\llvm-project\clang\include\clang/Basic/SourceLocation.h, line 135
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0. Program arguments: f:\\apps\\llvm.built\\bin\\clang++ -std=c++20 -fprebuilt-module-path=. --precompile -o .\\Object.pcm .\\Object.cppm
1. <eof> parser at end of file
Exception Code: 0x80000003
#0 0x00007ff7b337c14c HandleAbort F:\Sandbox\LLVM\llvm-project\llvm\lib\Support\Windows\Signals.inc:419:0
#1 0x00007ff822d1bc31 (C:\WINDOWS\SYSTEM32\ucrtbased.dll+0x6bc31)
#2 0x00007ff822d1d889 (C:\WINDOWS\SYSTEM32\ucrtbased.dll+0x6d889)
#3 0x00007ff822d234b5 (C:\WINDOWS\SYSTEM32\ucrtbased.dll+0x734b5)
#4 0x00007ff822d23027 (C:\WINDOWS\SYSTEM32\ucrtbased.dll+0x73027)
#5 0x00007ff822d21091 (C:\WINDOWS\SYSTEM32\ucrtbased.dll+0x71091)
#6 0x00007ff822d23a1f (C:\WINDOWS\SYSTEM32\ucrtbased.dll+0x73a1f)
#7 0x00007ff7b37e4037 clang::SourceLocation::getLocWithOffset(int) const F:\Sandbox\LLVM\llvm-project\clang\include\clang\Basic\SourceLocation.h:135:0
#8 0x00007ff7b504ada1 clang::ASTReader::TranslateSourceLocation(class clang::serialization::ModuleFile &, class clang::SourceLocation) const F:\Sandbox\LLVM\llvm-project\clang\include\clang\Serialization\ASTReader.h:2234:0
#9 0x00007ff7b5048791 clang::ASTReader::ReadSourceLocation(class clang::serialization::ModuleFile &, unsigned int, class clang::SourceLocationSequence *) const F:\Sandbox\LLVM\llvm-project\clang\include\clang\Serialization\ASTReader.h:2220:0
#10 0x00007ff7b5048705 clang::ASTReader::ReadSourceLocation(class clang::serialization::ModuleFile &, class llvm::SmallVectorImpl<unsigned __int64> const &, unsigned int &, class clang::SourceLocationSequence *) F:\Sandbox\LLVM\llvm-project\clang\include\clang\Serialization\ASTReader.h:2241:0
#11 0x00007ff7b4f92200 clang::ASTReader::ReadToken(class clang::serialization::ModuleFile &, class llvm::SmallVectorImpl<unsigned __int64> const &, unsigned int &) F:\Sandbox\LLVM\llvm-project\clang\lib\Serialization\ASTReader.cpp:1688:0
#12 0x00007ff7b4f8da67 clang::ASTReader::ReadLateParsedTemplates(class llvm::MapVector<class clang::FunctionDecl const *, class std::unique_ptr<struct clang::LateParsedTemplate, struct std::default_delete<struct clang::LateParsedTemplate>>, class llvm::DenseMap<class clang::FunctionDecl const *, unsigned int, struct llvm::DenseMapInfo<class clang::FunctionDecl const *, void>, struct llvm::detail::DenseMapPair<class clang::FunctionDecl const *, unsigned int>>, class std::vector<struct std::pair<class clang::FunctionDecl const *, class std::unique_ptr<struct clang::LateParsedTemplate, struct std::default_delete<struct clang::LateParsedTemplate>>>, class std::allocator<struct std::pair<class clang::FunctionDecl const *, class std::unique_ptr<struct clang::LateParsedTemplate, struct std::default_delete<struct clang::LateParsedTemplate>>>>>> &) F:\Sandbox\LLVM\llvm-project\clang\lib\Serialization\ASTReader.cpp:8617:0
#13 0x00007ff7ba73494f clang::Sema::InstantiateFunctionDefinition(class clang::SourceLocation, class clang::FunctionDecl *, bool, bool, bool) F:\Sandbox\LLVM\llvm-project\clang\lib\Sema\SemaTemplateInstantiateDecl.cpp:4943:0
#14 0x00007ff7ba7320fe clang::Sema::PerformPendingInstantiations(bool) F:\Sandbox\LLVM\llvm-project\clang\lib\Sema\SemaTemplateInstantiateDecl.cpp:6374:0
#15 0x00007ff7b8b5d91d clang::Sema::ActOnEndOfTranslationUnitFragment(enum clang::Sema::TUFragmentKind) F:\Sandbox\LLVM\llvm-project\clang\lib\Sema\Sema.cpp:1083:0
#16 0x00007ff7b8b5b2fa clang::Sema::ActOnEndOfTranslationUnit(void) F:\Sandbox\LLVM\llvm-project\clang\lib\Sema\Sema.cpp:1124:0
#17 0x00007ff7b896d6e2 clang::Parser::ParseTopLevelDecl(class clang::OpaquePtr<class clang::DeclGroupRef> &, enum clang::Sema::ModuleImportState &) F:\Sandbox\LLVM\llvm-project\clang\lib\Parse\Parser.cpp:711:0
#18 0x00007ff7b8968d7d clang::ParseAST(class clang::Sema &, bool, bool) F:\Sandbox\LLVM\llvm-project\clang\lib\Parse\ParseAST.cpp:163:0
#19 0x00007ff7b4cf635a clang::ASTFrontendAction::ExecuteAction(void) F:\Sandbox\LLVM\llvm-project\clang\lib\Frontend\FrontendAction.cpp:1170:0
#20 0x00007ff7b4cf5c4e clang::FrontendAction::Execute(void) F:\Sandbox\LLVM\llvm-project\clang\lib\Frontend\FrontendAction.cpp:1062:0
#21 0x00007ff7b4c6b36c clang::CompilerInstance::ExecuteAction(class clang::FrontendAction &) F:\Sandbox\LLVM\llvm-project\clang\lib\Frontend\CompilerInstance.cpp:1049:0
#22 0x00007ff7b4f41610 clang::ExecuteCompilerInvocation(class clang::CompilerInstance *) F:\Sandbox\LLVM\llvm-project\clang\lib\FrontendTool\ExecuteCompilerInvocation.cpp:272:0
#23 0x00007ff7b0acbbc5 cc1_main(class llvm::ArrayRef<char const *>, char const *, void *) F:\Sandbox\LLVM\llvm-project\clang\tools\driver\cc1_main.cpp:249:0
#24 0x00007ff7b0aac9c3 ExecuteCC1Tool F:\Sandbox\LLVM\llvm-project\clang\tools\driver\driver.cpp:366:0
#25 0x00007ff7b0aaec31 `clang_main'::`43'::<lambda_1>::operator() F:\Sandbox\LLVM\llvm-project\clang\tools\driver\driver.cpp:507:0
#26 0x00007ff7b0ab5e46 llvm::function_ref<int __cdecl(llvm::SmallVectorImpl<char const *> &)>::callback_fn<`clang_main'::`43'::<lambda_1> > F:\Sandbox\LLVM\llvm-project\llvm\include\llvm\ADT\STLFunctionalExtras.h:47:0
#27 0x00007ff7b4a19027 llvm::function_ref<(class llvm::SmallVectorImpl<char const *> &)>::operator()(class llvm::SmallVectorImpl<char const *> &) const F:\Sandbox\LLVM\llvm-project\llvm\include\llvm\ADT\STLFunctionalExtras.h:69:0
#28 0x00007ff7b4a14a71 `clang::driver::CC1Command::Execute'::`7'::<lambda_1>::operator() F:\Sandbox\LLVM\llvm-project\clang\lib\Driver\Job.cpp:440:0
#29 0x00007ff7b4a180d4 llvm::function_ref<void __cdecl(void)>::callback_fn<`clang::driver::CC1Command::Execute'::`7'::<lambda_1> > F:\Sandbox\LLVM\llvm-project\llvm\include\llvm\ADT\STLFunctionalExtras.h:47:0
#30 0x00007ff7b17be3a4 llvm::function_ref<(void)>::operator()(void) const F:\Sandbox\LLVM\llvm-project\llvm\include\llvm\ADT\STLFunctionalExtras.h:69:0
#31 0x00007ff7b334d23b llvm::CrashRecoveryContext::RunSafely(class llvm::function_ref<(void)>) F:\Sandbox\LLVM\llvm-project\llvm\lib\Support\CrashRecoveryContext.cpp:235:0
#32 0x00007ff7b4a12a04 clang::driver::CC1Command::Execute(class llvm::ArrayRef<class std::optional<class llvm::StringRef>>, class std::basic_string<char, struct std::char_traits<char>, class std::allocator<char>> *, bool *) const F:\Sandbox\LLVM\llvm-project\clang\lib\Driver\Job.cpp:440:0
#33 0x00007ff7b48da5ae clang::driver::Compilation::ExecuteCommand(class clang::driver::Command const &, class clang::driver::Command const *&, bool) const F:\Sandbox\LLVM\llvm-project\clang\lib\Driver\Compilation.cpp:199:0
#34 0x00007ff7b48da82a clang::driver::Compilation::ExecuteJobs(class clang::driver::JobList const &, class llvm::SmallVectorImpl<struct std::pair<int, class clang::driver::Command const *>> &, bool) const F:\Sandbox\LLVM\llvm-project\clang\lib\Driver\Compilation.cpp:253:0
#35 0x00007ff7b4869e03 clang::driver::Driver::ExecuteCompilation(class clang::driver::Compilation &, class llvm::SmallVectorImpl<struct std::pair<int, class clang::driver::Command const *>> &) F:\Sandbox\LLVM\llvm-project\clang\lib\Driver\Driver.cpp:1896:0
#36 0x00007ff7b0aae813 clang_main(int, char **, struct llvm::ToolContext const &) F:\Sandbox\LLVM\llvm-project\clang\tools\driver\driver.cpp:542:0
#37 0x00007ff7b0af9f16 main F:\Sandbox\LLVM\build\tools\clang\tools\driver\clang-driver.cpp:16:0
#38 0x00007ff7bc953409 invoke_main D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:79:0
#39 0x00007ff7bc9532ae __scrt_common_main_seh D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288:0
#40 0x00007ff7bc95316e __scrt_common_main D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:331:0
#41 0x00007ff7bc95349e mainCRTStartup D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_main.cpp:17:0
#42 0x00007ff8484c7614 (C:\WINDOWS\System32\KERNEL32.DLL+0x17614)
#43 0x00007ff8495226b1 (C:\WINDOWS\SYSTEM32\ntdll.dll+0x526b1)
clang++: error: clang frontend command failed due to signal (use -v to see invocation)
clang version 17.0.0
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: f:\apps\llvm.built\bin
clang++: note: diagnostic msg:
********************
PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang++: note: diagnostic msg: C:\Users\eduar\AppData\Local\Temp\Object-644e9f.cppm
clang++: note: diagnostic msg: C:\Users\eduar\AppData\Local\Temp\Object-644e9f.sh
clang++: note: diagnostic msg:
********************
Also, in my personal endeavours, I found #61065 which has a root cause very similar to what I see in this one:
The rootcause seems to be that we do not run codegen for an instantiation of a member function coming from a module, but run codegen for a call to this function. This happens only in one particular TU that imports certain combination of template instantiations from other modules and instantiates their uses on its own.
But, #61065 failure is related to ctors, while this one looks more like function instantiation.
Original reproduction code:
Causes this:
This is archive with code dependencies, generated by Clang on crash:
WorldObject-669c29.zip