search

xmake-io / xmake

🔥 A cross-platform build utility based on Lua
https://xmake.io
Apache License 2.0
9.85k stars 774 forks source link

Module compilation error #2716

Closed Latias94 closed 2 years ago

Latias94 commented 2 years ago

Xmake Version

xmake v2.6.9+dev.0864717

Operating System Version and Architecture

WSL2 Ubuntu 22.04

Describe Bug

Test files and build logs are as below.
ModuleTest.zip

It contains two targets:

  1. module include legacy header files (build success with msvc,but failed with gcc)
  2. module partition

The code is following the book <Professional C++, 5th Edition>, so maybe there are compiler issues in it, but I don't dive into it.

Expected Behavior

Build

Project Configuration

set_languages("c++20")
target("hello")
    set_kind("binary")
    add_files("src/*.cpp","src/*.cppm")

Additional Information and Error Logs

1.

~/study/cpp/01_Person via 🌙 v5.4.4 
23:32:23 ❯ xmake -rv
checking for platform ... linux
checking for architecture ... x86_64
checking for gcc ... /usr/bin/gcc
checking for dmd ... no
checking for ldc2 ... no
checking for gdc ... no
checking for zig ... no
checking for zig ... no
checking for gcc ... /usr/bin/gcc
checking for the c++ compiler (cxx) ... gcc
checking for /usr/bin/gcc ... ok
checking for flags (-fPIC) ... ok
checking for flags (gcc_modules_ts) ... ok
checking for flags (gcc_module_mapper) ... ok
checking for flags (gcc_dep_format) ... no
checking for flags (gcc_dep_file) ... no
checking for flags (gcc_dep_output) ... no
[  0%]: generating.cxx.module.deps src/test.cpp
checking for gcc ... /usr/bin/gcc
checking for the c compiler (cc) ... gcc
[  0%]: generating.cxx.module.deps src/Person.cppm
[ 18%]: generating.cxx.headerunit.bmi iostream
[ 18%]: generating.cxx.headerunit.bmi string
checking for flags (-std=c++20) ... ok
checking for flags (-fmodules-ts) ... ok
/usr/bin/gcc -m64 -std=c++20 -fmodules-ts -fmodule-mapper=/dev/shm/.xmake1000/220823/_3385625651234B2083BF312399354B50 -c -x c++-system-header iostream
/usr/bin/gcc -m64 -std=c++20 -fmodules-ts -fmodule-mapper=/dev/shm/.xmake1000/220823/_337647DD4D164C3083FE32036A715770 -c -x c++-system-header string
[ 63%]: generating.cxx.module.bmi person
/usr/bin/gcc -m64 -std=c++20 -fmodules-ts -fmodule-mapper=build/mapper.txt -x c++ -o build/.objs/hello/linux/x86_64/release/src/Person.cppm.o -c src/Person.cppm
[ 72%]: ccache compiling.release src/test.cpp
/usr/bin/gcc -c -m64 -std=c++20 -fmodules-ts -fmodule-mapper=build/mapper.txt -o build/.objs/hello/linux/x86_64/release/src/test.cpp.o src/test.cpp
checking for flags (-MMD -MF) ... ok
checking for flags (-fdiagnostics-color=always) ... ok
checking for g++ ... /usr/bin/g++
checking for the linker (ld) ... g++
checking for /usr/bin/g++ ... ok
checking for flags (-fPIC) ... ok
[ 81%]: linking.release hello
/usr/bin/g++ -o build/linux/x86_64/release/hello build/.objs/hello/linux/x86_64/release/src/test.cpp.o build/.objs/hello/linux/x86_64/release/src/Person.cppm.o -m64
error: /usr/bin/ld: build/.objs/hello/linux/x86_64/release/src/test.cpp.o: in function `main':
test.cpp:(.text+0x15a): undefined reference to `_ZNKW6person6Person11getLastNameEv'
/usr/bin/ld: test.cpp:(.text+0x190): undefined reference to `_ZNKW6person6Person12getFirstNameEv'
collect2: error: ld returned 1 exit status

2.

~/study/cpp/08_ImplementationPartitions via 🌙 v5.4.4 
23:33:16 ❯ xmake -rv
checking for platform ... linux
checking for architecture ... x86_64
checking for gcc ... /usr/bin/gcc
checking for dmd ... no
checking for ldc2 ... no
checking for gdc ... no
checking for zig ... no
checking for zig ... no
checking for gcc ... /usr/bin/gcc
checking for the c++ compiler (cxx) ... gcc
checking for /usr/bin/gcc ... ok
checking for flags (-fPIC) ... ok
checking for flags (gcc_modules_ts) ... ok
checking for flags (gcc_module_mapper) ... ok
checking for flags (gcc_dep_format) ... no
checking for flags (gcc_dep_file) ... no
checking for flags (gcc_dep_output) ... no
[  0%]: generating.cxx.module.deps src/math.cpp
error: attempt to concatenate a nil value (local 'module_name')
waruqi commented 2 years ago

01_Person/gcc

Different c++abi are generated in cpp.o and cppm.o. @Arthapz 

01_Person$ nm build/.objs/hello/linux/x86_64/release/src/test.cpp.o | grep getLast
                 U _ZNK6Person11getLastNameEv
01_Person$ nm build/.objs/hello/linux/x86_64/release/src/Person.cppm.o | grep getLast
00000000000001ba T _ZNK6Person11getLastNameB5cxx11Ev
01_Person$ c++filt _ZNK6Person11getLastNameB5cxx11Ev
Person::getLastName[abi:cxx11]() const
01_Person$ c++filt _ZNK6Person11getLastNameEv
Person::getLastName() const

I don't know why, maybe it's a gcc bug, but we can avoid it by defining _GLIBCXX_USE_CXX11_ABI=0. This is probably just a workaround solution.

01_Person$ xmake f --cxflags="-D_GLIBCXX_USE_CXX11_ABI=0" -c;xmake -r 
checking for platform ... linux
checking for architecture ... x86_64
[  0%]: generating.cxx.module.deps src/test.cpp
[  0%]: generating.cxx.module.deps src/Person.cppm
[ 55%]: generating.cxx.module.bmi person
[ 66%]: ccache compiling.release src/test.cpp
[ 77%]: linking.release hello
[100%]: build ok!
waruqi commented 2 years ago

you can try dev version, xmake update -s dev

I added _GLIBCXX_USE_CXX11_ABI=0 to fix this problem 1.

About problem 1, module partition, you need modify example.

rename math_helpers.cpp to math_helpers.cppm, and modify module math:details; to export module math:details;

see https://github.com/xmake-io/xmake/tree/dev/tests/projects/c%2B%2B/modules/partitions2/src

Latias94 commented 2 years ago

thanks

Latias94 commented 2 years ago

About problem 2 module partition on my machine, it compile with msvc and WSL2 ubuntu's g++. but failed with msys2 mingw64.

checking for flags (gcc_dep_format) ... no
checking for flags (gcc_dep_file) ... no
checking for flags (gcc_dep_output) ... no
[  0%]: generating.cxx.module.deps src\math.cpp
[  0%]: generating.cxx.module.deps src\test.cpp
[  0%]: generating.cxx.module.deps src\math.mpp
[  0%]: generating.cxx.module.deps src\math_helpers.mpp
[ 45%]: generating.cxx.module.bmi math:details
[ 45%]: generating.cxx.module.bmi math
checking for flags (-std=c++20) ... ok
checking for flags (-fmodules-ts) ... ok
checking for flags (-D_GLIBCXX_USE_CXX11_ABI=0) ... ok
F:\MagicRoot\msys64\mingw64\bin\x86_64-w64-mingw32-g++ -m64 -std=c++20 -fmodules-ts -fmodule-mapper=build\mapper.txt -D_GLIBCXX_USE_CXX11_ABI=0 -x c++ -o build\.objs\hello\mingw\x86_64\release\src\math_helpers.mpp.obj -c src\math_helpers.mpp
F:\MagicRoot\msys64\mingw64\bin\x86_64-w64-mingw32-g++ -m64 -std=c++20 -fmodules-ts -fmodule-mapper=build\mapper.txt -D_GLIBCXX_USE_CXX11_ABI=0 -x c++ -o build\.objs\hello\mingw\x86_64\release\src\math.mpp.obj -c src\math.mpp
src\math_helpers.mpp:1:8: error: unknown Compiled Module Interface: no such module
    1 | export module math:details;  // math:details implementation partition
      |        ^~~~~~
src\math_helpers.mpp:1:8: warning: not writing module 'math:details' due to errors
src\math.mpp:1:8: error: unknown Compiled Module Interface: no such module
    1 | export module math; // math module declaration
      |        ^~~~~~
src\math.mpp:1:8: warning: not writing module 'math' due to errors
error: execv(F:\MagicRoot\msys64\mingw64\bin\x86_64-w64-mingw32-g++ -m64 -std=c++20 -fmodules-ts -fmodule-mapper=build\mapper.txt -D_GLIBCXX_USE_CXX11_ABI=0 -x c++ -o build\.objs\hello\mingw\x86_64\release\src\math_helpers.mpp.obj -c src\math_helpers.mpp) failed(1)
waruqi commented 2 years ago

gcc11? let me see your build\mapper.txt

Latias94 commented 2 years ago 
❯ x86_64-w64-mingw32-g++ -v
Using built-in specs.
COLLECT_GCC=F:\MagicRoot\msys64\mingw64\bin\x86_64-w64-mingw32-g++.exe
COLLECT_LTO_WRAPPER=F:/MagicRoot/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/lto-wrapper.exe
Target: x86_64-w64-mingw32
Configured with: ../gcc-12.2.0/configure --prefix=/mingw64 --with-local-prefix=/mingw64/local --build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --with-native-system-header-dir=/mingw64/include --libexecdir=/mingw64/lib --enable-bootstrap --enable-checking=release --with-arch=x86-64 --with-tune=generic --enable-languages=c,lto,c++,fortran,ada,objc,obj-c++,jit --enable-shared --enable-static --enable-libatomic --enable-threads=posix --enable-graphite --enable-fully-dynamic-string --enable-libstdcxx-filesystem-ts --enable-libstdcxx-time --disable-libstdcxx-pch --enable-lto --enable-libgomp --disable-multilib --disable-rpath --disable-win32-registry --disable-nls --disable-werror --disable-symvers --with-libiconv --with-system-zlib --with-gmp=/mingw64 --with-mpfr=/mingw64 --with-mpc=/mingw64 --with-isl=/mingw64 --with-pkgversion='Rev1, Built by MSYS2 project' --with-bugurl=https://github.com/msys2/MINGW-packages/issues --with-gnu-as --with-gnu-ld --disable-libstdcxx-debug --with-boot-ldflags=-static-libstdc++ --with-stage1-ldflags=-static-libstdc++
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 12.2.0 (Rev1, Built by MSYS2 project)

mapper.txt

math:details D:\Github-Projects\xmake\tests\projects\c++\modules\partitions2\build\.gens\hello\mingw\x86_64\release\rules\modules\cache\math:details.gcm
math D:\Github-Projects\xmake\tests\projects\c++\modules\partitions2\build\.gens\hello\mingw\x86_64\release\rules\modules\cache\math.gcm
Latias94 commented 2 years ago 
❯ cd D:\Github-Projects\xmake\tests\projects\c++\modules\hello
c++\modules\hello [master][🌙 v5.1.5]
❯ xmake f -p mingw
c++\modules\hello [master][🌙 v5.1.5]
❯ xmake -rv
[  0%]: generating.cxx.module.deps src\main.cpp
[  0%]: generating.cxx.module.deps src\hello.mpp
[ 55%]: generating.cxx.module.bmi hello
F:\MagicRoot\msys64\mingw64\bin\x86_64-w64-mingw32-g++ -m64 -std=c++20 -fmodules-ts -fmodule-mapper=d:\Github-Projects\xmake\tests\projects\c++\modules\hello\build\mapper.txt -D_GLIBCXX_USE_CXX11_ABI=0 -x c++ -o d:\Github-Projects\xmake\tests\projects\c++\modules\hello\build\.objs\hello\mingw\x86_64\release\src\hello.mpp.obj -c src\hello.mpp
src\hello.mpp:4:8: error: unknown Compiled Module Interface: no such module
    4 | export module hello;
      |        ^~~~~~
src\hello.mpp:4:8: warning: not writing module 'hello' due to errors
error: execv(F:\MagicRoot\msys64\mingw64\bin\x86_64-w64-mingw32-g++ -m64 -std=c++20 -fmodules-ts -fmodule-mapper=d:\Github-Projects\xmake\tests\projects\c++\modules\hello\build\mapper.txt -D_GLIBCXX_USE_CXX11_ABI=0 -x c++ -o d:\Github-Projects\xmake\tests\projects\c++\modules\hello\build\.objs\hello\mingw\x86_64\release\src\hello.mpp.obj -c src\hello.mpp) failed(1)

mapper.txt

hello d:\Github-Projects\xmake\tests\projects\c++\modules\hello\build\.gens\hello\mingw\x86_64\release\rules\modules\cache\hello.gcm
waruqi commented 2 years ago

maybe path issues in mapper, you can try use / instead of \

Latias94 commented 2 years ago

maybe path issues in mapper, you can try use / instead of \

not work.

I try compile module manually, it seems the mpp extension not work.

c++\modules\hello [master][!][🌙 v5.1.5]
❯ cd src
modules\hello\src [master][!]
❯ ls

   Directory: D:\Github-Projects\xmake\tests\projects\c++\modules\hello\src

hello.mpp  main.cpp
modules\hello\src [master][!]
❯ g++ -std=c++20 -fmodules-ts -c .\hello.mpp
g++.exe: warning: .\hello.mpp: linker input file unused because linking not done
modules\hello\src [master][!]
❯ mv .\hello.mpp .\hello.cpp 
modules\hello\src [master][✘!?]
❯ g++ -std=c++20 -fmodules-ts -c .\hello.cpp
modules\hello\src [master][✘!?]
❯ g++ -std=c++20 -fmodules-ts -c .\main.cpp 
modules\hello\src [master][✘!?]
❯ g++ main.o hello.o -o hello
modules\hello\src [master][✘!?]
❯ .\hello
hello module!

Then I don't want to touch mingw toolchain on windows any more 🤦‍♂️

waruqi commented 2 years ago

I tried to upgrade mingw gcc to 12.0, but it does not work for all cpp files https://github.com/msys2/MINGW-packages/issues/12791. I may not be able to help you much.

Latias94 commented 2 years ago

fine, i will stick with msvc on window

Latias94 commented 2 years ago

test_module_partition_with_template.zip

I get another demo of module partition with template here. Please ignore if it is a compiler issue.

❯ xmake
checking for platform ... windows
checking for architecture ... x64
checking for Microsoft Visual Studio (x64) version ... 2022
[  0%]: generating.cxx.module.deps src\GridTest.cpp
[  0%]: generating.cxx.module.deps src\Grid.cppm
[  0%]: generating.cxx.module.deps src\GridDefinition.cppm
[  0%]: generating.cxx.module.deps src\GridImplementation.cppm
[ 10%]: generating.cxx.headerunit.bmi memory
[ 10%]: generating.cxx.headerunit.bmi vector
[ 10%]: generating.cxx.headerunit.bmi string
[ 10%]: generating.cxx.headerunit.bmi optional
[ 10%]: generating.cxx.headerunit.bmi stdexcept
[ 10%]: generating.cxx.headerunit.bmi utility
[ 60%]: generating.cxx.module.bmi grid:definition
[ 65%]: generating.cxx.module.bmi grid:implementation
[ 70%]: generating.cxx.module.bmi grid
[ 85%]: cache compiling.release src\GridTest.cpp
[ 90%]: linking.release module_template.exe
error: GridTest.cpp.obj : error LNK2019: 无法解析的外部符号 "public: __cdecl Grid<int>::Grid<int>(unsigned __int64,unsigned __int64)" (??0?$Grid@H@@QEAA@_K0@Z::<!grid>)，函数 main 中引用了该符号
GridTest.cpp.obj : error LNK2019: 无法解析的外部符号 "public: class std::optional<int> & __cdecl Grid<int>::at(unsigned __int64,unsigned __int64)" (?at@?$Grid@H@@QEAAAEAV?$optional@H@std@@_K0@Z::<!grid>)，函数 main 中引用了该符号
GridTest.cpp.obj : error LNK2019: 无法解析的外部符号 "public: __cdecl Grid<double>::Grid<double>(unsigned __int64,unsigned __int64)" (??0?$Grid@N@@QEAA@_K0@Z::<!grid>)，函数 main 中引用了该符号
GridTest.cpp.obj : error LNK2019: 无法解析的外部符号 "public: __cdecl Grid<char const *>::Grid<char const *>(unsigned __int64,unsigned __int64)" (??0?$Grid@PEBD@@QEAA@_K0@Z::<!grid>)，函数 main 中引用了该符号
GridTest.cpp.obj : error LNK2019: 无法解析的外部符号 "public: class std::optional<char const *> & __cdecl Grid<char const *>::at(unsigned __int64,unsigned __int64)" (?at@?$Grid@PEBD@@QEAAAEAV?$optional@PEBD@std@@_K0@Z::<!grid>)，函数 main 中引用了该符号
GridTest.cpp.obj : error LNK2019: 无法解析的外部符号 "public: __cdecl Grid<class std::vector<int,class std::allocator<int> > >::Grid<class std::vector<int,class std::allocator<int> > >(unsigned __int64,unsigned __int64)" (??0?$Grid@V?$vector@HV?$allocator@H@std@@@std@@@@QEAA@_K0@Z::<!grid>)，函数 main 中引用了该符号
GridTest.cpp.obj : error LNK2019: 无法解析的外部符号 "public: class std::optional<class std::vector<int,class std::allocator<int> > > & __cdecl Grid<class std::vector<int,class std::allocator<int> > >::at(unsigned __int64,unsigned __int64)" (?at@?$Grid@V?$vector@HV?$allocator@H@std@@@std@@@@QEAAAEAV?$optional@V?$vector@HV?$allocator@H@std@@@std@@@std@@_K0@Z::<!grid>)，函数 main 中引用了该符号
build\windows\x64\release\module_template.exe : fatal error LNK1120: 7 个无法解析的外部命令
waruqi commented 2 years ago

test_module_partition_with_template.zip

I get another demo of module partition with template here. Please ignore if it is a compiler issue.

❯ xmake
checking for platform ... windows
checking for architecture ... x64
checking for Microsoft Visual Studio (x64) version ... 2022
[  0%]: generating.cxx.module.deps src\GridTest.cpp
[  0%]: generating.cxx.module.deps src\Grid.cppm
[  0%]: generating.cxx.module.deps src\GridDefinition.cppm
[  0%]: generating.cxx.module.deps src\GridImplementation.cppm
[ 10%]: generating.cxx.headerunit.bmi memory
[ 10%]: generating.cxx.headerunit.bmi vector
[ 10%]: generating.cxx.headerunit.bmi string
[ 10%]: generating.cxx.headerunit.bmi optional
[ 10%]: generating.cxx.headerunit.bmi stdexcept
[ 10%]: generating.cxx.headerunit.bmi utility
[ 60%]: generating.cxx.module.bmi grid:definition
[ 65%]: generating.cxx.module.bmi grid:implementation
[ 70%]: generating.cxx.module.bmi grid
[ 85%]: cache compiling.release src\GridTest.cpp
[ 90%]: linking.release module_template.exe
error: GridTest.cpp.obj : error LNK2019: 无法解析的外部符号 "public: __cdecl Grid<int>::Grid<int>(unsigned __int64,unsigned __int64)" (??0?$Grid@H@@QEAA@_K0@Z::<!grid>)，函数 main 中引用了该符号
GridTest.cpp.obj : error LNK2019: 无法解析的外部符号 "public: class std::optional<int> & __cdecl Grid<int>::at(unsigned __int64,unsigned __int64)" (?at@?$Grid@H@@QEAAAEAV?$optional@H@std@@_K0@Z::<!grid>)，函数 main 中引用了该符号
GridTest.cpp.obj : error LNK2019: 无法解析的外部符号 "public: __cdecl Grid<double>::Grid<double>(unsigned __int64,unsigned __int64)" (??0?$Grid@N@@QEAA@_K0@Z::<!grid>)，函数 main 中引用了该符号
GridTest.cpp.obj : error LNK2019: 无法解析的外部符号 "public: __cdecl Grid<char const *>::Grid<char const *>(unsigned __int64,unsigned __int64)" (??0?$Grid@PEBD@@QEAA@_K0@Z::<!grid>)，函数 main 中引用了该符号
GridTest.cpp.obj : error LNK2019: 无法解析的外部符号 "public: class std::optional<char const *> & __cdecl Grid<char const *>::at(unsigned __int64,unsigned __int64)" (?at@?$Grid@PEBD@@QEAAAEAV?$optional@PEBD@std@@_K0@Z::<!grid>)，函数 main 中引用了该符号
GridTest.cpp.obj : error LNK2019: 无法解析的外部符号 "public: __cdecl Grid<class std::vector<int,class std::allocator<int> > >::Grid<class std::vector<int,class std::allocator<int> > >(unsigned __int64,unsigned __int64)" (??0?$Grid@V?$vector@HV?$allocator@H@std@@@std@@@@QEAA@_K0@Z::<!grid>)，函数 main 中引用了该符号
GridTest.cpp.obj : error LNK2019: 无法解析的外部符号 "public: class std::optional<class std::vector<int,class std::allocator<int> > > & __cdecl Grid<class std::vector<int,class std::allocator<int> > >::at(unsigned __int64,unsigned __int64)" (?at@?$Grid@V?$vector@HV?$allocator@H@std@@@std@@@@QEAAAEAV?$optional@V?$vector@HV?$allocator@H@std@@@std@@@std@@_K0@Z::<!grid>)，函数 main 中引用了该符号
build\windows\x64\release\module_template.exe : fatal error LNK1120: 7 个无法解析的外部命令

I don't know, you can check the full command line arguments in xmake -v, I don't see any problems. You can also run the command manually to troubleshoot the problem.

waruqi commented 7 months ago

try this patch, it should work now. https://github.com/xmake-io/xmake/pull/4673