xmake-io / xmake

🔥 A cross-platform build utility based on Lua
https://xmake.io
Apache License 2.0
9.79k stars 773 forks source link

Cannot create static library providing C++ module #5359

Open iDingDong opened 1 month ago

iDingDong commented 1 month ago

Xmake Version

2.9.3

Operating System Version and Architecture

Windows 11 23H2

Describe Bug

I attempted to compile the following code with xmake and MSVC:

test.cppm

export module t1;

xmake.lua

target("testm")
  set_kind("static")
  set_languages("cxxlatest")
  add_files("./test.cppm")
target_end()

The command xmake failed with the following output:

checking for platform ... windows
checking for architecture ... x64
checking for Microsoft Visual Studio (x64) version ... 2022
checking for Microsoft C/C++ Compiler (x64) version ... 19.40.33812
[  0%]: <testm> generating.module.deps test.cppm
[  0%]: <testm> generating.module.deps C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\modules\std.ixx
[  0%]: <testm> generating.module.deps C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\modules\std.compat.ixx
[ 50%]: <testm> compiling.bmi.release std
[ 66%]: <testm> compiling.bmi.release std.compat
[ 83%]: archiving.release testm.lib
error: LINK : fatal error LNK1181: cannot open input file 'build\.objs\testm\windows\x64\release\test.cppm.obj'

Expected Behavior

The code compiles and gives me testm.lib

Project Configuration

testsxm.zip

Additional Information and Error Logs

When compiling with `_vD' xmake outputs:

checking for platform ... windows
checking for architecture ... x64
checking for cl.exe ... C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\bin\HostX64\x64\cl.exe
checking for Microsoft Visual Studio (x64) version ... 2022
checking for Microsoft C/C++ Compiler (x64) version ... 19.40.33812
checkinfo: cannot runv(zig.exe version), No such file or directory
checking for zig ... no
checkinfo: cannot runv(zig.exe version), No such file or directory
checking for zig ... no
checkinfo: cannot runv(nim.exe --version), No such file or directory
checking for nim ... no
checkinfo: cannot runv(nim.exe --version), No such file or directory
checking for nim ... no
checking for cl.exe ... C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\bin\HostX64\x64\cl.exe
checking for the c++ compiler (cxx) ... cl.exe
checking for C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\bin\HostX64\x64\cl.exe ... ok
checking for flags (cl_scan_dependencies) ... ok
> cl.exe "-scanDependencies" "C:\Users\*****~1\AppData\Local\Temp\.xmake\240718\_3D6AA89633D343608A3B474C0A3F5760.json" "-nologo"
checking for flags (cl_ifc_output) ... ok
> cl.exe "-ifcOutput" "C:\Users\*****~1\AppData\Local\Temp\.xmake\240718\_9FE53270D1A34A50891A83A095BCD9A0" "-nologo"
[  0%]: <testm> generating.module.deps test.cppm
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\bin\HostX64\x64\cl.exe -nologo -std:c++latest /EHsc -TP -scanDependencies build\.gens\testm\windows\x64\release\rules\bmi\cache\modules\ac7906e3\test.cppm.module.json test.cppm -ifcOutput build\.gens\testm\windows\x64\release\rules\bmi\cache\modules\ac7906e3 -Fobuild\.objs\testm\windows\x64\release\test.cppm.obj
test.cppm
[  0%]: <testm> generating.module.deps C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\modules\std.ixx
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\bin\HostX64\x64\cl.exe -nologo -std:c++latest /EHsc -TP -scanDependencies build\.gens\testm\windows\x64\release\rules\bmi\cache\modules\539befab\std.ixx.module.json "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.40.33807\\modules\\std.ixx" -ifcOutput build\.gens\testm\windows\x64\release\rules\bmi\cache\modules\539befab -Fobuild\.objs\testm\windows\x64\release\889e61fa4b2b4a678264d65ff7ac448b\std.ixx.obj
[  0%]: <testm> generating.module.deps C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\modules\std.compat.ixx
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\bin\HostX64\x64\cl.exe -nologo -std:c++latest /EHsc -TP -scanDependencies build\.gens\testm\windows\x64\release\rules\bmi\cache\modules\539befab\std.compat.ixx.module.json "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.40.33807\\modules\\std.compat.ixx" -ifcOutput build\.gens\testm\windows\x64\release\rules\bmi\cache\modules\539befab -Fobuild\.objs\testm\windows\x64\release\889e61fa4b2b4a678264d65ff7ac448b\std.compat.ixx.obj
std.ixx
std.compat.ixx
[ 50%]: <testm> compiling.bmi.release std
checking for flags (cl_interface) ... ok
> cl.exe "-interface" "-nologo"
checking for flags (cl_ifc_only) ... ok
> cl.exe "-ifcOnly" "-nologo"
"C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.40.33807\\bin\\HostX64\\x64\\cl.exe" -c -nologo -std:c++latest /EHsc -TP -ifcOutput build\.gens\testm\windows\x64\release\rules\bmi\cache\modules\539befab\std.ifc -interface -ifcOnly -Fobuild\.gens\testm\windows\x64\release\rules\bmi\cache\modules\539befab\std.ifc "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.40.33807\\modules\\std.ixx"
checking for flags (cl_reference) ... ok
> cl.exe "-reference" "Foo=C:\Users\*****~1\AppData\Local\Temp\.xmake\240718\_F8DF50D89A734400861D092D6AE01860" "-nologo"
checking for flags (cl_header_unit_quote) ... ok
> cl.exe "-std:c++latest" "-headerUnit:quote" "foo.h=C:\Users\*****~1\AppData\Local\Temp\.xmake\240718\_ECE628A3379B43008C669A1D7BC05260" "-nologo"
checking for flags (cl_header_unit_angle) ... ok
> cl.exe "-std:c++latest" "-headerUnit:angle" "foo.h=C:\Users\*****~1\AppData\Local\Temp\.xmake\240718\_ECE628A3379B43008C669A1D7BC05260" "-nologo"
[ 66%]: <testm> compiling.bmi.release std.compat
"C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.40.33807\\bin\\HostX64\\x64\\cl.exe" -c -nologo -std:c++latest /EHsc -reference std=build\.gens\testm\windows\x64\release\rules\bmi\cache\modules\539befab\std.ifc -TP -ifcOutput build\.gens\testm\windows\x64\release\rules\bmi\cache\modules\539befab\std.compat.ifc -interface -ifcOnly -Fobuild\.gens\testm\windows\x64\release\rules\bmi\cache\modules\539befab\std.compat.ifc "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.40.33807\\modules\\std.compat.ixx"
checking for link.exe ... C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\bin\HostX64\x64\link.exe
checking for the static library archiver (ar) ... link.exe
[ 83%]: archiving.release testm.lib
"C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.40.33807\\bin\\HostX64\\x64\\link.exe" -lib -nologo -machine:x64 -out:build\windows\x64\release\testm.lib build\.objs\testm\windows\x64\release\test.cppm.obj
error: @programdir\core\main.lua:329: @programdir\actions\build\main.lua:148: @programdir\modules\async\runjobs.lua:325: @programdir\actions\build\kinds\static.lua:53: @programdir\modules\core\tools\link.lua:175: LINK : fatal error LNK1181: cannot open input file 'build\.objs\testm\windows\x64\release\test.cppm.obj'

stack traceback:
    [C]: in function 'error'
    [@programdir\core\base\os.lua:973]:
    [@programdir\modules\core\tools\link.lua:175]: in function 'catch'
    [@programdir\core\sandbox\modules\try.lua:123]: in function 'try'
    [@programdir\modules\core\tools\link.lua:150]:
    [C]: in function 'xpcall'
    [@programdir\core\base\utils.lua:275]:
    [@programdir\core\tool\linker.lua:221]: in function 'link'
    [@programdir\actions\build\kinds\static.lua:53]: in function 'callback'
    [@programdir\modules\core\project\depend.lua:217]: in function 'on_changed'
    [@programdir\actions\build\kinds\static.lua:41]: in function '_do_link_target'
    [@programdir\actions\build\kinds\static.lua:84]:
    [@programdir\actions\build\kinds\static.lua:111]: in function '_link_target'
    [@programdir\actions\build\kinds\static.lua:139]: in function 'jobfunc'
    [@programdir\modules\async\runjobs.lua:241]:
    [C]: in function 'xpcall'
    [@programdir\core\base\utils.lua:275]: in function 'trycall'
    [@programdir\core\sandbox\modules\try.lua:117]: in function 'try'
    [@programdir\modules\async\runjobs.lua:223]: in function 'cotask'
    [@programdir\core\base\scheduler.lua:406]:

stack traceback:
        [C]: in function 'error'
        @programdir\core\base\os.lua:973: in function 'base/os.raiselevel'
        (...tail calls...)
        @programdir\core\main.lua:329: in upvalue 'cotask'
        @programdir\core\base\scheduler.lua:406: in function <@programdir\core\base\scheduler.lua:399>
star-hengxing commented 1 month ago

Please use set_kind("moduleonly")

waruqi commented 1 month ago

如果没有 .cpp 只要纯 module 文件,得用 moduleonly 而不是 static

iDingDong commented 1 month ago

Please use set_kind("moduleonly")

That would not produce a static library for me. Is it impossible to build a static library that exports modules?

waruqi commented 1 month ago

see https://github.com/xmake-io/xmake/pull/4707#discussion_r1479565656

iDingDong commented 1 month ago

see #4707 (comment)

This comment doesn't make sense. Module units do get built into .lib at least for MSVC.

iDingDong commented 1 month ago

You can easily produce a huge static library by exporting large global variables that is not zero-initialized. For example, Visual studio would produce a 9KB+ static library containing only the following module unit.

export module Module;

import std;

struct A {
    int i = 123;
};

export constinit std::array<A, 1000000> ExportedArray = {};
waruqi commented 1 month ago

@Arthapz

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.


@arthapaj

Arthapz commented 1 month ago

You can easily produce a huge static library by exporting large global variables that is not zero-initialized. For example, Visual studio would produce a 9KB+ static library containing only the following module unit.

export module Module;

import std;

struct A {
  int i = 123;
};

export constinit std::array<A, 1000000> ExportedArray = {};

and it work (but i get a 4KB static library :D)

image image

Arthapz commented 1 month ago

as for your first example, i think the module get culled as it import and export nothing, we should add a proper error

but if ur static library only contains named module, u should use moduleonly targetkind to avoid flag compatibility issues with the consumer

iDingDong commented 1 month ago

You can easily produce a huge static library by exporting large global variables that is not zero-initialized. For example, Visual studio would produce a 9KB+ static library containing only the following module unit.

export module Module;

import std;

struct A {
    int i = 123;
};

export constinit std::array<A, 1000000> ExportedArray = {};

and it work (but i get a 4KB static library :D)

Size doesn't matter as long as it is reasonably larger than an empty library. It could be affected by other configurations.

Latest xmake would compile it but failed to resolve dependencies when another target attempt to add the library as its dependency and imports the module.

as for your first example, i think the module get culled as it import and export nothing, we should add a proper error

The initial example serves as a minimal repro case which does not really need to define another symbol to make itself clear.

I do appreciate a warning but an empty lib is prefered than an error.

but if ur static library only contains named module, u should use moduleonly targetkind to avoid flag compatibility issues with the consumer

That kind of problem is not exclusive to modules that any static library built out of old-fashion source/header can be troubled the same way. Since it has never been a legit reason to drop static library support for tranditional sources and switch, modules should be nothing different.

Arthapz commented 1 month ago

u need to flag ur named module as public to get them accessible from other targets and not being culled

target("testm")
  set_kind("static")
  set_languages("cxxlatest")
  add_files("./test.cppm", {public = true})
target_end()

by default we cull non-public unreferenced named module (a named module which is netheir imported inside the library or exposed to be consumed) to avoid building unused modules

    local objectfiles_sorted_set = hashset.from(objectfiles_sorted)
    for _, objectfile in ipairs(objectfiles) do
        if not objectfiles_sorted_set:has(objectfile) then
            -- cull unreferenced non-public named module but add non-module files and implementation modules
            local _, provide, cppfile = compiler_support.get_provided_module(modules[objectfile])
            local fileconfig = target:fileconfig(cppfile)
            local public = fileconfig and fileconfig.public
            if not provide or public then
                table.insert(result, objectfile)
            end
        end
    end
    return result

https://github.com/xmake-io/xmake/blob/2412ed94e70b00b0c2f3bd8c4553828efd458b1b/xmake/rules/c%2B%2B/modules/modules_support/dependency_scanner.lua#L423

Arthapz commented 1 month ago

https://github.com/xmake-io/xmake/pull/5369 i also added a policy to disable project or target wise module culling and a fileconfig entry to disable culling on a file

add_rules("mode.release", "mode.debug")
set_languages("c++20")

-- project wise
-- set_policy("build.c++.modules.culling", false)

target("culling")
    set_kind("static")
    add_files("src/*.mpp")
    -- target wise
    set_policy("build.c++.modules.culling", false)
target("culling")
    set_kind("static")
    add_files("src/*.mpp", {cull = false}) -- on files

this should provide to the user the flexibility to choose what he want, but i don't think it's a good default so please use public visibility in first attempt

and i added a proper warning for culled modules image

waruqi commented 1 month ago

Does it work now? try dev again. xmake update -s dev