Linking fails with zstd dependency on GitHub Actions targetting Windows

[niklasf]

Reduced example: https://github.com/niklasf/cargo-auditable-issue-141 (where cargo build --release works but cargo auditable build --release fails)

Workflow: https://github.com/niklasf/cargo-auditable-issue-141/blob/main/.github/workflows/demo.yml

Error: https://github.com/niklasf/cargo-auditable-issue-141/actions/runs/8866193441/job/24343338399

Log snippet:

2024-04-28T08:15:04.6004775Z   = note: LINK : warning LNK4044: unrecognized option '/Wl,--undefined=AUDITABLE_VERSION_INFO'; ignored
2024-04-28T08:15:04.6005683Z           libzstd_sys-2baf2e10c8662d45.rlib(5eeec2f4c1e9b3b6-zstd_v03.o) : error LNK2001: unresolved external symbol ___chkstk_ms
2024-04-28T08:15:04.6006640Z           libzstd_sys-2baf2e10c8662d45.rlib(1ddd1489cc625d97-hist.o) : error LNK2001: unresolved external symbol ___chkstk_ms
2024-04-28T08:15:04.6007577Z           libzstd_sys-2baf2e10c8662d45.rlib(5eeec2f4c1e9b3b6-zstd_v06.o) : error LNK2001: unresolved external symbol ___chkstk_ms
2024-04-28T08:15:04.6008518Z           libzstd_sys-2baf2e10c8662d45.rlib(5eeec2f4c1e9b3b6-zstd_v04.o) : error LNK2001: unresolved external symbol ___chkstk_ms
2024-04-28T08:15:04.6009431Z           libzstd_sys-2baf2e10c8662d45.rlib(5eeec2f4c1e9b3b6-zstd_v01.o) : error LNK2001: unresolved external symbol ___chkstk_ms
2024-04-28T08:15:04.6010344Z           libzstd_sys-2baf2e10c8662d45.rlib(5eeec2f4c1e9b3b6-zstd_v02.o) : error LNK2001: unresolved external symbol ___chkstk_ms
2024-04-28T08:15:04.6011312Z           libzstd_sys-2baf2e10c8662d45.rlib(1ddd1489cc625d97-zstd_compress.o) : error LNK2001: unresolved external symbol ___chkstk_ms
2024-04-28T08:15:04.6012284Z           libzstd_sys-2baf2e10c8662d45.rlib(b0401a448be314bb-zdict.o) : error LNK2001: unresolved external symbol ___chkstk_ms
2024-04-28T08:15:04.6013221Z           libzstd_sys-2baf2e10c8662d45.rlib(5eeec2f4c1e9b3b6-zstd_v05.o) : error LNK2001: unresolved external symbol ___chkstk_ms
2024-04-28T08:15:04.6014149Z           libzstd_sys-2baf2e10c8662d45.rlib(5eeec2f4c1e9b3b6-zstd_v07.o) : error LNK2001: unresolved external symbol ___chkstk_ms
2024-04-28T08:15:04.6015225Z           libzstd_sys-2baf2e10c8662d45.rlib(b0401a448be314bb-zdict.o) : error LNK2019: unresolved external symbol __mingw_vfprintf referenced in function fprintf
2024-04-28T08:15:04.6016319Z           libzstd_sys-2baf2e10c8662d45.rlib(b0401a448be314bb-fastcover.o) : error LNK2001: unresolved external symbol __mingw_vfprintf
2024-04-28T08:15:04.6017298Z           libzstd_sys-2baf2e10c8662d45.rlib(b0401a448be314bb-cover.o) : error LNK2001: unresolved external symbol __mingw_vfprintf
2024-04-28T08:15:04.6018495Z           D:\a\cargo-auditable-issue-x\cargo-auditable-issue-x\target\release\deps\cargo_auditable_issue_x-477cc37b35224d7f.exe : fatal error LNK1120: 2 unresolved externals
2024-04-28T08:15:04.6019329Z           
2024-04-28T08:15:04.6019444Z 
2024-04-28T08:15:05.1457161Z error: could not compile `cargo-auditable-issue-x` (bin "cargo-auditable-issue-x") due to 1 previous error
2024-04-28T08:15:05.1457962Z 
2024-04-28T08:15:05.1458107Z Caused by:
2024-04-28T08:15:05.1472415Z   process didn't exit successfully: `set CARGO='\\?\D:\a\_temp\msys64\mingw64\bin\cargo.exe'&& set CARGO_BIN_NAME=cargo-auditable-issue-x&& set CARGO_CRATE_NAME=cargo_auditable_issue_x&& set CARGO_MANIFEST_DIR='D:\a\cargo-auditable-issue-x\cargo-auditable-issue-x'&& set CARGO_PKG_AUTHORS=''&& set CARGO_PKG_DESCRIPTION=''&& set CARGO_PKG_HOMEPAGE=''&& set CARGO_PKG_LICENSE=''&& set CARGO_PKG_LICENSE_FILE=''&& set CARGO_PKG_NAME=cargo-auditable-issue-x&& set CARGO_PKG_README=''&& set CARGO_PKG_REPOSITORY=''&& set CARGO_PKG_RUST_VERSION=''&& set CARGO_PKG_VERSION=0.0.0&& set CARGO_PKG_VERSION_MAJOR=0&& set CARGO_PKG_VERSION_MINOR=0&& set CARGO_PKG_VERSION_PATCH=0&& set CARGO_PKG_VERSION_PRE=''&& set CARGO_PRIMARY_PACKAGE=1&& set PATH='D:\a\cargo-auditable-issue-x\cargo-auditable-issue-x\target\release\deps;C:\Users\runneradmin\.rustup\toolchains\stable-x86_64-pc-windows-msvc\bin;D:\a\_temp\msys64\mingw64\bin;D:\a\_temp\msys64\usr\local\bin;D:\a\_temp\msys64\usr\bin;D:\a\_temp\msys64\usr\bin;C:\Windows\System32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;D:\a\_temp\msys64\usr\bin\site_perl;D:\a\_temp\msys64\usr\bin\vendor_perl;D:\a\_temp\msys64\usr\bin\core_perl'&& C:\Users\runneradmin\.cargo\bin\cargo-auditable.exe rustc --crate-name cargo_auditable_issue_x --edition=2021 'src\main.rs' --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C opt-level=3 -C embed-bitcode=no -C metadata=477cc37b35224d7f -C extra-filename=-477cc37b35224d7f --out-dir 'D:\a\cargo-auditable-issue-x\cargo-auditable-issue-x\target\release\deps' -L 'dependency=D:\a\cargo-auditable-issue-x\cargo-auditable-issue-x\target\release\deps' --extern 'zstd=D:\a\cargo-auditable-issue-x\cargo-auditable-issue-x\target\release\deps\libzstd-c8cf5de31f378c0a.rlib' -L 'native=D:\a\cargo-auditable-issue-x\cargo-auditable-issue-x\target\release\build\zstd-sys-33d347c9f0b79841\out'` (exit code: 1)
2024-04-28T08:15:05.1585513Z ##[error]Process completed with exit code 1

[Shnatsel]

Thank you for the report!

unrecognized option '/Wl,--undefined=AUDITABLE_VERSION_INFO'; ignored

This is caused by cargo auditable prefixing the --undefined option with -Wl because typically rustc doesn't use the linker directly, it uses a compiler such as GCC as a wrapper around the linker, and -Wl is the GCC and clang option to pass an argument to the linker.

This could be an issue with cargo auditable that was undiscovered until now because it is only a warning. I'll investigate it. However, it is only a warning, so that's not what is causing compilation to fail.

unresolved external symbol __mingw_vfprintf

This indicates that libzstd has been build with mingw (GCC for Windows) toolchain; linking it with the MSVC toolchain then fails, as expected.

So it seems that something confuses zstd-sys and makes it build with MinGW instead of MSVC toolchain when run under cargo auditable.

I don't think cargo auditable emits the wrong kind of object file, because then we would be seeing linking errors from things other than libzstd - it would either complain about cargo_auditable_issue_x_audit_data.o or about the toplevel binary being compiled.

This is the extent of the changes that cargo auditable makes to the compilation command.

Perhaps zstd-sys sees the options starting with - or -- and decides that it should be using MinGW instead of MSVC?

[Shnatsel]

After I figure out the -Wl issue, the compilation failure may or may not go away. I suggest reporting this to zstd-sys maintainers regardless. If the issue is indeed with it incorrectly picking MinGW instead of MSVC, then zstd-sys should be fixed regardless because their toolchain detection is fragile and will break not just under cargo auditable.

[Shnatsel]

I've created a branch that might fix the issue: #142

Could you test it and see if the problem still occurs?

[niklasf]

New run: https://github.com/niklasf/cargo-auditable-issue-141/actions/runs/8866914352/job/24344898298

unrecognized option '/Wl,--undefined=AUDITABLE_VERSION_INFO'; ignored

This is now fixed.

However, it is only a warning, so that's not what is causing compilation to fail.

And indeed it still fails with libzstd_sys-2baf2e10c8662d45.rlib(b0401a448be314bb-fastcover.o) : error LNK2001: unresolved external symbol __mingw_vfprintf etc.

[Shnatsel]

I've double-checked the way cargo auditable write object files, and there is no distinction between MinGW and MSVC targets. And this logic were incorrect (already a stretch because it's copied from rustc), all builds would be broken, not just zstd.

I'm afraid that's an issue with zstd-sys getting confused about what toolchain should be used, and mistakenly picking MinGW. That's not something I can fix in cargo auditable; you will have to report it to zstd-sys maintainers.

[niklasf]

Thanks for looking into it!