rust-lang / rust

Empowering everyone to build reliable and efficient software.
https://www.rust-lang.org
Other
98.79k stars 12.77k forks source link

Linking native code with the crate-type "staticlib" causes link failures with internal Windows APIs on Windows #119109

Open TrebuchKill opened 11 months ago

TrebuchKill commented 11 months ago

Code

I tried this project:

Cargo.toml:

[package]
name = "example"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = [ "lib", "staticlib", "cdylib" ]

rust-toolchain.toml:

[toolchain]
# channel = "1.56.0" # Does work
# channel = "1.60.0" # Does work
# channel = "1.69.0" # Does work
# channel = "1.70.0" # Does not work
# channel = "1.74.1" # Does not work
channel = "stable" # Does not work
# channel = "beta" # Does not work
# channel = "nightly" # Does not work

src/lib.rs:

#[no_mangle]
pub extern "C" fn hello_world()
{
    println!("Hello World");
}

src/main.cpp:

#if USE_STATIC_API
#define EXAMPLE_API __cdecl
#else
#define EXAMPLE_API __declspec(dllimport) __cdecl
#endif

extern "C"
{
    void EXAMPLE_API hello_world();
}

int wmain()
{
    hello_world();
    return 0;
}

build.ps1:

cargo clean

cargo b --release
cl /nologo /utf-8 /EHsc /std:c++latest /Zc:preprocessor /permissive- /W4 /Zc:__cplusplus /MD /Fotarget\release\build\ /Fetarget\release\main_static.exe /DUSE_STATIC_API=1 .\src\main.cpp .\target\release\example.lib OneCore.lib
cl /nologo /utf-8 /EHsc /std:c++latest /Zc:preprocessor /permissive- /W4 /Zc:__cplusplus /MD /Fotarget\release\build\ /Fetarget\release\main_dynamic.exe .\src\main.cpp .\target\release\example.dll.lib

build.ps1 is not required, but it was helpful for testing and documents how I'm building the code.

I expected to see this happen:

A C++ application linking against example.lib links & runs successfully.

> .\build.ps1
   Compiling example v0.1.0 (C:\Users\bartw\source\rust\Testing)
    Finished release [optimized] target(s) in 0.31s
main.cpp # using staticlib, all fine
main.cpp # using cdylib, all fine
> .\target\release\main_static.exe
Hello World

Instead, this happened:

Linking errors to internal Windows APIs:

> .\build.ps1
     Removed 23 files, 22.3MiB total
   Compiling example v0.1.0 (C:\Users\bartw\source\rust\Testing)
    Finished release [optimized] target(s) in 0.35s
main.cpp # using staticlib, linker errors
example.lib(std-0953e6946d141dd5.std.1acf644841c643e8-cgu.0.rcgu.o) : error LNK2019: unresolved external symbol __imp_NtCreateFile referenced in function _ZN3std3sys7windows2fs20open_link_no_reparse17hca07eefe7f3c23d8E
example.lib(std-0953e6946d141dd5.std.1acf644841c643e8-cgu.0.rcgu.o) : error LNK2019: unresolved external symbol __imp_RtlNtStatusToDosError referenced in function _ZN3std3sys7windows2fs20open_link_no_reparse17hca07eefe7f3c23d8E
example.lib(std-0953e6946d141dd5.std.1acf644841c643e8-cgu.0.rcgu.o) : error LNK2019: unresolved external symbol __imp_NtReadFile referenced in function _ZN3std3sys7windows6handle6Handle16synchronous_read17h5729adfb8e80a411E
example.lib(std-0953e6946d141dd5.std.1acf644841c643e8-cgu.0.rcgu.o) : error LNK2019: unresolved external symbol __imp_NtWriteFile referenced in function _ZN3std3sys7windows6handle6Handle17synchronous_write17h87d0e51b32ace2e3E
target\release\main_static.exe : fatal error LNK1120: 4 unresolved externals
main.cpp # using cdylib, all fine

Version it worked on

It most recently worked on:

rustc 1.69.0 (84c898d65 2023-04-16)
binary: rustc
commit-hash: 84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc
commit-date: 2023-04-16
host: x86_64-pc-windows-msvc
release: 1.69.0
LLVM version: 15.0.7

Version with regression

First noticed with:

rustc 1.74.1 (a28077b28 2023-12-04)
binary: rustc
commit-hash: a28077b28a02b92985b3a3faecf92813155f1ea1
commit-date: 2023-12-04
host: x86_64-pc-windows-msvc
release: 1.74.1
LLVM version: 17.0.4

Lowest version tested which did not work:

rustc 1.70.0 (90c541806 2023-05-31)
binary: rustc
commit-hash: 90c541806f23a127002de5b4038be731ba1458ca
commit-date: 2023-05-31
host: x86_64-pc-windows-msvc
release: 1.70.0
LLVM version: 16.0.2

Versions not listed in the rust-toolchain.toml example above where not tested.

ChrisDenton commented 11 months ago

See rustc src/lib.rs --print native-static-libs --crate-type staticlib. It should print something like:

note: native-static-libs: advapi32.lib bcrypt.lib kernel32.lib msvcrt.lib ntdll.lib userenv.lib ws2_32.lib

You will need to add these import libraries to your C project. Some will be included by default, others won't. Though the linker errors will only happen if your program actually tries to make use of the missing symbols so not all of these libs will be strictly necessary.

TrebuchKill commented 11 months ago

Adding ntdll.lib made the example link. Thank you @ChrisDenton.

Before posting this issue, one of the docs I checked was the NtCreateFile docs, which specifies no LIB file. At the beginning of this page is a note linking to Calling Internal APIs, which mentions, that no import library is provided. This is why I didn't look elsewhere if such import library exists.

ChrisDenton commented 11 months ago

Yeah, the docs are fairly outdated. ntdll.lib has been distributed with the Windows SDK for over a decade at this point.

Rust itself tends not to use Nt* functions but there are a few places where it's used to avoid UB or security issues. For example, someone from Azure noted that stdout (or any File for that matter) may or may not be an overlapped handle and may or may not be accessed concurrently from multiple threads. Therefore they suggested we use NtWriteFile to make sure we can address any possible scenario. Another example is fixing CVE-2022-21658 where remove_dir_all following a path may be tricked into following a symlink and deleting arbitrary files. NtCreateFile was used to address this because it allows specifying a base directory handle, thus avoiding the need to traverse the full path each time it deletes something.

riverar commented 10 months ago

Before posting this issue, one of the docs I checked was the NtCreateFile docs, which specifies no LIB file.

The docs you linked to actually say:

The associated import library, NtDll.lib is available in the WDK [...]

But it's hard to see, I'll submit an edit to add it to the information block at the bottom like all the other APIs.

ChrisDenton commented 10 months ago

@riverar Note that ntdll.lib is also available in the SDK too (as of Windows 8/8.1) which kind of undermines the current rationale given in Calling Internal APIs, I guess it could be written to highlight the positive aspects of using the Win32 API vs. NT API (take advantage of new features without changing your code!) but that's a bigger job.

riverar commented 10 months ago

@ChrisDenton Agreed, removing that sentence about the WDK too.

Tracking: https://github.com/MicrosoftDocs/sdk-api/pull/1738