corrosion-rs / corrosion

Marrying Rust and CMake - Easy Rust and C/C++ Integration!
https://corrosion-rs.github.io/corrosion/
MIT License
1.02k stars 97 forks source link

[Bug]: undefined symbols during link-time on Windows with LLVM #524

Closed ClayCore closed 2 months ago

ClayCore commented 2 months ago

Current Behavior

After adding a rust cdylib into a C++ project via corrosion_import_crate the project fails at lld-link and throws undefined symbol errors. The rust library contains two #[no_mangle] pub functions, one of which is the default add function created by the cargo new with the previously mentioned attribute.

My cmake setup is based on cpp-best-practices/cmake_template. I've added corrosion with the template's CPM.cmake and Dependencies.cmake as follows:

CPMAddPackage(
    NAME corrosion
    VERSION 0.5
    GITHUB_REPOSITORY
    "corrosion-rs/corrosion"
)

My main.cpp file:

extern void greet();
extern size_t add(size_t, size_t)

int main()
{
    greet();

    auto err = add(1, 2);

    return 0;
}

the crate import:

corrosion_import_crate(
  MANIFEST_PATH
  ${CMAKE_SOURCE_DIR}/rlib/Cargo.toml
)

rlib/src/lib.rs file:

#[no_mangle]
pub fn add(a: usize, b: usize) -> usize {
    a + b
}

#[no_mangle]
pub fn greet() -> i32 {
    println!("hello from rust");
    return 0;
}

The symbols missing according to lld-link are:

Apologies in advance for making some obvious mistake somewhere.

Expected Behavior

Succesful linking of the C++ executable with the Rust library.

Steps To Reproduce

No response

Environment

- OS: Windows 10 Enterprise LTSC (1809)
- CMake: 3.27.4
- CMake Generator: Ninja
- Rust default toolchain: stable-x86-64-pc-windows-msvc
- clang target: x86_64-pc-windows-msvc
- rustc version: 1.78.0
- clang version: 18.1.0rc
- CXX compiler: clang++

CMake configure log with Debug log-level

-- CPM: Adding package corrosion@0.5 (v0.5)
-- Using Corrosion 0.5.0 with CMake 3.27.4 and the `Ninja` Generator
-- Rust Toolchain: stable-x86_64-pc-windows-msvc
-- Rust toolchain stable-x86_64-pc-windows-msvc
-- Rust toolchain path Q:\Users\Claymore\.rustup\toolchains\stable-x86_64-pc-windows-msvc
-- Rust Target: x86_64-pc-windows-msvc
-- Parsed Target triple: arch: x86_64, vendor: pc, OS: windows, env: msvc
-- Parsed Target triple: arch: x86_64, vendor: pc, OS: windows, env: msvc
-- Cargo target x86_64-pc-windows-msvc is an official target-triple
-- Installed targets: wasm32-unknown-unknown;x86_64-pc-windows-msvc
-- Using Corrosion as a subdirectory
--  * setting build type to 'RelWithDebInfo' by default
--  * trying to find and to run 'vcvarsall.bat'...
--  * S:/msys64/opt/bin/ccache.exe found and enabled
--  * setting clang-tidy globally...
-- Found 1 targets in package rlib
-- TARGET rlib produces byproducts rlib.dll.lib;rlib.dll;rlib.pdb
-- Corrosion created the following CMake targets: rlib
-- Output directory property (target rlib): LIBRARY_OUTPUT_DIRECTORY dir: output_directory-NOTFOUND
-- Setting IMPORTED_LOCATION for target rlib-shared to `S:/wplace/rust/linked_lib/build/bin/cli/rlib.dll`.
-- Output directory property (target rlib): ARCHIVE_OUTPUT_DIRECTORY dir: output_directory-NOTFOUND
-- Setting IMPORTED_IMPLIB for target rlib-shared to `S:/wplace/rust/linked_lib/build/bin/cli/rlib.dll.lib`.
-- Adding command to copy byproducts `rlib.dll.lib` to S:/wplace/rust/linked_lib/build/bin/cli/rlib.dll.lib
-- Adding command to copy byproducts `rlib.dll` to S:/wplace/rust/linked_lib/build/bin/cli/rlib.dll
-- Adding command to copy byproducts `rlib.pdb` to S:/wplace/rust/linked_lib/build/bin/cli/rlib.pdb
-- Configuring done (4.0s)
-- Generating done (0.0s)
-- Build files have been written to: S:/wplace/rust/linked_lib/build

CMake Build step log

Change Dir: 'S:/wplace/rust/linked_lib/build'

Run Build Command(s): S:/wplace/sdk/Python/Python310/Scripts/ninja.exe -v
[0/4] cmd.exe /C "cd /D S:\wplace\rust\linked_lib\rlib && "E:\Program Files\CMake\bin\cmake.exe" -E env CXX_x86_64_pc_windows_msvc=S:/wplace/sdk/compilers/clang/bin/clang++.exe CORROSION_BUILD_DIR=S:/wplace/rust/linked_lib/build/bin/cli CARGO_BUILD_RUSTC=Q:/Users/Claymore/.rustup/toolchains/stable-x86_64-pc-windows-msvc/bin/rustc.exe Q:/Users/Claymore/.rustup/toolchains/stable-x86_64-pc-windows-msvc/bin/cargo.exe rustc --lib --target=x86_64-pc-windows-msvc --package rlib --manifest-path S:/wplace/rust/linke    Finished `release` profile [optimized] target(s) in 0.00s
d_lib/rlib/Cargo.toml --target-dir S:/wplace/rust/linked_lib/build/./cargo/build --release -- -Cdefault-linker-libraries=yes"
[2/4] cmd.exe /C "cd /D S:\wplace\rust\linked_lib\build\bin\cli && "E:\Program Files\CMake\bin\cmake.exe" -E make_directory S:/wplace/rust/linked_lib/build/bin/cli && "E:\Program Files\CMake\bin\cmake.exe" -E copy_if_different S:/wplace/rust/linked_lib/build/./cargo/build/x86_64-pc-windows-msvc/release/rlib.dll.lib S:/wplace/rust/linked_lib/build/bin/cli && cd /D S:\wplace\rust\linked_lib\build\bin\cli && "E:\Program Files\CMake\bin\cmake.exe" -E make_directory S:/wplace/rust/linked_lib/build/bin/cli && "E:\Program Files\CMake\bin\cmake.exe" -E copy_if_different S:/wplace/rust/linked_lib/build/./cargo/build/x86_64-pc-windows-msvc/release/rlib.dll S:/wplace/rust/linked_lib/build/bin/cli && cd /D S:\wplace\rust\linked_lib\build\bin\cli && "E:\Program Files\CMake\bin\cmake.exe" -E make_directory S:/wplace/rust/linked_lib/build/bin/cli && "E:\Program Files\CMake\bin\cmake.exe" -E copy_if_different S:/wplace/rust/linked_lib/build/./cargo/build/x86_64-pc-windows-msvc/release/rlib.pdb S:/wplace/rust/linked_lib/build/bin/cli"
[3/4] "E:\Program Files\CMake\bin\cmake.exe" -E __run_co_compile --launcher=S:/msys64/opt/bin/ccache.exe --tidy="E:/Program Files/LLVM/bin/clang-tidy.exe;-extra-arg=-Wno-unknown-warning-option;-extra-arg=-Wno-ignored-optimization-argument;-extra-arg=-Wno-unused-command-line-argument;-p;--extra-arg-before=--driver-mode=g++" --source=S:\wplace\rust\linked_lib\bin\cli\main.cpp -- S:\wplace\sdk\compilers\clang\bin\clang++.exe  -IS:/wplace/rust/linked_lib/cli -IS:/wplace/rust/linked_lib/build/cli -O2 -DNDEBUG -g -Xclang -gcodeview -std=c++20 -D_DLL -D_MT -Xclang --dependent-lib=msvcrt -flto=thin -fvisibility-inlines-hidden -fcolor-diagnostics -Wall -Wextra -Wshadow -Wnon-virtual-dtor -Wold-style-cast -Wcast-align -Wunused -Woverloaded-virtual -Wpedantic -Wconversion -Wsign-conversion -Wnull-dereference -Wdouble-promotion -Wformat=2 -Wimplicit-fallthrough -MD -MT bin/cli/CMakeFiles/cli.dir/main.cpp.obj -MF bin\cli\CMakeFiles\cli.dir\main.cpp.obj.d -o bin/cli/CMakeFiles/cli.dir/main.cpp.obj -c S:/wplace/rust/linked_lib/bin/cli/main.cpp
S:\wplace\rust\linked_lib\bin\cli\main.cpp:2:8: warning: no header providing "size_t" is directly included [misc-include-cleaner]
    1 | extern void greet();
    2 | extern size_t add(size_t, size_t);
      |        ^
S:\wplace\rust\linked_lib\bin\cli\main.cpp:8:10: warning: Value stored to 'err' during its initialization is never read [clang-analyzer-deadcode.DeadStores]
    8 |     auto err = add(1, 2);
      |          ^~~   ~~~~~~~~~
S:\wplace\rust\linked_lib\bin\cli\main.cpp:8:10: note: Value stored to 'err' during its initialization is never read
    8 |     auto err = add(1, 2);
      |          ^~~   ~~~~~~~~~
S:\wplace\rust\linked_lib\bin\cli\main.cpp:8:10: warning: unused variable 'err' [clang-diagnostic-unused-variable]
    8 |     auto err = add(1, 2);
      |          ^~~
S:/wplace/rust/linked_lib/bin/cli/main.cpp:8:10: warning: unused variable 'err' [-Wunused-variable]
    8 |     auto err = add(1, 2);
      |          ^~~
1 warning generated.
[4/4] cmd.exe /C "cd . && S:\wplace\sdk\compilers\clang\bin\clang++.exe -fuse-ld=lld-link -nostartfiles -nostdlib -O2 -DNDEBUG -g -Xclang -gcodeview -D_DLL -D_MT -Xclang --dependent-lib=msvcrt -flto=thin -Xlinker /subsystem:console bin/cli/CMakeFiles/cli.dir/main.cpp.obj -o bin\cli\cli.exe -Xlinker /MANIFEST:EMBED -Xlinker /implib:bin\cli\cli.lib -Xlinker /pdb:bin\cli\cli.pdb -Xlinker /version:0.1   bin/cli/rlib.dll.lib  -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 -loldnames  && cmd.exe /C "cd /D S:\wplace\rust\linked_lib\build\bin\cli && "E:\Program Files\CMake\bin\cmake.exe" -E copy S:/wplace/rust/linked_lib/build/bin/cli/cli.exe S:/wplace/rust/linked_lib/target""
FAILED: bin/cli/cli.exe
cmd.exe /C "cd . && S:\wplace\sdk\compilers\clang\bin\clang++.exe -fuse-ld=lld-link -nostartfiles -nostdlib -O2 -DNDEBUG -g -Xclang -gcodeview -D_DLL -D_MT -Xclang --dependent-lib=msvcrt -flto=thin -Xlinker /subsystem:console bin/cli/CMakeFiles/cli.dir/main.cpp.obj -o bin\cli\cli.exe -Xlinker /MANIFEST:EMBED -Xlinker /implib:bin\cli\cli.lib -Xlinker /pdb:bin\cli\cli.pdb -Xlinker /version:0.1   bin/cli/rlib.dll.lib  -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 -loldnames  && cmd.exe /C "cd /D S:\wplace\rust\linked_lib\build\bin\cli && "E:\Program Files\CMake\bin\cmake.exe" -E copy S:/wplace/rust/linked_lib/build/bin/cli/cli.exe S:/wplace/rust/linked_lib/target""
lld-link: error: undefined symbol: void __cdecl greet(void)
>>> referenced by S:/wplace/rust/linked_lib/bin/cli/main.cpp
>>>               bin/cli/CMakeFiles/cli.dir/main.cpp.obj

lld-link: error: undefined symbol: unsigned __int64 __cdecl add(unsigned __int64, unsigned __int64)
>>> referenced by S:/wplace/rust/linked_lib/bin/cli/main.cpp
>>>               bin/cli/CMakeFiles/cli.dir/main.cpp.obj
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.
jschwe commented 2 months ago

How does your CMakeLists.txt look like? Did you do anything like the following:

target_link_libraries(my_cpp_bin PUBLIC my_rust_lib)
jschwe commented 2 months ago

Also please note that no mangle is not sufficient. You need to specify that your Rust function should use the C-ABI, otherwise behavior when calling from C or C++ is undefined.

ClayCore commented 2 months ago

Also please note that no mangle is not sufficient. You need to specify that your Rust function should use the C-ABI, otherwise behavior when calling from C or C++ is undefined.

I've added extern "C" to the function definitions, but I'm getting the same errors. Wasn't aware I needed this though

How does your CMakeLists.txt look like? Did you do anything like the following:

target_link_libraries(my_cpp_bin PUBLIC my_rust_lib)

The CMakeLists.txt relies on the template, but the one used to build the executable is this:

add_executable(
  cli
  main.cpp
)
add_executable(dev::cli ALIAS cli)

corrosion_import_crate(
  MANIFEST_PATH
  ${CMAKE_SOURCE_DIR}/rlib/Cargo.toml
)

if(NOT TARGET rlib)
  message(FATAL_ERROR " !! rust library not built/imported")
endif()

target_link_libraries(cli PRIVATE dev_options dev_warnings)
target_link_libraries(cli PUBLIC rlib)
target_include_directories(
  cli
  ${WARNING_GUARD}
  PUBLIC
  $<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}/cli>
  $<BUILD_INTERFACE:${PROJECT_BINARY_DIR}/cli>
)
target_compile_features(cli PUBLIC cxx_std_20)

set_target_properties(
  cli
  PROPERTIES
  VERSION ${PROJECT_VERSION}
  CXX_VISIBILITY_PRESET hidden
  VISIBILITY_INLINES_HIDDEN YES
)

if(BUILD_TESTING)
  message(TRACE "Building tests...")
  add_subdirectory(test)
endif()

add_custom_command(
  TARGET
  cli
  POST_BUILD
  COMMAND
  ${CMAKE_COMMAND} -E copy $<TARGET_FILE:cli> ${PROJECT_SOURCE_DIR}/target
)
jschwe commented 2 months ago

I've added extern "C" to the function definitions, but I'm getting the same errors. Wasn't aware I needed this though

It's not related to the linking issue. By default Rust functions use the Rust-ABI, which is neither stable nor guaranteed in any way to be compatible with C/C++ ABIs. If your function is Rust-ABI, but you call it from C/C++ the behavior is undefined. For such a simple function as in your example the ABIs will probably be the same, but you can't know that.

I believe the same also applies for C++ code - you should declare the Rust function as having C-ABI. I'm not really thatfamiliar with C++, so perhaps a simple extern is equivalent with extern "C", but I would still recommend writing it out to be explicit.

extern "C" void greet();
extern "C" size_t add(size_t, size_t);

That being said, I'm not sure whats going wrong for you - We do test windows -msvc with clang-cl and the Ninja Generator in CI. Could you try removing the custom command that moves the cli target to another location post-build?

add_custom_command(
  TARGET
  cli
  POST_BUILD
  COMMAND
  ${CMAKE_COMMAND} -E copy $<TARGET_FILE:cli> ${PROJECT_SOURCE_DIR}/target
)
ClayCore commented 2 months ago

I've added extern "C" to the function definitions, but I'm getting the same errors. Wasn't aware I needed this though

It's not related to the linking issue. By default Rust functions use the Rust-ABI, which is neither stable nor guaranteed in any way to be compatible with C/C++ ABIs. If your function is Rust-ABI, but you call it from C/C++ the behavior is undefined. For such a simple function as in your example the ABIs will probably be the same, but you can't know that.

I believe the same also applies for C++ code - you should declare the Rust function as having C-ABI. I'm not really thatfamiliar with C++, so perhaps a simple extern is equivalent with extern "C", but I would still recommend writing it out to be explicit.

extern "C" void greet();
extern "C" size_t add(size_t, size_t);

Adding extern "C" to function definitions in C++ source fixed it. I wasn't aware that it needs to be added. Oops.

That being said, I'm not sure whats going wrong for you - We do test windows -msvc with clang-cl and the Ninja Generator in CI. Could you try removing the custom command that moves the cli target to another location post-build?

add_custom_command(
  TARGET
  cli
  POST_BUILD
  COMMAND
  ${CMAKE_COMMAND} -E copy $<TARGET_FILE:cli> ${PROJECT_SOURCE_DIR}/target
)

To be precise, I'm using clang LLVM binaries, not clang-cl. Removing the post-build command didn't help on its own. I had to do the above of adding extern "C" to C++ source.

It does appear I'll have to add a custom command to copy the compiled rust shared library to the same directory that I'm copying the compiled executable to, corrosion doesn't seem to do it on its own

jschwe commented 2 months ago

You should have a look at the Cmake output directory variables, which corrosion will respect and copy the rust artifacts to.

ClayCore commented 2 months ago

You should have a look at the Cmake output directory variables, which corrosion will respect and copy the rust artifacts to.

Yeah, I should probably stop using custom commands everywhere. Thank you! I believe this solves my issue

jschwe commented 2 months ago

Great to hear, closing the issue.