Standard library headers not found by bindgen when cross-compiling for Windows

bluebear94 commented 1 year ago

Checklist

[X] I've looked through the issues and pull requests for similar reports

Describe your issue

When cross-compiling for Windows, bindgen fails to find standard C++ library headers.

What target(s) are you cross-compiling for?

x86_64-pc-windows-gnu

Which operating system is the host (e.g computer cross is on) running?

[ ] macOS
[ ] Windows
[X] Linux / BSD
[ ] other OS (specify in description)

What architecture is the host?

[X] x86_64 / AMD64
[ ] arm32
[ ] arm64 (including Mac M1)

What container engine is cross using?

[X] docker
[ ] podman
[ ] other container engine (specify in description)

cross version

cross 0.2.4, on cargo 1.67.0-nightly (a3dfea71c 2022-11-11)

Example

git clone --recursive https://github.com/Penple/msdf-rs
cd msdf-rs

cat << EOF > Dockerfile
FROM ghcr.io/cross-rs/x86_64-pc-windows-gnu:main
RUN apt-get update && apt-get install --assume-yes --no-install-recommends libclang-dev clang
EOF
docker build -t test .

cat << EOF > Cross.toml
[target.x86_64-pc-windows-gnu]
image = "test"
EOF

cross build --target x86_64-pc-windows-gnu

This produces the following error:

error: failed to run custom build command for `msdf-sys v0.1.0 (/project/msdf-sys)`

Caused by:
  process didn't exit successfully: `/target/debug/build/msdf-sys-223228ec41ec6e67/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-changed=wrapper.h
  cargo:rustc-link-lib=static=msdfgen-core
  CMAKE_TOOLCHAIN_FILE_x86_64-pc-windows-gnu = None
  CMAKE_TOOLCHAIN_FILE_x86_64_pc_windows_gnu = None
  TARGET_CMAKE_TOOLCHAIN_FILE = None
  CMAKE_TOOLCHAIN_FILE = None
  CMAKE_GENERATOR_x86_64-pc-windows-gnu = None
  CMAKE_GENERATOR_x86_64_pc_windows_gnu = None
  TARGET_CMAKE_GENERATOR = None
  CMAKE_GENERATOR = None
  CMAKE_PREFIX_PATH_x86_64-pc-windows-gnu = None
  CMAKE_PREFIX_PATH_x86_64_pc_windows_gnu = None
  TARGET_CMAKE_PREFIX_PATH = None
  CMAKE_PREFIX_PATH = None
  CMAKE_x86_64-pc-windows-gnu = None
  CMAKE_x86_64_pc_windows_gnu = None
  TARGET_CMAKE = None
  CMAKE = None
  running: "cmake" "/target/x86_64-pc-windows-gnu/debug/build/msdf-sys-e7a130b7a8548441/out/msdfgen" "-DCMAKE_SYSTEM_NAME=Windows" "-DMSDFGEN_BUILD_STANDALONE=OFF" "-DMSDFGEN_CORE_ONLY=ON" "-DCMAKE_INSTALL_PREFIX=/target/x86_64-pc-windows-gnu/debug/build/msdf-sys-e7a130b7a8548441/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -m64" "-DCMAKE_C_COMPILER=/usr/bin/x86_64-w64-mingw32-gcc-posix" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -m64" "-DCMAKE_CXX_COMPILER=/usr/bin/x86_64-w64-mingw32-g++-posix" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -m64" "-DCMAKE_ASM_COMPILER=/usr/bin/x86_64-w64-mingw32-gcc-posix" "-DCMAKE_BUILD_TYPE=Release"
  -- The CXX compiler identification is GNU 9.3.0
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /usr/bin/x86_64-w64-mingw32-g++-posix - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Could NOT find Freetype (missing: FREETYPE_LIBRARY FREETYPE_INCLUDE_DIRS) 
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /target/x86_64-pc-windows-gnu/debug/build/msdf-sys-e7a130b7a8548441/out/build
  running: "cmake" "--build" "." "--target" "msdfgen-core" "--config" "Release" "--parallel" "16"
  [ 13%] Building CXX object CMakeFiles/msdfgen-core.dir/core/MSDFErrorCorrection.cpp.obj
  [ 13%] Building CXX object CMakeFiles/msdfgen-core.dir/core/EdgeHolder.cpp.obj
  [ 18%] Building CXX object CMakeFiles/msdfgen-core.dir/core/Contour.cpp.obj
  [ 18%] Building CXX object CMakeFiles/msdfgen-core.dir/core/Projection.cpp.obj
  [ 27%] Building CXX object CMakeFiles/msdfgen-core.dir/core/Scanline.cpp.obj
  [ 27%] Building CXX object CMakeFiles/msdfgen-core.dir/core/Shape.cpp.obj
  [ 31%] Building CXX object CMakeFiles/msdfgen-core.dir/core/Vector2.cpp.obj
  [ 40%] Building CXX object CMakeFiles/msdfgen-core.dir/core/edge-coloring.cpp.obj
  [ 40%] Building CXX object CMakeFiles/msdfgen-core.dir/core/SignedDistance.cpp.obj
  [ 50%] Building CXX object CMakeFiles/msdfgen-core.dir/core/contour-combiners.cpp.obj
  [ 50%] Building CXX object CMakeFiles/msdfgen-core.dir/core/edge-segments.cpp.obj
  [ 54%] Building CXX object CMakeFiles/msdfgen-core.dir/core/edge-selectors.cpp.obj
  [ 59%] Building CXX object CMakeFiles/msdfgen-core.dir/core/equation-solver.cpp.obj
  [ 68%] Building CXX object CMakeFiles/msdfgen-core.dir/core/msdf-error-correction.cpp.obj
  [ 68%] Building CXX object CMakeFiles/msdfgen-core.dir/core/msdfgen.cpp.obj
  [ 72%] Building CXX object CMakeFiles/msdfgen-core.dir/core/rasterization.cpp.obj
  [ 77%] Building CXX object CMakeFiles/msdfgen-core.dir/core/render-sdf.cpp.obj
  [ 81%] Building CXX object CMakeFiles/msdfgen-core.dir/core/save-bmp.cpp.obj
  [ 86%] Building CXX object CMakeFiles/msdfgen-core.dir/core/save-tiff.cpp.obj
  [ 90%] Building CXX object CMakeFiles/msdfgen-core.dir/core/sdf-error-estimation.cpp.obj
  [ 95%] Building CXX object CMakeFiles/msdfgen-core.dir/core/shape-description.cpp.obj
  [100%] Linking CXX static library libmsdfgen-core.a
  [100%] Built target msdfgen-core
  cargo:root=/target/x86_64-pc-windows-gnu/debug/build/msdf-sys-e7a130b7a8548441/out
  cargo:rustc-link-search=native=/target/x86_64-pc-windows-gnu/debug/build/msdf-sys-e7a130b7a8548441/out/build
  cargo:rustc-link-lib=dylib=stdc++

  --- stderr
  CMake Warning:
    Manually-specified variables were not used by the project:

      CMAKE_ASM_COMPILER
      CMAKE_ASM_FLAGS
      CMAKE_C_COMPILER
      CMAKE_C_FLAGS

  make: warning: -j16 forced in submake: resetting jobserver mode.
  msdfgen/core/arithmetics.hpp:4:10: fatal error: 'cstdlib' file not found
  thread 'main' panicked at 'Unable to generate bindings: ClangDiagnostic("msdfgen/core/arithmetics.hpp:4:10: fatal error: 'cstdlib' file not found\n")', msdf-sys/build.rs:67:10
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...

Additional information / notes

A workaround is to pass -I/usr/lib/gcc/x86_64-w64-mingw32/9.3-win32/include/c++ -I/usr/lib/gcc/x86_64-w64-mingw32/9.3-posix/include/c++/x86_64-w64-mingw32 as arguments to the underlying Clang process:

diff --git a/caxton-impl/msdf-rs/msdf-sys/build.rs b/caxton-impl/msdf-rs/msdf-sys/build.rs
index 0fb7624..0b88fa0 100644
--- a/msdf-sys/build.rs
+++ b/msdf-sys/build.rs
@@ -56,6 +56,8 @@ fn main() {

     let bindings = bindgen::Builder::default()
         .clang_arg("-Imsdfgen")
+        .clang_arg("-I/usr/lib/gcc/x86_64-w64-mingw32/9.3-win32/include/c++")
+        .clang_arg("-I/usr/lib/gcc/x86_64-w64-mingw32/9.3-posix/include/c++/x86_64-w64-mingw32")
         .clang_arg("-x")
         .clang_arg("c++")
         .opaque_type("std::.*")

Building this crate for x86_64-unknown-linux-gnu succeeds.

Emilgardis commented 1 year ago

just a tip for you.

[target.x86_64-pc-windows-gnu]
dockerfile = "Dockerfile"

would automatically build the dockerfile for you. But also it could be replaced with

[target.x86_64-pc-windows-gnu]
image = "ghcr.io/cross-rs/x86_64-pc-windows-gnu:main"
pre-build = ["apt-get update && apt-get install --assume-yes --no-install-recommends libclang-dev clang"]

Alexhuszagh commented 1 year ago

This is partially a duplicate of #1112, and also linked to #1110. However, there's still issues afterwards. The relevant part of the log is below, and as we can see, the toolchain and CMake configuration is done, however, the build script is now unable to find cstdlib, which suggests the include directory path for C++ is off. It looks like /usr/lib/gcc/x86_64-w64-mingw32/9.3-posix/include/c++ is now not part of our search path, which normally should be part of our include directories.

error: failed to run custom build command for `msdf-sys v0.1.0 (/home/ahuszagh/git/msdf-rs/msdf-sys)`                                                                                               

Caused by:                                                                                                                                                                                          
  process didn't exit successfully: `/target/debug/build/msdf-sys-813f8f5b6bc6e1e5/build-script-build` (exit status: 101)                                                                           
  --- stdout                                                                                                                                                                                        
  cargo:rerun-if-changed=wrapper.h                                                                                                                                                                  
  cargo:rustc-link-lib=static=msdfgen-core                                                                                                                                                          
  CMAKE_TOOLCHAIN_FILE_x86_64-pc-windows-gnu = None                                                                                                                                                 
  CMAKE_TOOLCHAIN_FILE_x86_64_pc_windows_gnu = Some("/opt/toolchain.cmake")                                                                                                                         
  CMAKE_GENERATOR_x86_64-pc-windows-gnu = None                                                                                                                                                      
  CMAKE_GENERATOR_x86_64_pc_windows_gnu = None                                                                                                                                                      
  TARGET_CMAKE_GENERATOR = None                                                                                                                                                                     
  CMAKE_GENERATOR = None                                                                                                                                                                            
  CMAKE_PREFIX_PATH_x86_64-pc-windows-gnu = None                                                                                                                                                    
  CMAKE_PREFIX_PATH_x86_64_pc_windows_gnu = None                                                                                                                                                    
  TARGET_CMAKE_PREFIX_PATH = None                                                                                                                                                                   
  CMAKE_PREFIX_PATH = None                                                                                                                                                                          
  CMAKE_x86_64-pc-windows-gnu = None                                                                                                                                                                
  CMAKE_x86_64_pc_windows_gnu = None                                                                                                                                                                
  TARGET_CMAKE = None                                                                                                                                                                               
  CMAKE = None                                                                                                                                                                                      
  running: "cmake" "/target/x86_64-pc-windows-gnu/debug/build/msdf-sys-e08a5d644d75ca9a/out/msdfgen" "-DCMAKE_SYSTEM_NAME=Windows" "-DMSDFGEN_BUILD_STANDALONE=OFF" "-DMSDFGEN_CORE_ONLY=ON" "-DCMAK
E_TOOLCHAIN_FILE=/opt/toolchain.cmake" "-DCMAKE_INSTALL_PREFIX=/target/x86_64-pc-windows-gnu/debug/build/msdf-sys-e08a5d644d75ca9a/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -m64" 
"-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -m64" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -m64" "-DCMAKE_BUILD_TYPE=Release"                                            
  -- Could NOT find Freetype (missing: FREETYPE_LIBRARY FREETYPE_INCLUDE_DIRS)                                                                                                                      
  -- Configuring done                                                                                                                                                                               
  -- Generating done                                                                                                                                                                                
  -- Build files have been written to: /target/x86_64-pc-windows-gnu/debug/build/msdf-sys-e08a5d644d75ca9a/out/build                                                                                
  running: "cmake" "--build" "." "--target" "msdfgen-core" "--config" "Release" "--parallel" "4"                                                                                                    
  Consolidate compiler generated dependencies of target msdfgen-core                                                                                                                                
  [  4%] Building CXX object CMakeFiles/msdfgen-core.dir/core/Contour.cpp.obj                                                                                                                       
  [  9%] Building CXX object CMakeFiles/msdfgen-core.dir/core/MSDFErrorCorrection.cpp.obj
  [ 13%] Building CXX object CMakeFiles/msdfgen-core.dir/core/EdgeHolder.cpp.obj
  [ 18%] Building CXX object CMakeFiles/msdfgen-core.dir/core/Projection.cpp.obj
  [ 22%] Building CXX object CMakeFiles/msdfgen-core.dir/core/Scanline.cpp.obj
  [ 27%] Building CXX object CMakeFiles/msdfgen-core.dir/core/Shape.cpp.obj
  [ 31%] Building CXX object CMakeFiles/msdfgen-core.dir/core/SignedDistance.cpp.obj
  [ 36%] Building CXX object CMakeFiles/msdfgen-core.dir/core/Vector2.cpp.obj
  [ 40%] Building CXX object CMakeFiles/msdfgen-core.dir/core/contour-combiners.cpp.obj
  [ 45%] Building CXX object CMakeFiles/msdfgen-core.dir/core/edge-coloring.cpp.obj
  [ 50%] Building CXX object CMakeFiles/msdfgen-core.dir/core/edge-segments.cpp.obj
  [ 54%] Building CXX object CMakeFiles/msdfgen-core.dir/core/edge-selectors.cpp.obj
  [ 59%] Building CXX object CMakeFiles/msdfgen-core.dir/core/equation-solver.cpp.obj
  [ 63%] Building CXX object CMakeFiles/msdfgen-core.dir/core/msdf-error-correction.cpp.obj
  [ 68%] Building CXX object CMakeFiles/msdfgen-core.dir/core/msdfgen.cpp.obj
  [ 72%] Building CXX object CMakeFiles/msdfgen-core.dir/core/rasterization.cpp.obj
  [ 77%] Building CXX object CMakeFiles/msdfgen-core.dir/core/save-bmp.cpp.obj
  [ 81%] Building CXX object CMakeFiles/msdfgen-core.dir/core/render-sdf.cpp.obj
  [ 86%] Building CXX object CMakeFiles/msdfgen-core.dir/core/save-tiff.cpp.obj
  [ 90%] Building CXX object CMakeFiles/msdfgen-core.dir/core/sdf-error-estimation.cpp.obj
  [ 95%] Building CXX object CMakeFiles/msdfgen-core.dir/core/shape-description.cpp.obj
  [100%] Linking CXX static library libmsdfgen-core.a
  [100%] Built target msdfgen-core
  cargo:root=/target/x86_64-pc-windows-gnu/debug/build/msdf-sys-e08a5d644d75ca9a/out
  cargo:rustc-link-search=native=/target/x86_64-pc-windows-gnu/debug/build/msdf-sys-e08a5d644d75ca9a/out/build
  cargo:rustc-link-lib=dylib=stdc++

  --- stderr
  CMake Warning:
    Manually-specified variables were not used by the project:

      CMAKE_TOOLCHAIN_FILE

  make: warning: -j4 forced in submake: resetting jobserver mode.
  msdfgen/core/arithmetics.hpp:4:10: fatal error: 'cstdlib' file not found
  thread 'main' panicked at 'Unable to generate bindings: ClangDiagnostic("msdfgen/core/arithmetics.hpp:4:10: fatal error: 'cstdlib' file not found\n")', msdf-sys/build.rs:67:10
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu

This should be part of the search path, but oddly, it's not found:

$ `x86_64-w64-mingw32-gcc-posix -print-prog-name=cc1plus` -v
ignoring nonexistent directory "/usr/lib/gcc/x86_64-w64-mingw32/9.3-posix/../../../../x86_64-w64-mingw32/sys-include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/lib/gcc/x86_64-w64-mingw32/9.3-posix/include/c++
 /usr/lib/gcc/x86_64-w64-mingw32/9.3-posix/include/c++/x86_64-w64-mingw32
 /usr/lib/gcc/x86_64-w64-mingw32/9.3-posix/include/c++/backward
 /usr/lib/gcc/x86_64-w64-mingw32/9.3-posix/include
 /usr/lib/gcc/x86_64-w64-mingw32/9.3-posix/include-fixed
 /usr/lib/gcc/x86_64-w64-mingw32/9.3-posix/../../../../x86_64-w64-mingw32/include
End of search list.

We might need to modify the bindgen sysroot or something. EDIT: A similar issue seems to occur on aarch64-unknown-linux-gnu:

  cargo:root=/target/aarch64-unknown-linux-gnu/debug/build/msdf-sys-653dff40f8c6e658/out
  cargo:rustc-link-search=native=/target/aarch64-unknown-linux-gnu/debug/build/msdf-sys-653dff40f8c6e658/out/build
  cargo:rustc-link-lib=dylib=stdc++

  --- stderr
  make: warning: -j4 forced in submake: resetting jobserver mode.
  /usr/bin/../lib/gcc-cross/aarch64-linux-gnu/9/../../../../include/c++/9/cstdlib:41:10: fatal error: 'bits/c++config.h' file not found
  thread 'main' panicked at 'Unable to generate bindings: ClangDiagnostic("/usr/bin/../lib/gcc-cross/aarch64-linux-gnu/9/../../../../include/c++/9/cstdlib:41:10: fatal error: 'bits/c++config.h' file not found\n")', msdf-sys/build.rs:67:10
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu

Here, for some reason, it's trying to include the host cstdlib (/usr/include/c++/9/cstdlib, which is the cononicalized path of /usr/bin/../lib/gcc-cross/aarch64-linux-gnu/9/../../../../include/c++/9/cstdlib), and therefore cannot find /usr/aarch64-linux-gnu/include/c++/9/aarch64-linux-gnu/bits/c++config.h. It should be using /usr/aarch64-linux-gnu/include/c++/9/cstdlib.

cross-rs / cross