Building on nightly-x86_64-pc-windows-gnu?

[norru]

Hi, I need to build on Windows 10 with the nightly-x86_64-pc-windows-gnu toolchain.

All libs are installed via MSYS2's pacman, lncluding hdf5. HDF5 headers are in /mingw64/include; libs are in /mingw64/lib

When I'm try to build my project on Windows I get the following error. Linux build is fine.

I've had a brief look at the source code and there seems to be a lot of best-effort guessing going on there. Is there any way to make the script's work easier by somehow telling it where the libs actually are?

thread 'main' panicked at 'Unable to locate HDF5 root directory and/or headers.', C:\Users\XXXXXXXXXX\.cargo\registry\src\github.com-1ecc6299db9ec823\hdf5-sys-0.5.2\build.rs:528:13
stack backtrace:
   0: backtrace::backtrace::dbghelp::trace
             at C:\Users\VssAdministrator\.cargo\registry\src\github.com-1ecc6299db9ec823\backtrace-0.3.44\src\backtrace/dbghelp.rs:88
   1: backtrace::backtrace::trace_unsynchronized
             at C:\Users\VssAdministrator\.cargo\registry\src\github.com-1ecc6299db9ec823\backtrace-0.3.44\src\backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print_fmt
             at src\libstd\sys_common/backtrace.rs:78
   3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
             at src\libstd\sys_common/backtrace.rs:59
   4: core::fmt::write
             at src\libcore\fmt/mod.rs:1063
   5: std::io::Write::write_fmt
             at src\libstd\io/mod.rs:1426
   6: std::sys_common::backtrace::_print
             at src\libstd\sys_common/backtrace.rs:62
   7: std::sys_common::backtrace::print
             at src\libstd\sys_common/backtrace.rs:49
   8: std::panicking::default_hook::{{closure}}
             at src\libstd/panicking.rs:204
   9: std::panicking::default_hook
             at src\libstd/panicking.rs:224
  10: std::panicking::rust_panic_with_hook
             at src\libstd/panicking.rs:470
  11: std::panicking::begin_panic
             at /rustc/c20d7eecbc0928b57da8fe30b2ef8528e2bdd5be\src\libstd/panicking.rs:397
  12: build_script_build::LibrarySearcher::try_locate_hdf5_library
             at .\build.rs:528
  13: build_script_build::main
             at .\build.rs:606
  14: std::rt::lang_start::{{closure}}
             at /rustc/c20d7eecbc0928b57da8fe30b2ef8528e2bdd5be\src\libstd/rt.rs:67
  15: std::rt::lang_start_internal::{{closure}}
             at src\libstd/rt.rs:52
  16: std::panicking::try::do_call
             at src\libstd/panicking.rs:303
  17: __rust_maybe_catch_panic
             at src\libpanic_unwind/lib.rs:86
  18: std::panicking::try
             at src\libstd/panicking.rs:281
  19: std::panic::catch_unwind
             at src\libstd/panic.rs:394
  20: std::rt::lang_start_internal
             at src\libstd/rt.rs:51
  21: std::rt::lang_start
             at /rustc/c20d7eecbc0928b57da8fe30b2ef8528e2bdd5be\src\libstd/rt.rs:67
  22: main
  23: _tmainCRTStartup
  24: mainCRTStartup
  25: unit_addrs_search
  26: unit_addrs_search
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

[mulimoen]

You can use HDF5_DIR to set the install directory. #67 will hopefully make this step easier if one only requires a static build.

[norru]

Interesting, thanks! Would HDF5_DIR=/mingw64 do?

[mulimoen]

I am not sure how well cross compilation works, the build script is still somewhat naive. I would recommend trying the branch in #67, which could be a bit easier to get working

[norru]

I'm not cross-compiling though, I'm building on Windows/MSYS/gcc, with a Windows-gnu target.

[norru]

HDF5_DIR does something but the build still fails:

--- stdout
Setting HDF5 root from environment variable:
    HDF5_DIR = "C:/msys64/mingw64"
Found in PATH: "C:\\msys64\\mingw64\\bin"
Parsing HDF5 config from:
    "C:/msys64/mingw64\\include\\H5pubconf.h"
Looking for HDF5 library binary...

--- stderr
thread 'main' panicked at 'Unable to infer HDF5 library runtime version (can't find the binary).', C:\Users\...\.cargo\registry\src\github.com-1ecc6299db9ec823\hdf5-sys-0.5.2\build.rs:155:5

[norru]

It looks like the build script is looking for 'hdf5.dll`, which is not what MSYS2 installs:

find /mingw64 -name "*hdf5.*"
/mingw64/include/hdf5.h
/mingw64/include/shared/hdf5.mod
/mingw64/include/static/hdf5.mod
/mingw64/lib/libhdf5.a
/mingw64/lib/libhdf5.dll.a
/mingw64/lib/libhdf5.settings

[mulimoen]

I would use the static PR. If you manage to get it working, please let us know!

[aldanor]

@norru What is libhdf5.dll.a, is that a dynamic binary? (why the .a then?)

I have a win10 VM, if you could suggest what's the easiest way for me to reproduce this locally (e.g. what do I have to install and where and how) I'll look into it.

[jmrgibson]

Should we be linking against libhdf5.dll.a in the lib dir? or the libhdf5-0.dll in the bin directory? (for reference, here are the package contents)

Does anyone know why the bin dir is only added to the link path if the target is mscv?

I've fixed this locally for myself by adding the bin dir to the library search path by removing the mscv check, and adding libhdf5-0.dll as a valid name for a dll, but I'm not sure what other things I should check to ensure I haven't broken msvc builds.

[andrewsno1]

I had a similar issue when trying to build on Windows 10 with the GNU target.

I could fix the issue for me by manually adding the path "...\HDF5\1.10.3\bin" to the link_paths variable in the function validate_runtime_version (like @jmrgibson, even though my solution is dirtier).

[aldanor]

Would it work the same way with msys2, cygwin and whatever else is out there? (i.e. the /bin folder?)

[windelbouwman]

I can confirm that within an MSYS2 environment, the filepath to the DLL is for example: C:\msys64\ucrt64\bin\libhdf5-0.dll.

Note that on windows pkg-config also might work, so this might be another viable way to find the hdf5 system library.

Edit Possible workarounds / hacks:

1. Copy bin/libhdf5-0.dll to lib/hdf5.dll (when using the gnu toolchain). This is a bit odd, since DLL files are supposed to live in the bin folder in windows.
2. Copy bin/libhdf5-0.dll to bin/hdf5-0.dll (when using msvc toolchain, this will fail on finding .lib files, which are no part of msys2)

[mulimoen]

I would feel more comfortable if we had some typical platforms in CI which we could try this on. Are there some github actions that mimics msys2? Otherwise a PR would be very welcome as I don't have a windows computer myself

[norru]

Just to follow up on this, I am no longer working on the project in which I encountered the issue. Fishing from memory, I ended up using numpy and the hdf5 python bindings.

[windelbouwman]

I would feel more comfortable if we had some typical platforms in CI which we could try this on. Are there some github actions that mimics msys2? Otherwise a PR would be very welcome as I don't have a windows computer myself

Yes, there are github actions for msys2, see this page: https://www.msys2.org/docs/ci/

I would like to give this a shot as well for my own projects. Note that for this to work, you would require rustup toolchain install stable-gnu (https://rust-lang.github.io/rustup/installation/windows.html).

So the way this would work, is adding an msys2 action (with the hdf5 package installed), selecting the proper rust toolchain, and building this crate.

[mulimoen]

It does seem msys2 is installed by default on the runners. I am not quite sure which environment variables to add. At least "C:\msys64\usr\bin" needs to be in path