search

aldanor / hdf5-rust

HDF5 for Rust
https://docs.rs/hdf5
Apache License 2.0
310 stars 85 forks source link

Windows installation #194

Open dwerner95 opened 2 years ago

dwerner95 commented 2 years ago

Hey all,

I am having some trouble compiling on a Windows 10 VM, compilation leads to the error: thread 'main' panicked at 'Unable to infer HDF5 library runtime version (can't find the binary).' Steps I conducted on a complete new windows 10 instance:

Caused by: process didn't exit successfully: C:\Users\rust\Desktop\Programmieren\hdf5_test\target\debug\build\hdf5-sys-755ef1912c6fbc06\build-script-build (exit code: 101) --- stderr thread 'main' panicked at 'Unable to locate HDF5 root directory and/or headers.', C:\Users\rust.cargo\registry\src\github.com-1ecc6299db9ec823\hdf5-sys-0.8.1\build.rs:548:13

--> It is not able to locate the HDF5 directory, which is already odd because i remember on my windows machine it finds all installed HDF5 directories.
so i set the HDF5_DIR variable:
![image](https://user-images.githubusercontent.com/56682866/150740038-2e2f67b4-c90d-431d-8cfd-766cfc923fa7.png)
Now compilation fails to:

Compiling hdf5-sys v0.8.1 error: failed to run custom build command for hdf5-sys v0.8.1

Caused by: process didn't exit successfully: C:\Users\rust\Desktop\Programmieren\hdf5_test\target\debug\build\hdf5-sys-755ef1912c6fbc06\build-script-build (exit code: 101) --- stdout Setting HDF5 root from environment variable: HDF5_DIR = "C:\Program Files\HDF_Group\HDF5\1.12.1" Found in PATH: "C:\Program Files\HDF_Group\HDF5\1.12.1\bin" Parsing HDF5 config from: "C:\Program Files\HDF_Group\HDF5\1.12.1\include\H5pubconf.h" Looking for HDF5 library binary...

--- stderr thread 'main' panicked at 'Unable to infer HDF5 library runtime version (can't find the binary).', C:\Users\rust.cargo\registry\src\github.com-1ecc6299db9ec823\hdf5-sys-0.8.1\build.rs:155:5



I honestly don't know where to go from here.
Can you help me with that? I actually would appreciate if there where some better examples on windows. I also tried using the conda based HDF5 without success.

Thanks!
mulimoen commented 2 years ago

It seems quite weird we can't detect the library. We have some tests in CI to install hdf5 and run tests on this, how do you install the library?

Could you try running cargo build -vvv and post the output from hdf5-sys?

dwerner95 commented 2 years ago

I installed HDF5 with the executable that is delivered with the download. Here is the link of the downloaded file. It seems odd to me, too, especially because I know that it works on my "normal" windows machine.

Here the hdf5-sys output of cargo build -vvv (without HDF5_DIR set):

 Running `set CARGO=\\?\C:\Users\rust\.rustup\toolchains\stable-x86_64-pc-windows-gnu\bin\cargo.exe&& set CARGO_CRATE_NAME=build_script_build&& set CARGO_MANIFEST_DIR=C:\Users\rust\.cargo\registry\src\github.com-1ecc6299db9ec823\hdf5-sys-0.8.1&& set CARGO_PKG_AUTHORS="Ivan Smirnov <i.s.smirnov@gmail.com>"&& set CARGO_PKG_DESCRIPTION="Native bindings to the HDF5 library."&& set CARGO_PKG_HOMEPAGE=https://github.com/aldanor/hdf5-rust&& set CARGO_PKG_LICENSE="MIT OR Apache-2.0"&& set CARGO_PKG_LICENSE_FILE=""&& set CARGO_PKG_NAME=hdf5-sys&& set CARGO_PKG_REPOSITORY=https://github.com/aldanor/hdf5-rust&& set CARGO_PKG_VERSION=0.8.1&& set CARGO_PKG_VERSION_MAJOR=0&& set CARGO_PKG_VERSION_MINOR=8&& set CARGO_PKG_VERSION_PATCH=1&& set CARGO_PKG_VERSION_PRE=""&& set PATH="C:\Users\rust\Desktop\Programmieren\hdf5_test\target\debug\deps;C:\Users\rust\.rustup\toolchains\stable-x86_64-pc-windows-gnu\bin;C:\Users\rust\.cargo\bin;C:\Users\rust\.rustup\toolchains\stable-x86_64-pc-windows-gnu\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files\Git\cmd;C:\Program Files\dotnet\;C:\Program Files\HDF_Group\HDF5\1.12.1\bin;C:\Program Files\HDF_Group\HDF5\1.10.7\bin;C:\Program Files\HDF_Group\HDF5\1.8.22\bin;C:\Program Files\HDF_Group\HDF5\1.13.0\bin;C:\Users\rust\.cargo\bin;C:\Program Files\HDF_Group\HDF5\1.10.7\bin;C:\Users\rust\AppData\Local\Microsoft\WindowsApps;C:\Users\rust\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\rust\.dotnet\tools;"&& rustc --crate-name build_script_build --edition=2018 C:\Users\rust\.cargo\registry\src\github.com-1ecc6299db9ec823\hdf5-sys-0.8.1\build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 --cfg "feature=\"default\"" -C metadata=755ef1912c6fbc06 -C extra-filename=-755ef1912c6fbc06 --out-dir C:\Users\rust\Desktop\Programmieren\hdf5_test\target\debug\build\hdf5-sys-755ef1912c6fbc06 -L dependency=C:\Users\rust\Desktop\Programmieren\hdf5_test\target\debug\deps --extern libloading=C:\Users\rust\Desktop\Programmieren\hdf5_test\target\debug\deps\liblibloading-e066db6ad2f5c92e.rlib --extern regex=C:\Users\rust\Desktop\Programmieren\hdf5_test\target\debug\deps\libregex-cc347d04cbd4196b.rlib --extern serde=C:\Users\rust\Desktop\Programmieren\hdf5_test\target\debug\deps\libserde-e1ceb92f94cbd4e1.rlib --extern serde_derive=C:\Users\rust\Desktop\Programmieren\hdf5_test\target\debug\deps\serde_derive-657830166642c598.dll --extern winreg=C:\Users\rust\Desktop\Programmieren\hdf5_test\target\debug\deps\libwinreg-2b5ba6965704e83a.rlib --cap-lints warn -L native=C:\Users\rust\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-x86_64-pc-windows-gnu-0.4.0\lib`
     Running `C:\Users\rust\Desktop\Programmieren\hdf5_test\target\debug\build\hdf5-sys-755ef1912c6fbc06\build-script-build`
[hdf5-sys 0.8.1] thread 'main' panicked at 'Unable to locate HDF5 root directory and/or headers.', C:\Users\rust\.cargo\registry\src\github.com-1ecc6299db9ec823\hdf5-sys-0.8.1\build.rs:548:13
[hdf5-sys 0.8.1] note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: failed to run custom build command for `hdf5-sys v0.8.1`

Caused by:
  process didn't exit successfully: `C:\Users\rust\Desktop\Programmieren\hdf5_test\target\debug\build\hdf5-sys-755ef1912c6fbc06\build-script-build` (exit code: 101)
  --- stderr
  thread 'main' panicked at 'Unable to locate HDF5 root directory and/or headers.', C:\Users\rust\.cargo\registry\src\github.com-1ecc6299db9ec823\hdf5-sys-0.8.1\build.rs:548:13
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
dwerner95 commented 2 years ago

Here the hdf5-sys output of cargo build -vvv (with HDF5_DIR set):

Compiling hdf5-sys v0.8.1
     Running `set CARGO=\\?\C:\Users\rust\.rustup\toolchains\stable-x86_64-pc-windows-gnu\bin\cargo.exe&& set CARGO_CRATE_NAME=build_script_build&& set CARGO_MANIFEST_DIR=C:\Users\rust\.cargo\registry\src\github.com-1ecc6299db9ec823\hdf5-sys-0.8.1&& set CARGO_PKG_AUTHORS="Ivan Smirnov <i.s.smirnov@gmail.com>"&& set CARGO_PKG_DESCRIPTION="Native bindings to the HDF5 library."&& set CARGO_PKG_HOMEPAGE=https://github.com/aldanor/hdf5-rust&& set CARGO_PKG_LICENSE="MIT OR Apache-2.0"&& set CARGO_PKG_LICENSE_FILE=""&& set CARGO_PKG_NAME=hdf5-sys&& set CARGO_PKG_REPOSITORY=https://github.com/aldanor/hdf5-rust&& set CARGO_PKG_VERSION=0.8.1&& set CARGO_PKG_VERSION_MAJOR=0&& set CARGO_PKG_VERSION_MINOR=8&& set CARGO_PKG_VERSION_PATCH=1&& set CARGO_PKG_VERSION_PRE=""&& set PATH="C:\Users\rust\Desktop\Programmieren\hdf5_test\target\debug\deps;C:\Users\rust\.rustup\toolchains\stable-x86_64-pc-windows-gnu\bin;C:\Users\rust\.cargo\bin;C:\Users\rust\.rustup\toolchains\stable-x86_64-pc-windows-gnu\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files\Git\cmd;C:\Program Files\dotnet\;C:\Program Files\HDF_Group\HDF5\1.12.1\bin;C:\Program Files\HDF_Group\HDF5\1.10.7\bin;C:\Program Files\HDF_Group\HDF5\1.8.22\bin;C:\Program Files\HDF_Group\HDF5\1.13.0\bin;C:\Users\rust\.cargo\bin;C:\Program Files\HDF_Group\HDF5\1.10.7\bin;C:\Users\rust\AppData\Local\Microsoft\WindowsApps;C:\Users\rust\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\rust\.dotnet\tools;"&& rustc --crate-name build_script_build --edition=2018 C:\Users\rust\.cargo\registry\src\github.com-1ecc6299db9ec823\hdf5-sys-0.8.1\build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 --cfg "feature=\"default\"" -C metadata=755ef1912c6fbc06 -C extra-filename=-755ef1912c6fbc06 --out-dir C:\Users\rust\Desktop\Programmieren\hdf5_test\target\debug\build\hdf5-sys-755ef1912c6fbc06 -L dependency=C:\Users\rust\Desktop\Programmieren\hdf5_test\target\debug\deps --extern libloading=C:\Users\rust\Desktop\Programmieren\hdf5_test\target\debug\deps\liblibloading-e066db6ad2f5c92e.rlib --extern regex=C:\Users\rust\Desktop\Programmieren\hdf5_test\target\debug\deps\libregex-cc347d04cbd4196b.rlib --extern serde=C:\Users\rust\Desktop\Programmieren\hdf5_test\target\debug\deps\libserde-e1ceb92f94cbd4e1.rlib --extern serde_derive=C:\Users\rust\Desktop\Programmieren\hdf5_test\target\debug\deps\serde_derive-657830166642c598.dll --extern winreg=C:\Users\rust\Desktop\Programmieren\hdf5_test\target\debug\deps\libwinreg-2b5ba6965704e83a.rlib --cap-lints warn -L native=C:\Users\rust\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-x86_64-pc-windows-gnu-0.4.0\lib`
     Running `C:\Users\rust\Desktop\Programmieren\hdf5_test\target\debug\build\hdf5-sys-755ef1912c6fbc06\build-script-build`
[hdf5-sys 0.8.1] Setting HDF5 root from environment variable:
[hdf5-sys 0.8.1]     HDF5_DIR = "C:\\Program Files\\HDF_Group\\HDF5\\1.12.1"
[hdf5-sys 0.8.1] Found in PATH: "C:\\Program Files\\HDF_Group\\HDF5\\1.12.1\\bin"
[hdf5-sys 0.8.1] Parsing HDF5 config from:
[hdf5-sys 0.8.1]     "C:\\Program Files\\HDF_Group\\HDF5\\1.12.1\\include\\H5pubconf.h"
[hdf5-sys 0.8.1] Looking for HDF5 library binary...
[hdf5-sys 0.8.1] thread 'main' panicked at 'Unable to infer HDF5 library runtime version (can't find the binary).', C:\Users\rust\.cargo\registry\src\github.com-1ecc6299db9ec823\hdf5-sys-0.8.1\build.rs:155:5
[hdf5-sys 0.8.1] note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: failed to run custom build command for `hdf5-sys v0.8.1`

Caused by:
  process didn't exit successfully: `C:\Users\rust\Desktop\Programmieren\hdf5_test\target\debug\build\hdf5-sys-755ef1912c6fbc06\build-script-build` (exit code: 101)
  --- stdout
  Setting HDF5 root from environment variable:
      HDF5_DIR = "C:\\Program Files\\HDF_Group\\HDF5\\1.12.1"
  Found in PATH: "C:\\Program Files\\HDF_Group\\HDF5\\1.12.1\\bin"
  Parsing HDF5 config from:
      "C:\\Program Files\\HDF_Group\\HDF5\\1.12.1\\include\\H5pubconf.h"
  Looking for HDF5 library binary...

  --- stderr
  thread 'main' panicked at 'Unable to infer HDF5 library runtime version (can't find the binary).', C:\Users\rust\.cargo\registry\src\github.com-1ecc6299db9ec823\hdf5-sys-0.8.1\build.rs:155:5
mulimoen commented 2 years ago

What is the name of the library? We expect libhdf5.dylib, libhdf5.so, or hdf5.dll, could there be a mismatch there?

dwerner95 commented 2 years ago

There is a file called hdf5.dll in C:\Program Files\HDF_Group\HDF5\1.12.1\bin

aldanor commented 2 years ago

Could it be a symlink or something along those lines? How come it can't find that binary?

dwerner95 commented 2 years ago

The file seems to be a normal file. no symlink whatsoever. (sorry for it being in german) image

I changed access rights so that all access is granted but nothing changed

mulimoen commented 2 years ago

Seems to work if adding to CI: https://github.com/mulimoen/hdf5-rust/runs/4929209159?check_suite_focus=true

Could it be the searcher is confused by the many installed version of hdf5?

dwerner95 commented 2 years ago

Thanks both! Unfortunately that didn't help @mulimoen. I tried installing it using your modified branch without luck. I used the most advanced debugging technique (prinln) to trace down what the program is doing and it seems as if the function validate_runtime_version gets a config not containing the bin folder, hence it looks for hdf5.dll only in the lib folder. And this comes from the finalize function, it seems that this if statement:

if cfg!(target_env = "msvc") {
   link_paths.push(root_dir.join("bin"));
 }

is false. my question is why? i am a absolute newbie and have no idea what that means.

mulimoen commented 2 years ago

@dwerner95 I only added 1.12.1 to CI, no other changes. Which toolchain do you have installed? (rustup toolchain list)

dwerner95 commented 2 years ago

I maybe need to change the default?

PS C:\Users\rust\Desktop\Programmieren\uPPPP> rustup toolchain list                                                                   
stable-x86_64-pc-windows-gnu (default)                                                                                                
stable-x86_64-pc-windows-msvc
mulimoen commented 2 years ago

You'll have to use msvc with the precompiled hdf5. If you need gnu you can build statically from source

dwerner95 commented 2 years ago

Thank you @mulimoen! that helped. Reason for this bad configuration was a initially missing Visual Studio installation, hence I changed the default to the gnu-toolchain, my bad!

mulimoen commented 2 years ago

We should improve the error message when trying to use a precompiled library with windows-gnu

HyeokSuLee commented 1 year ago

I got 'thread 'main' panicked at 'Unable to locate HDF5 root directory and/or headers.' in windows10. Installed hdf5 1.14 and path added. But error occured.

So I debugged and the problem was hdf5-sys\build.rs:410. It cannot decode reg's value into struct 'App'. I have hdf5 registry {410xxx~ - xxx~ - xxx~} and it also has DisplayName, DisplayVersion,InstallLocation in it. But It failed to decode. This is error message and code for debug.

let app = root.open_subkey(key.to_owned()).ok().and_then(|v| match v.decode::<App>() {
                Ok(app) => Some(app),
                Err(e) => {
                    eprintln!("Failed to decode {:?}: {:?}", key, e);
                    None
                }
            });
>>>
  Failed to decode "{xxxxxxx-xxxx-xxx-xxxx-xxxxxx}": DeserializerError("invalid version")

I think there is some deserialize problem with struct App's version field.