search

microsoft / vcpkg

C++ Library Manager for Windows, Linux, and MacOS
MIT License
22.86k stars 6.3k forks source link

[hdf5] Build error for Windows Arm64 platforms #24898

Open niyas-sait opened 2 years ago

niyas-sait commented 2 years ago

Host Environment

To Reproduce Steps to reproduce the behavior: .\vcpkg.exe install hdf5 --triplet=arm64-windows

Failure logs

From CMake error log

CMake Error at config/cmake/ConfigureChecks.cmake:334 (file):
  file failed to open for reading (No such file or directory):

    C:/Users/niysai01/Workspace/vcpkg/buildtrees/hdf5/arm64-windows-dbg/pac_Cconftest.out
Call Stack (most recent call first):
  CMakeLists.txt:464 (include)

Additional Context

Static builds seem to be working

.\vcpkg.exe install hdf5 --triplet=arm64-windows-static

LilyWangLL commented 2 years ago

Thanks for posting this issue. Please provide the complete error log file. Because when I installing hdf5:arm64-windows locally, except this error, I also have the following error:

CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
   HAVE_IOEO_EXITCODE (advanced)
For details see E:/Lily/hdf5/vcpkg/buildtrees/hdf5/arm64-windows-dbg/TryRunResults.cmake
CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
   H5_PRINTF_LL_TEST_RUN (advanced)
   H5_PRINTF_LL_TEST_RUN__TRYRUN_OUTPUT (advanced)
For details see E:/Lily/hdf5/vcpkg/buildtrees/hdf5/arm64-windows-dbg/TryRunResults.cmake
CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
   RUN_RESULT_VAR (advanced)
   RUN_RESULT_VAR__TRYRUN_OUTPUT (advanced)
For details see E:/Lily/hdf5/vcpkg/buildtrees/hdf5/arm64-windows-dbg/TryRunResults.cmake
CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
   H5_LDOUBLE_TO_LONG_SPECIAL_RUN (advanced)
   H5_LDOUBLE_TO_LONG_SPECIAL_RUN__TRYRUN_OUTPUT (advanced)
For details see E:/Lily/hdf5/vcpkg/buildtrees/hdf5/arm64-windows-dbg/TryRunResults.cmake
CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
   H5_LONG_TO_LDOUBLE_SPECIAL_RUN (advanced)
   H5_LONG_TO_LDOUBLE_SPECIAL_RUN__TRYRUN_OUTPUT (advanced)
For details see E:/Lily/hdf5/vcpkg/buildtrees/hdf5/arm64-windows-dbg/TryRunResults.cmake
CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
   H5_LDOUBLE_TO_LLONG_ACCURATE_RUN (advanced)
   H5_LDOUBLE_TO_LLONG_ACCURATE_RUN__TRYRUN_OUTPUT (advanced)
For details see E:/Lily/hdf5/vcpkg/buildtrees/hdf5/arm64-windows-dbg/TryRunResults.cmake
CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
   H5_LLONG_TO_LDOUBLE_CORRECT_RUN (advanced)
   H5_LLONG_TO_LDOUBLE_CORRECT_RUN__TRYRUN_OUTPUT (advanced)
For details see E:/Lily/hdf5/vcpkg/buildtrees/hdf5/arm64-windows-dbg/TryRunResults.cmake
CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
   H5_NO_ALIGNMENT_RESTRICTIONS_RUN (advanced)
   H5_NO_ALIGNMENT_RESTRICTIONS_RUN__TRYRUN_OUTPUT (advanced)
For details see E:/Lily/hdf5/vcpkg/buildtrees/hdf5/arm64-windows-dbg/TryRunResults.cmake

And your error in this issue will be fixed by new version of hdf5, I saw PR #24758 is doing update hdf5.

niyas-sait commented 2 years ago

Attached console and CMake log.

hd5.log dbg-error.log dbg-out.log

For reference, I am building on windows/arm64 platform

niyas-sait commented 2 years ago

And I tried PR https://github.com/microsoft/vcpkg/pull/24758 but unfortunately getting some error

> cat C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\config-arm64-windows-dbg-err.log
CMake Error at config/cmake/ConfigureChecks.cmake:320 (list):
  list index: 1 out of range (-1, 0)
Call Stack (most recent call first):
  CMakeLists.txt:464 (include)
LilyWangLL commented 2 years ago

Hi @nsait-linaro, PR #24758 has been merged, and it has new change before merged, could you update your vcpkg and install version 1.2.2 of hdf5:arm64-windows? If it still install failed, please also provide the error log. Thanks~

niyas-sait commented 2 years ago

Thanks, @LilyWangLL.

I am getting the exact same error (logs in https://github.com/microsoft/vcpkg/issues/24898#issuecomment-1139452001 ) even with the latest vcpkg.

LilyWangLL commented 2 years ago

Sorry, if your error still is 

CMake Error at config/cmake/ConfigureChecks.cmake:334 (file):
  file failed to open for reading (No such file or directory):

    C:/Users/niysai01/Workspace/vcpkg/buildtrees/hdf5/arm64-windows-dbg/pac_Cconftest.out

This has been merged into new version 1.2.2, the related upstream issue: https://github.com/HDFGroup/hdf5/issues/1558, fixed by https://github.com/HDFGroup/hdf5/pull/1089.

LilyWangLL commented 2 years ago

Ping @nsait-linaro for response.

niyas-sait commented 2 years ago

Unfortunately, It is still failing but with a different error this time.

.\vcpkg.exe install hdf5 --triplet=arm64-windows
Computing installation plan...
The following packages will be built and installed:
    hdf5[core,szip,zlib]:arm64-windows -> 1.12.2
Detecting compiler hash for triplet arm64-windows...
Restored 0 packages from C:\Users\niysai01\AppData\Local\vcpkg\archives in 774.5 us. Use --debug to see more details.
Starting package 1/1: hdf5:arm64-windows
Building package hdf5[core,szip,zlib]:arm64-windows...
-- Using cached HDFGroup-hdf5-hdf5-1_12_2.tar.gz.
-- Cleaning sources at C:/Users/niysai01/Workspace/vcpkg/buildtrees/hdf5/src/df5-1_12_2-c0e67ca243.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source C:/Users/niysai01/Workspace/vcpkg/downloads/HDFGroup-hdf5-hdf5-1_12_2.tar.gz
-- Applying patch hdf5_config.patch
-- Applying patch szip.patch
-- Applying patch pkgconfig-requires.patch
-- Applying patch pkgconfig-link-order.patch
-- Using source at C:/Users/niysai01/Workspace/vcpkg/buildtrees/hdf5/src/df5-1_12_2-c0e67ca243.clean
-- Found external ninja('1.10.2').
-- Configuring arm64-windows-dbg
-- Configuring arm64-windows-rel
-- Building arm64-windows-dbg
-- Building arm64-windows-rel
CMake Error at scripts/cmake/vcpkg_execute_build_process.cmake:158 (message):
    Command failed: C:/Users/niysai01/Workspace/vcpkg/downloads/tools/cmake-3.22.2-windows/cmake-3.22.2-windows-i386/bin/cmake.exe --build . --config Release --target install -- -v -j9
    Working Directory: C:/Users/niysai01/Workspace/vcpkg/buildtrees/hdf5/arm64-windows-rel
    See logs for more information:
      C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\install-arm64-windows-rel-out.log

Call Stack (most recent call first):
  installed/arm64-windows/share/vcpkg-cmake/vcpkg_cmake_build.cmake:111 (vcpkg_execute_build_process)
  installed/arm64-windows/share/vcpkg-cmake/vcpkg_cmake_install.cmake:41 (vcpkg_cmake_build)
  ports/hdf5/portfile.cmake:77 (vcpkg_cmake_install)
  scripts/ports.cmake:146 (include)

Error: Building package hdf5:arm64-windows failed with: BUILD_FAILED

install-arm64-windows-rel-out.log

[1/347] C:\PROGRA~2\MICROS~2\2019\COMMUN~1\VC\Tools\MSVC\1429~1.301\bin\Hostx86\arm64\cl.exe   -D_BIND_TO_CURRENT_VCLIBS_VERSION=1 -D_CONSOLE -D_CRT_NONSTDC_NO_DEPRECATE -D_CRT_SECURE_NO_DEPRECATE -D_CRT_SECURE_NO_WARNINGS -IC:\Users\niysai01\Workspace\vcpkg\installed\arm64-windows\include -IC:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\src\df5-1_12_2-c0e67ca243.clean\src -IC:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\arm64-windows-rel\src /nologo /DWIN32 /D_WINDOWS /utf-8 /MP  -wd5105 /MD /O2 /Oi /Gy /DNDEBUG /Z7 /showIncludes /Fosrc\CMakeFiles\H5make_libsettings.dir\H5make_libsettings.c.obj /Fdsrc\CMakeFiles\H5make_libsettings.dir\ /FS -c C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\src\df5-1_12_2-c0e67ca243.clean\src\H5make_libsettings.c
[2/347] cmd.exe /C "cd . && C:\Users\niysai01\Workspace\vcpkg\downloads\tools\cmake-3.22.2-windows\cmake-3.22.2-windows-i386\bin\cmake.exe -E vs_link_exe --intdir=src\CMakeFiles\H5make_libsettings.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x86\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x86\mt.exe --manifests  -- C:\PROGRA~2\MICROS~2\2019\COMMUN~1\VC\Tools\MSVC\1429~1.301\bin\Hostx86\arm64\link.exe  src\CMakeFiles\H5make_libsettings.dir\H5make_libsettings.c.obj  /out:bin\H5make_libsettings.exe /implib:bin\H5make_libsettings.lib /pdb:bin\H5make_libsettings.pdb /version:0.0 /machine:ARM64 -stack:10000000 /nologo /DEBUG /INCREMENTAL:NO /OPT:REF /OPT:ICF   /subsystem:console  ws2_32.lib  kernel32.lib user32.lib
gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
[3/347] C:\PROGRA~2\MICROS~2\2019\COMMUN~1\VC\Tools\MSVC\1429~1.301\bin\Hostx86\arm64\cl.exe   -D_BIND_TO_CURRENT_VCLIBS_VERSION=1 -D_CONSOLE -D_CRT_NONSTDC_NO_DEPRECATE -D_CRT_SECURE_NO_DEPRECATE -D_CRT_SECURE_NO_WARNINGS -IC:\Users\niysai01\Workspace\vcpkg\installed\arm64-windows\include -IC:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\src\df5-1_12_2-c0e67ca243.clean\src -IC:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\arm64-windows-rel\src /nologo /DWIN32 /D_WINDOWS /utf-8 /MP  -wd5105 /MD /O2 /Oi /Gy /DNDEBUG /Z7 /showIncludes /Fosrc\CMakeFiles\H5detect.dir\H5detect.c.obj /Fdsrc\CMakeFiles\H5detect.dir\ /FS -c C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\src\df5-1_12_2-c0e67ca243.clean\src\H5detect.c
[4/347] cmd.exe /C "cd . && C:\Users\niysai01\Workspace\vcpkg\downloads\tools\cmake-3.22.2-windows\cmake-3.22.2-windows-i386\bin\cmake.exe -E vs_link_exe --intdir=src\CMakeFiles\H5detect.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x86\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x86\mt.exe --manifests  -- C:\PROGRA~2\MICROS~2\2019\COMMUN~1\VC\Tools\MSVC\1429~1.301\bin\Hostx86\arm64\link.exe  src\CMakeFiles\H5detect.dir\H5detect.c.obj  /out:bin\H5detect.exe /implib:bin\H5detect.lib /pdb:bin\H5detect.pdb /version:0.0 /machine:ARM64 -stack:10000000 /nologo /DEBUG /INCREMENTAL:NO /OPT:REF /OPT:ICF   /subsystem:console  ws2_32.lib  kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
[5/347] cmd.exe /C "cd /D C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\arm64-windows-rel\src && C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\arm64-windows-rel\bin\H5detect.exe H5Tinit.c && C:\Users\niysai01\Workspace\vcpkg\downloads\tools\cmake-3.22.2-windows\cmake-3.22.2-windows-i386\bin\cmake.exe -E touch gen_SRCS.stamp1"
FAILED: src/gen_SRCS.stamp1 src/H5Tinit.c C:/Users/niysai01/Workspace/vcpkg/buildtrees/hdf5/arm64-windows-rel/src/gen_SRCS.stamp1 C:/Users/niysai01/Workspace/vcpkg/buildtrees/hdf5/arm64-windows-rel/src/H5Tinit.c
cmd.exe /C "cd /D C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\arm64-windows-rel\src && C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\arm64-windows-rel\bin\H5detect.exe H5Tinit.c && C:\Users\niysai01\Workspace\vcpkg\downloads\tools\cmake-3.22.2-windows\cmake-3.22.2-windows-i386\bin\cmake.exe -E touch gen_SRCS.stamp1"
[6/347] cmd.exe /C "cd /D C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\arm64-windows-rel\src && C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\arm64-windows-rel\bin\H5make_libsettings.exe H5lib_settings.c && C:\Users\niysai01\Workspace\vcpkg\downloads\tools\cmake-3.22.2-windows\cmake-3.22.2-windows-i386\bin\cmake.exe -E touch gen_SRCS.stamp2"
FAILED: src/gen_SRCS.stamp2 src/H5lib_settings.c C:/Users/niysai01/Workspace/vcpkg/buildtrees/hdf5/arm64-windows-rel/src/gen_SRCS.stamp2 C:/Users/niysai01/Workspace/vcpkg/buildtrees/hdf5/arm64-windows-rel/src/H5lib_settings.c
cmd.exe /C "cd /D C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\arm64-windows-rel\src && C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\arm64-windows-rel\bin\H5make_libsettings.exe H5lib_settings.c && C:\Users\niysai01\Workspace\vcpkg\downloads\tools\cmake-3.22.2-windows\cmake-3.22.2-windows-i386\bin\cmake.exe -E touch gen_SRCS.stamp2"
ninja: build stopped: subcommand failed.
LilyWangLL commented 2 years ago

This issue is similar with https://github.com/microsoft/vcpkg/issues/16203, @JackBoosY could you help take a look this issue?

dg0yt commented 2 years ago

C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\arm64-windows-rel\bin\H5detect.exe

IIRC there is a general issue with cross builds of hdf5. It wants to run an executable (on the host, x64) that it builds for the target (arm64). This also prevents android and (cross) mingw builds.

JackBoosY commented 2 years ago

To add arm support, we should do the following steps:

  1. add the following code to dependencies in vcpkg.json: 
    {
"name": "hdf5",
"default-features": false,
"host": true,
"features": [
    "tool"
]
}
  2. find and pass the host tool path to vcpkg_cmake_configure in portfile.cmake if TARGET_TRIPLET is not equal HOST_TRIPLET.
dg0yt commented 2 years ago

To add arm support, we should do the following steps:

  1. add the following code to dependencies in vcpkg.json:
{
    "name": "hdf5",
    "default-features": false,
    "host": true,
    "features": [
        "tool"
    ]
}
  1. find and pass the host tool path to vcpkg_cmake_configure in portfile.cmake if TARGET_TRIPLET is not equal HOST_TRIPLET.

I don't think it is so easy. (And that's why I didn't add it yet.) IIRC the tool is to provide information about the target platform, and the host tool cannot provide this. You really need to run the tool at least once on the target platform, capture the required configuration information, and supply this information directly to the CMake cross build.

e.g. https://forum.hdfgroup.org/t/cross-compiling-for-windows/6735

JackBoosY commented 2 years ago

I don't think it is so easy. (And that's why I didn't add it yet.) IIRC the tool is to provide information about the target platform, and the host tool cannot provide this. You really need to run the tool at least once on the target platform, capture the required configuration information, and supply this information directly to the CMake cross build.

e.g. https://forum.hdfgroup.org/t/cross-compiling-for-windows/6735

I will have a try when I have spare time.

MarkVabulas commented 2 years ago

This also is a major limitation to compiling any other libraries which depend upon this one for arm64 builds.

russelltg commented 2 years ago

I have managed to hack together an hdf5 build for iOS here https://github.com/russelltg/vcpkg/tree/ios_changes_old/ports/hdf5 The files I generated are probably not 100% correct, but at least it builds. Idk if this is a direction that makes sense for this.

MarkVabulas commented 2 years ago

I have managed to hack together an hdf5 build for iOS here https://github.com/russelltg/vcpkg/tree/ios_changes_old/ports/hdf5 The files I generated are probably not 100% correct, but at least it builds. Idk if this is a direction that makes sense for this.

Will this also work for other arm64 builds, or only iOS ones? All of the cross-compiling is broken for arm64 builds, as far as I understand it.

spinicist commented 2 years ago

Hello,

I updated my vcpkg dependencies for the first time in a long time today, and I've hit this issue trying to cross-compile HDF5 for macos-x86 on an M1 (macos-arm64) box. This didn't happen before, so I will try rolling vcpkg back.

The problem also occurs in szip, which is one of HDF5's default dependencies. However, when I set default features to false, the problem occurs within hdf5 itself.

spinicist commented 2 years ago

PS - as @dg0yt noted above, this is a problem with cross-compilation, not with ARM support for HDF5.

I can compile the same project on my M1 box targeting macos-arm64 and everything is fine.

spinicist commented 2 years ago

Apologies for three messages in short succession, and for perhaps confusing matters.

I have rolled back my vcpkg repo to the tag 2022.07.25. HDF5 now builds fine. If I come up to the most recent tag, 2022.08.15, it stops building. I have no idea whether it is a change in the HDF5 port or vcpkg itself between these dates which causes the issue.

This doesn't make a great deal of sense to me because this issue was originally reported in May.

RDCH106 commented 10 months ago

Do you know if there has been any progress here?

I have also experimented a build error using vcpkg 2023.06.20 and arm64-android => #34824

github-actions[bot] commented 4 months ago

This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 180 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.