search

conda-forge / hdf5-feedstock

A conda-smithy repository for hdf5.
BSD 3-Clause "New" or "Revised" License
13 stars 57 forks source link

bump to latest hdf5 version and windows fortran support w/flang #217

Open Krande opened 5 months ago

Krande commented 5 months ago

I thought I might give fortran support for windows a try here as well (seeing as we've had some success in adding fortran support on windows using flang in mumps, mgis, mfront).

However, it failed locally in my initial attempts on windows with Fortran compiler requires either intrinsic functions SIZEOF or STORAGE_SIZE

If that is not a quick fix, I can take the fortran support out and put it in a different PR.

Checklist

Krande commented 5 months ago

@conda-forge-admin, please rerender

conda-forge-webservices[bot] commented 5 months ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

Krande commented 5 months ago

@conda-forge-admin, please rerender

Krande commented 5 months ago

Hey @h-vetinari, I tried flang-new here in hopes of compiling HDF5 with fortran enabled on windows.

Unfortunately it fails during the configuration :( Any ideas where I ought to start looking? (I have posted a question about this on the flang-compiler slack channel also).

-- The Fortran compiler identification is LLVMFlang 18.1.3
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Check for working Fortran compiler: D:/bld/hdf5_1713270629406/_build_env/Library/bin/flang-new.exe - skipped
-- Detecting Fortran/C Interface
-- Detecting Fortran/C Interface - Found GLOBAL and MODULE mangling
-- Verifying Fortran/C Compiler Compatibility
-- Verifying Fortran/C Compiler Compatibility - Success
-- Performing Test H5_FORTRAN_HAVE_SIZEOF
-- Performing Test H5_FORTRAN_HAVE_SIZEOF - Failed
-- Performing Test H5_FORTRAN_HAVE_C_SIZEOF
-- Performing Test H5_FORTRAN_HAVE_C_SIZEOF - Failed
-- Performing Test H5_FORTRAN_HAVE_STORAGE_SIZE
-- Performing Test H5_FORTRAN_HAVE_STORAGE_SIZE - Failed
-- Performing Test H5_FORTRAN_HAVE_ISO_C_BINDING
-- Performing Test H5_FORTRAN_HAVE_ISO_C_BINDING - Failed
-- Performing Test FORTRAN_HAVE_C_LONG_DOUBLE
-- Performing Test FORTRAN_HAVE_C_LONG_DOUBLE - Failed
-- Performing Test FORTRAN_C_LONG_DOUBLE_IS_UNIQUE
-- Performing Test FORTRAN_C_LONG_DOUBLE_IS_UNIQUE - Failed
-- Performing Test FORTRAN_C_BOOL_IS_UNIQUE
-- Performing Test FORTRAN_C_BOOL_IS_UNIQUE - Failed
-- Performing Test HAVE_ISO_FORTRAN_ENV
-- Performing Test HAVE_ISO_FORTRAN_ENV - Failed
CMake Error at config/cmake/HDF5UseFortran.cmake:128 (message):
  Fortran compiler requires either intrinsic functions SIZEOF or STORAGE_SIZE
Call Stack (most recent call first):
  CMakeLists.txt:1076 (include)

link to build logs

Based on

I had hopes that this could work given that it seems LLVM flang compiles HDF5 on linux.

Krande commented 5 months ago

@conda-forge-admin, please rerender

Krande commented 5 months ago

Okay, here's another update.

Regarding the flang windows error. It seems the previous reported error was a simple CMake config forcing invalid flang comands. I added a patch for that which seems to work.

The error we're seeing now is

-- The Fortran compiler identification is LLVMFlang 18.1.3
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Check for working Fortran compiler: C:/Work/miniforge3/envs/msvc-flang/Library/bin/flang-new.exe - skipped
-- Detecting Fortran/C Interface
-- Detecting Fortran/C Interface - Found GLOBAL and MODULE mangling
-- Verifying Fortran/C Compiler Compatibility
-- Verifying Fortran/C Compiler Compatibility - Success
-- Performing Test H5_FORTRAN_HAVE_SIZEOF
-- Performing Test H5_FORTRAN_HAVE_SIZEOF - Success
-- Performing Test H5_FORTRAN_HAVE_C_SIZEOF
-- Performing Test H5_FORTRAN_HAVE_C_SIZEOF - Success
-- Performing Test H5_FORTRAN_HAVE_STORAGE_SIZE
-- Performing Test H5_FORTRAN_HAVE_STORAGE_SIZE - Success
-- Performing Test H5_FORTRAN_HAVE_ISO_C_BINDING
-- Performing Test H5_FORTRAN_HAVE_ISO_C_BINDING - Success
-- Performing Test FORTRAN_HAVE_C_LONG_DOUBLE
-- Performing Test FORTRAN_HAVE_C_LONG_DOUBLE - Success
-- Performing Test FORTRAN_C_LONG_DOUBLE_IS_UNIQUE
-- Performing Test FORTRAN_C_LONG_DOUBLE_IS_UNIQUE - Success
-- Performing Test FORTRAN_C_BOOL_IS_UNIQUE
-- Performing Test FORTRAN_C_BOOL_IS_UNIQUE - Success
-- Performing Test HAVE_ISO_FORTRAN_ENV
-- Performing Test HAVE_ISO_FORTRAN_ENV - Success
-- Performing Test FORTRAN_CHAR_ALLOC
-- Performing Test FORTRAN_CHAR_ALLOC - Success
CMake Error at config/cmake/HDF5UseFortran.cmake:171 (list):
  list index: 2 out of range (-2, 1)
Call Stack (most recent call first):
  CMakeLists.txt:1076 (include)

CMake Error at config/cmake/HDF5UseFortran.cmake:178 (message):
  Failed to find available REAL KINDs for Fortran
Call Stack (most recent call first):
  CMakeLists.txt:1076 (include)

-- Configuring incomplete, errors occurred!

And by closer inspection of the CMakeConfigureLog.yaml

LINK: command "C:\\PROGRA~2\\MICROS~3\\2022\\BUILDT~1\\VC\\Tools\\MSVC\\1438~1.331\\bin\\Hostx64\\x64\\link.exe /nologo CMakeFiles\\cmTC_6dccc.dir\\testFortranCompiler1.f90.obj /out:cmTC_6dccc.exe /implib:cmTC_6dccc.lib /pdb:cmTC_6dccc.exe.dbg /version:0.0 /machine:x64 -stack:10000000 /INCREMENTAL:NO /subsystem:console ws2_32.lib wsock32.lib shlwapi.lib -libpath:C:/Work/miniforge3/envs/msvc-flang/Library/lib -libpath:C:/Work/miniforge3/envs/msvc-flang/Library/lib/clang/18/lib/windows /MANIFEST:EMBED,ID=1" failed (exit code 1120) with the following output:
        testFortranCompiler1.f90.obj : error LNK2019: unresolved external symbol _QMiso_fortran_envECinteger_kinds referenced in function _QQmain
        testFortranCompiler1.f90.obj : error LNK2019: unresolved external symbol _QMiso_fortran_envECreal_kinds referenced in function _QQmain\x0d
        testFortranCompiler1.f90.obj : error LNK2019: unresolved external symbol _QMiso_fortran_envEClogical_kinds referenced in function _QQmain\x0d
        cmTC_6dccc.exe : fatal error LNK1120: 3 unresolved externals\x0d
        ninja: build stopped: subcommand faile

it seems this might be related to https://github.com/llvm/llvm-project/issues/77282.

Krande commented 5 months ago

Hey, here's a small update.

Once https://github.com/llvm/llvm-project/issues/89403 is fixed it might be possible to compile HDF5 with fortran enabled on Windows using LLVM FLang. Related to https://github.com/HDFGroup/hdf5/issues/4419

I'll just convert this PR to a draft for now (pending fix for LLVM flang).

conda-forge-webservices[bot] commented 2 months ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

Krande commented 2 months ago

@conda-forge-admin, please rerender

Krande commented 2 months ago

@conda-forge-admin, please rerender

Krande commented 2 months ago

Hey @h-vetinari, I am testing the latest flang release candidate 19.1.0.rc to compile HDF5 on windows and it seems the previously reported error is gone and replaced by another. So a little bit progress it seems :)

(FYI @mjklemm. Thanks for the great work on making flang work with HDF5. I understand you might not be 100% finished with the necessary changes in flang for HDF5 to compile on windows. But I thought I could give it a try nonetheless :) )

Anyways, the new error is LINK : fatal error LNK1104: cannot open file 'clang_rt.builtins.lib'\x0d, which seems unrelated to previous errors. By doing a quick search in the conda build directories, and I do find similarly named files, such as _build_env\lib\clang\19\lib\windows\clang_rt.builtins-x86_64.lib.

You don't suppose this might be related to the conda flang build itself in the rc branch bld.bat?

For reference, here is the new error (from the CMakeConfigureLog.yaml in this run):

-
    kind: "try_run-v1"
    backtrace:
      - "config/cmake/HDF5UseFortran.cmake:46 (TRY_RUN)"
      - "config/cmake/HDF5UseFortran.cmake:149 (FORTRAN_RUN)"
      - "CMakeLists.txt:1076 (include)"
    directories:
      source: "D:/bld/hdf5_1722438888865/work/build/CMakeFiles/CMakeTmp"
      binary: "D:/bld/hdf5_1722438888865/work/build/CMakeFiles/CMakeTmp"
    cmakeVariables:
      CMAKE_EXE_LINKER_FLAGS: "/machine:x64 -stack:10000000"
      CMAKE_Fortran_FLAGS: ""
      CMAKE_Fortran_FLAGS_RELEASE: ""
      CMAKE_MODULE_PATH: "D:/bld/hdf5_1722438888865/work/config/cmake"
      CMAKE_POSITION_INDEPENDENT_CODE: "ON"
    buildResult:
      variable: "COMPILE_RESULT_VAR"
      cached: true
      stdout: |
        Change Dir: 'D:/bld/hdf5_1722438888865/work/build/CMakeFiles/CMakeTmp'

        Run Build Command(s): D:/bld/hdf5_1722438888865/_build_env/Library/bin/ninja.exe -v cmTC_3969e
        [1/4] C:\\Windows\\system32\\cmd.exe /C "D:\\bld\\hdf5_1722438888865\\_build_env\\Library\\bin\\flang-new.exe -cpp   -fms-runtime-lib=dll -E D:\\bld\\hdf5_1722438888865\\work\\build\\CMakeFiles\\CMakeTmp\\testFortranCompiler1.f90 > CMakeFiles\\cmTC_3969e.dir\\testFortranCompiler1.f90-pp.f90 && D:\\bld\\hdf5_1722438888865\\_build_env\\Library\\bin\\cmake.exe -E cmake_ninja_depends --tdi=CMakeFiles\\cmTC_3969e.dir\\FortranDependInfo.json --lang=Fortran --src=CMakeFiles\\cmTC_3969e.dir\\testFortranCompiler1.f90-pp.f90 --out=CMakeFiles\\cmTC_3969e.dir\\testFortranCompiler1.f90-pp.f90 --dep=CMakeFiles\\cmTC_3969e.dir\\testFortranCompiler1.f90-pp.f90.d --obj=CMakeFiles\\cmTC_3969e.dir\\testFortranCompiler1.f90.obj --ddi=CMakeFiles\\cmTC_3969e.dir\\testFortranCompiler1.f90.obj.ddi --src-orig=D:\\bld\\hdf5_1722438888865\\work\\build\\CMakeFiles\\CMakeTmp\\testFortranCompiler1.f90"
        [2/4] D:\\bld\\hdf5_1722438888865\\_build_env\\Library\\bin\\cmake.exe -E cmake_ninja_dyndep --tdi=CMakeFiles\\cmTC_3969e.dir\\FortranDependInfo.json --lang=Fortran --dd=CMakeFiles\\cmTC_3969e.dir\\Fortran.dd @CMakeFiles\\cmTC_3969e.dir\\Fortran.dd.rsp
        [3/4] D:\\bld\\hdf5_1722438888865\\_build_env\\Library\\bin\\flang-new.exe -ID:\\bld\\hdf5_1722438888865\\work\\build\\CMakeFiles\\CMakeTmp -fms-runtime-lib=dll -ffixed-line-length-72 -o CMakeFiles\\cmTC_3969e.dir\\testFortranCompiler1.f90.obj -c CMakeFiles\\cmTC_3969e.dir\\testFortranCompiler1.f90-pp.f90
        [4/4] C:\\Windows\\system32\\cmd.exe /C "cd . && D:\\bld\\hdf5_1722438888865\\_build_env\\Library\\bin\\cmake.exe -E vs_link_exe --intdir=CMakeFiles\\cmTC_3969e.dir --rc=C:\\PROGRA~2\\WI3CF2~1\\10\\bin\\100226~1.0\\x64\\rc.exe --mt="" --manifests  -- C:\\PROGRA~1\\MICROS~2\\2022\\ENTERP~1\\VC\\Tools\\MSVC\\1429~1.301\\bin\\HostX64\\x64\\link.exe /nologo CMakeFiles\\cmTC_3969e.dir\\testFortranCompiler1.f90.obj  /out:cmTC_3969e.exe /implib:cmTC_3969e.lib /pdb:cmTC_3969e.exe.dbg /version:0.0 /machine:x64 -stack:10000000  /INCREMENTAL:NO /subsystem:console  ws2_32.lib  wsock32.lib  shlwapi.lib -libpath:"D:/bld/hdf5_1722438888865/_build_env/Library/lib" -libpath:"D:/bld/hdf5_1722438888865/_build_env/Library/lib/clang/19/lib/windows"  && cd ."
        FAILED: cmTC_3969e.exe 
        C:\\Windows\\system32\\cmd.exe /C "cd . && D:\\bld\\hdf5_1722438888865\\_build_env\\Library\\bin\\cmake.exe -E vs_link_exe --intdir=CMakeFiles\\cmTC_3969e.dir --rc=C:\\PROGRA~2\\WI3CF2~1\\10\\bin\\100226~1.0\\x64\\rc.exe --mt="" --manifests  -- C:\\PROGRA~1\\MICROS~2\\2022\\ENTERP~1\\VC\\Tools\\MSVC\\1429~1.301\\bin\\HostX64\\x64\\link.exe /nologo CMakeFiles\\cmTC_3969e.dir\\testFortranCompiler1.f90.obj  /out:cmTC_3969e.exe /implib:cmTC_3969e.lib /pdb:cmTC_3969e.exe.dbg /version:0.0 /machine:x64 -stack:10000000  /INCREMENTAL:NO /subsystem:console  ws2_32.lib  wsock32.lib  shlwapi.lib -libpath:"D:/bld/hdf5_1722438888865/_build_env/Library/lib" -libpath:"D:/bld/hdf5_1722438888865/_build_env/Library/lib/clang/19/lib/windows"  && cd ."
        LINK: command "C:\\PROGRA~1\\MICROS~2\\2022\\ENTERP~1\\VC\\Tools\\MSVC\\1429~1.301\\bin\\HostX64\\x64\\link.exe /nologo CMakeFiles\\cmTC_3969e.dir\\testFortranCompiler1.f90.obj /out:cmTC_3969e.exe /implib:cmTC_3969e.lib /pdb:cmTC_3969e.exe.dbg /version:0.0 /machine:x64 -stack:10000000 /INCREMENTAL:NO /subsystem:console ws2_32.lib wsock32.lib shlwapi.lib -libpath:D:/bld/hdf5_1722438888865/_build_env/Library/lib -libpath:D:/bld/hdf5_1722438888865/_build_env/Library/lib/clang/19/lib/windows /MANIFEST:EMBED,ID=1" failed (exit code 1104) with the following output:
        LINK : fatal error LNK1104: cannot open file 'clang_rt.builtins.lib'\x0d
        ninja: build stopped: subcommand failed.

      exitCode: 1
    runResult:
      variable: "RUN_RESULT_VAR"
      cached: true

And also for reference, here is the previously reported error when compiling with flang v18.1:

LINK: command "C:\\PROGRA~2\\MICROS~3\\2022\\BUILDT~1\\VC\\Tools\\MSVC\\1438~1.331\\bin\\Hostx64\\x64\\link.exe /nologo CMakeFiles\\cmTC_6dccc.dir\\testFortranCompiler1.f90.obj /out:cmTC_6dccc.exe /implib:cmTC_6dccc.lib /pdb:cmTC_6dccc.exe.dbg /version:0.0 /machine:x64 -stack:10000000 /INCREMENTAL:NO /subsystem:console ws2_32.lib wsock32.lib shlwapi.lib -libpath:C:/Work/miniforge3/envs/msvc-flang/Library/lib -libpath:C:/Work/miniforge3/envs/msvc-flang/Library/lib/clang/18/lib/windows /MANIFEST:EMBED,ID=1" failed (exit code 1120) with the following output:
        testFortranCompiler1.f90.obj : error LNK2019: unresolved external symbol _QMiso_fortran_envECinteger_kinds referenced in function _QQmain
        testFortranCompiler1.f90.obj : error LNK2019: unresolved external symbol _QMiso_fortran_envECreal_kinds referenced in function _QQmain\x0d
        testFortranCompiler1.f90.obj : error LNK2019: unresolved external symbol _QMiso_fortran_envEClogical_kinds referenced in function _QQmain\x0d
        cmTC_6dccc.exe : fatal error LNK1120: 3 unresolved externals\x0d
        ninja: build stopped: subcommand faile

Best Regards Kristoffer

h-vetinari commented 2 months ago

Glad to hear that flang 19 is looking better! Same situation for openblas BTW. :)

The way I handle this in scipy currently is as follows:

:: need to read clang version for path to compiler-rt
FOR /F "tokens=* USEBACKQ" %%F IN (`clang.exe -dumpversion`) DO (
    SET "CLANG_VER=%%F"
)
set "FFLAGS=-D_CRT_SECURE_NO_WARNINGS -D_MT -D_DLL --target=x86_64-pc-windows-msvc"
set "LDFLAGS=--target=x86_64-pc-windows-msvc -fms-runtime-lib=dll -fuse-ld=lld"
set "LDFLAGS=%LDFLAGS% -Wl,-defaultlib:%BUILD_PREFIX%/Library/lib/clang/!CLANG_VER:~0,2!/lib/windows/clang_rt.builtins-x86_64.lib"

(though some of that should be possible to remove now, e.g. -D_MT -D_DLL is implied by -fms-runtime-lib=dll) I guess that makes a good case that the linkage to compiler-rt should go into the default arguments on the flang feedstock. :)

I've also put this on the list of possible improvements for flang-support in meson.

h-vetinari commented 2 months ago

Alternatively, you could also try

c_compiler:     # [win]
  - clang       # [win]
cxx_compiler:   # [win]
  - clangxx     # [win]

where the activation scripts already take care of linking compiler-rt. I've just opened https://github.com/conda-forge/clang-win-activation-feedstock/pull/40 to clean up the default flags there, and I'll follow on flang side later.

PS. This will probably conflict for now because the windows compiler activation hasn't been rebuilt for llvm 19 yet; I'll do that after the flags are sorted. In the meantime, you have manual work-arounds as mentioned above.

mjklemm commented 2 months ago

With Flang 19.x the error about unresolved external symbols for real_kinds and friends should be gone.

When you setup the Flang build, please make sure that you have compiler-rt in LLVM_ENABLE_PROJECTS. Otherwise, Flang somehow tries to pick up the one that comes with Visual Studio and that does not seem to work.

h-vetinari commented 2 months ago

When you setup the Flang build, please make sure that you have compiler-rt in LLVM_ENABLE_PROJECTS.

We cannot build monolithically for several reasons. But the llvm compiler-rt (with matching version) is present in the environment wherever our flang appears.

Flang somehow tries to pick up the one that comes with Visual Studio and that does not seem to work.

Yes, mainly because they don't have 128bit support (at least, that was the main reason the last time I looked at this; -DAVOID_NATIVE_UINT128_T=1 can help a bit)

Krande commented 2 months ago

Hey @h-vetinari, I seem to have misunderstood the manual flags you suggested. It seems cmake passes the flags to LINK.exe which doesnt like the flags. I tried changing the flags to MSVC (ie. /DEFAULTLIB etc) but no change. What helped though, was renaming clang_rt.builtins-x86_64.lib to clang_rt.builtins.lib and move it to the Library/lib dir.

Do you have any suggestions as to what I should do?

        LINK : warning LNK4044: unrecognized option '/-target=x86_64-pc-windows-msvc'; ignored
        LINK : warning LNK4044: unrecognized option '/fms-runtime-lib=dll'; ignored
        LINK : warning LNK4044: unrecognized option '/fuse-ld=lld'; ignored
        LINK : warning LNK4044: unrecognized option '/Wl,-defaultlib:D:\\bld\\hdf5_1722705974482\\_build_env/Library/lib/clang/19/lib/windows/clang_rt.builtins-x86_64.lib'; ignored\x0d
        LINK : fatal error LNK1104: cannot open file 'clang_rt.builtins.lib'\x0d
        ninja: build stopped: subcommand failed.
h-vetinari commented 2 months ago

Hey @h-vetinari, I seem to have misunderstood the manual flags you suggested.

No you didn't misunderstand, but there was some extra config missing to redirect to LLVM's lld-link.exe instead of MSVC's default link.exe. This should now be sorted through the compiler setup itself - fingers crossed 🤞

h-vetinari commented 2 months ago

Well, now I'm seriously confused. The same build (flang 19.1.0.rc1-h99b286e_3) that works successfully in https://github.com/conda-forge/mumps-feedstock/pull/119 doesn't work here. In particular, what we should be seeing in the logs is something like:

(base) %SRC_DIR%>set "FC=flang-new" 
(base) %SRC_DIR%>set "LD=lld-link.exe" 
(base) %SRC_DIR%>set "FFLAGS=-D_CRT_SECURE_NO_WARNINGS -fms-runtime-lib=dll -fuse-ld=lld -I%LIBRARY_INC%" 
(base) %SRC_DIR%>set "LDFLAGS=-nostdlib -Wl,-defaultlib:%BUILD_PREFIX%/lib/clang/19/windows/clang_rt.builtins-x86_64.lib"
h-vetinari commented 2 months ago

OK, I think I got it, I still need to remove -nostdlib from the linker flags; the mumps build didn't trigger on this, but this is apparently still a flag that flang doesn't support (yet).

The rest of the logs is not visible for some reasons, but the variables do get set.

h-vetinari commented 2 months ago

OK, there seems to be something else involved here:

-- Performing Test FORTRAN_CHAR_ALLOC
-- Performing Test FORTRAN_CHAR_ALLOC - Success
CMake Error at config/cmake/HDF5UseFortran.cmake:171 (list):
  list index: 2 out of range (-2, 1)
Call Stack (most recent call first):
  CMakeLists.txt:1076 (include)

This fails here, which seems like flang doesn't generate the output that HDF5 expects:

# dnl The output from the above program will be:
# dnl    -- LINE 1 --  valid integer kinds (comma separated list)
# dnl    -- LINE 2 --  valid real kinds (comma separated list)
# dnl    -- LINE 3 --  max decimal precision for reals
# dnl    -- LINE 4 --  number of valid integer kinds
# dnl    -- LINE 5 --  number of valid real kinds
# dnl    -- LINE 6 --  number of valid logical kinds
# dnl    -- LINE 7 --  valid logical kinds (comma separated list)
Krande commented 2 months ago

Yeah,

so the fortran code it tries to run and fails on is this

PROGRAM FC08_AVAIL_KINDS
      USE, INTRINSIC :: ISO_FORTRAN_ENV, ONLY : stdout=>OUTPUT_UNIT, integer_kinds, real_kinds, logical_kinds
      IMPLICIT NONE
      INTEGER :: ik, jk, k, max_decimal_prec
      INTEGER :: num_rkinds, num_ikinds, num_lkinds

      ! Find integer KINDs

      num_ikinds = SIZE(integer_kinds)

      DO k = 1, num_ikinds
         WRITE(stdout,'(I0)', ADVANCE='NO') integer_kinds(k)
         IF(k.NE.num_ikinds)THEN
            WRITE(stdout,'(A)',ADVANCE='NO') ','
         ELSE
            WRITE(stdout,'()')
         ENDIF
      ENDDO

      ! Find real KINDs

      num_rkinds = SIZE(real_kinds)

      max_decimal_prec = 1

      prec: DO ik = 2, 36
         exp: DO jk = 1, 700
            k = SELECTED_REAL_KIND(ik,jk)
            IF(k.LT.0) EXIT exp
            max_decimal_prec = ik
         ENDDO exp
      ENDDO prec

      DO k = 1, num_rkinds
         WRITE(stdout,'(I0)', ADVANCE='NO') real_kinds(k)
         IF(k.NE.num_rkinds)THEN
            WRITE(stdout,'(A)',ADVANCE='NO') ','
         ELSE
            WRITE(stdout,'()')
         ENDIF
      ENDDO

     WRITE(stdout,'(I0)') max_decimal_prec
     WRITE(stdout,'(I0)') num_ikinds
     WRITE(stdout,'(I0)') num_rkinds

     ! Find logical KINDs

     num_lkinds = SIZE(logical_kinds)
     WRITE(stdout,'(I0)') num_lkinds

     DO k = 1, num_lkinds
        WRITE(stdout,'(I0)', ADVANCE='NO') logical_kinds(k)
        IF(k.NE.num_lkinds)THEN
           WRITE(stdout,'(A)',ADVANCE='NO') ','
        ELSE
           WRITE(stdout,'()')
        ENDIF
     ENDDO

END PROGRAM FC08_AVAIL_KINDS

If I try to compile using a simple batch script it fails with the missing clang_rt.builtins.lib. If I simply copy clang_rt.builtins-x86_64.lib to make a clang_rt.builtins.lib and point the libdir to that directory it works fine. However, I have not yet been able to recreate it using the hdf5 cmake build :(

:: Make sure you've activated the flang environment
set clanglibdir=%CONDA_PREFIX:\=/%/lib/clang/19/lib/windows

REM It seems whatever I do, I have to rename the clang_rt.builtins-x86_64.lib to clang_rt.builtins.lib
copy "%clanglibdir%\clang_rt.builtins-x86_64.lib" "%clanglibdir%\clang_rt.builtins.lib"

flang-new -L=%CONDA_PREFIX:\=/%/lib/clang/19/lib/windows read_and_integer_kinds.f90 -o read_and_integer_kinds.exe
call read_and_integer_kinds.exe

Which outputs

(flangdev) C:\Work\code\flang_debug\flang_real2>call read_and_integer_kinds.exe
1,2,4,8,16
2,3,4,8,10,16
15
5
6
4
1,2,4,8

I'll try to investigate further down in the cmake config tomorrow.

h-vetinari commented 2 months ago

If I try to compile using a simple batch script it fails with the missing clang_rt.builtins.lib.

It sounds like the try_compile isn't using LDFLAGS, because otherwise there would be no problem.

h-vetinari commented 2 months ago

... which is because the FORTRAN_RUN CMake macro in HDF5 resets the flags as it wants. This is wrong and needs to be patched (or better: fixed upstream).

Krande commented 1 month ago

Hey @h-vetinari,

I was going to make a patch, so I have been trying a few different approaches locally where I force the appropriate flags to the FORTRAN_RUN (and the CMAKE build logs confirm that the flags are passed in). But it still seems like the could not open 'clang_rt.builtins.lib': no such file or directory is present no matter what I try..

Are you sure we aren't missing anything else?

I tried to simplify this by modifying the previous batch file example I made by doing this:

@echo off

setlocal enabledelayedexpansion

:: need to read clang version for path to compiler-rt
FOR /F "tokens=* USEBACKQ" %%F IN (`clang.exe -dumpversion`) DO (
    SET "CLANG_VER=%%F"
)

set CL_LIBDIR=%CONDA_PREFIX:/=/%/Library/lib/clang/!CLANG_VER:~0,2!/lib/windows

set "FFLAGS=-D_CRT_SECURE_NO_WARNINGS -D_MT -D_DLL --target=x86_64-pc-windows-msvc"
set "LDFLAGS=-fms-runtime-lib=dll -fuse-ld=lld"
set "LDFLAGS=%LDFLAGS% -Wl,-defaultlib:%CL_LIBDIR%/clang_rt.builtins-x86_64.lib"

set "CLI_ARGS=%FFLAGS% %LDFLAGS%"

flang-new %CLI_ARGS% read_and_integer_kinds.f90 -o read_and_integer_kinds.exe -v

call read_and_integer_kinds.exe

endlocal

That ends up with the following error output

flang-new version 19.1.0-rc2
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Work\Miniforge3\envs\flangdev\Library\bin
 "C:\\Work\\Miniforge3\\envs\\flangdev\\Library\\bin\\flang-new" -fc1 -triple x86_64-pc-windows-msvc19.40.33811 -emit-obj -D _CRT_SECURE_NO_WARNINGS -D _MT -D _DLL -mrelocation-model pic -pic-level 2 -target-cpu x86-64 --dependent-lib=clang_rt.builtins-x86_64.lib -D_MT -D_DLL --dependent-lib=msvcrt --dependent-lib=FortranRuntime.dynamic.lib --dependent-lib=FortranDecimal.dynamic.lib -D_MSC_VER=1940 -D_MSC_FULL_VER=194033811 -D_WIN32 -D_M_X64=100 -resource-dir "C:\\Work\\Miniforge3\\envs\\flangdev\\Library\\lib\\clang\\19" -mframe-pointer=none -o "C:\\Users\\KRISTO~1\\AppData\\Local\\Temp\\read_and_integer_kinds-87ae5c.o" -x f95-cpp-input read_and_integer_kinds.f90
 "C:\\Work\\Miniforge3\\envs\\flangdev\\Library\\bin\\lld-link" -out:read_and_integer_kinds.exe "-libpath:C:\\Program Files\\Microsoft Visual Studio\\2022\\Professional\\VC\\Tools\\MSVC\\14.40.33807\\lib\\x64" "-libpath:C:\\Program Files\\Microsoft Visual Studio\\2022\\Professional\\VC\\Tools\\MSVC\\14.40.33807\\atlmfc\\lib\\x64" "-libpath:C:\\Program Files (x86)\\Windows Kits\\10\\Lib\\10.0.26100.0\\ucrt\\x64" "-libpath:C:\\Program Files (x86)\\Windows Kits\\10\\Lib\\10.0.26100.0\\um\\x64" "-libpath:C:\\Work\\Miniforge3\\envs\\flangdev\\Library\\lib" /subsystem:console "-libpath:C:\\Work\\Miniforge3\\envs\\flangdev\\Library\\lib\\clang\\19\\lib\\windows" -nologo "-defaultlib:C:\\Work\\Miniforge3\\envs\\flangdev/Library/lib/clang/19/lib/windows/clang_rt.builtins-x86_64.lib" "C:\\Users\\KRISTO~1\\AppData\\Local\\Temp\\read_and_integer_kinds-87ae5c.o"
lld-link: error: could not open 'clang_rt.builtins.lib': no such file or directory
flang-new: error: linker command failed with exit code 1 (use -v to see invocation)
'read_and_integer_kinds.exe' is not recognized as an internal or external command,
operable program or batch file.

Is there something not working with the -defaultlib:<*.lib> flag? Or are we missing something else?

h-vetinari commented 1 month ago

The variables %FFLAGS% and %LDFLAGS% will already be populated correctly by the compiler activation. The error message

lld-link: error: could not open 'clang_rt.builtins.lib': no such file or directory

points to the "wrong" library (compare with LDFLAGS which points to [...]/clang_rt.builtins-x86_64.lib). Not sure how that happens.

But looking at some of the logs, it appears that lld-link is perhaps in windows-only mode, because otherwise it shouldn't complain about the following arguments:

        lld-link: warning: ignoring unknown argument '--target=x86_64-pc-windows-msvc'
        lld-link: warning: ignoring unknown argument '-fms-runtime-lib=dll'
        lld-link: warning: ignoring unknown argument '-fuse-ld=lld'
        lld-link: warning: ignoring unknown argument '-Wl,-defaultlib:D:\\bld\\hdf5_1723301788364\\_build_env/Library/lib/clang/19/lib/windows/clang_rt.builtins-x86_64.lib'