[Draft] Try to build with flang 18 on Windows

traversaro commented 7 months ago

(do not merge, just an experiment related to https://github.com/conda-forge/conda-forge-pinning-feedstock/pull/1359)

Checklist

[ ] Used a personal fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase code>@<space/conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

conda-forge-webservices[bot] commented 7 months ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

traversaro commented 7 months ago

@conda-forge-admin, please rerender

github-actions[bot] commented 7 months ago

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you but ran into some issues. Please check the output logs of the latest rerendering GitHub actions workflow run for errors. You can also ping conda-forge/core for further assistance or try re-rendering locally.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/mumps-feedstock/actions/runs/7783826409.

traversaro commented 7 months ago

@conda-forge-admin, please rerender

traversaro commented 7 months ago

@conda-forge-admin, please rerender

traversaro commented 7 months ago

@conda-forge-admin, please rerender

github-actions[bot] commented 7 months ago

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/mumps-feedstock/actions/runs/7783996051.

traversaro commented 7 months ago

Current failures:

2024-02-05T12:39:10.7068303Z -- The Fortran compiler identification is LLVMFlang 18.1.0
2024-02-05T12:39:11.6217895Z -- The C compiler identification is Clang 18.1.0 with MSVC-like command-line
2024-02-05T12:39:11.8279108Z -- Detecting Fortran compiler ABI info
2024-02-05T12:39:28.3126243Z -- Detecting Fortran compiler ABI info - done
2024-02-05T12:39:28.8365133Z -- Check for working Fortran compiler: D:/bld/mumps_1707136264872/_build_env/Library/bin/flang-new.exe - skipped
2024-02-05T12:39:28.8366882Z -- Detecting C compiler ABI info
2024-02-05T12:39:29.5516874Z -- Detecting C compiler ABI info - done
2024-02-05T12:39:29.5531558Z -- Check for working C compiler: D:/bld/mumps_1707136264872/_build_env/Library/bin/clang-cl.exe - skipped
2024-02-05T12:39:29.5547018Z -- Detecting C compile features
2024-02-05T12:39:29.5564718Z -- Detecting C compile features - done
2024-02-05T12:39:29.5644183Z -- Looking for Fortran sgemm
2024-02-05T12:39:29.9181059Z -- Looking for Fortran sgemm - not found
2024-02-05T12:39:29.9228196Z -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
2024-02-05T12:39:30.3316649Z -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
2024-02-05T12:39:30.3318355Z -- Looking for pthread_create in pthreads
2024-02-05T12:39:31.1050294Z -- Looking for pthread_create in pthreads - not found
2024-02-05T12:39:31.1051569Z -- Looking for pthread_create in pthread
2024-02-05T12:39:31.2674221Z -- Looking for pthread_create in pthread - not found
2024-02-05T12:39:31.2693903Z -- Found Threads: TRUE  
2024-02-05T12:39:32.4997284Z -- Looking for Fortran sgemm
2024-02-05T12:39:33.3419031Z -- Looking for Fortran sgemm - found
2024-02-05T12:39:33.3427729Z -- Found BLAS: D:/bld/mumps_1707136264872/_h_env/Library/lib/blas.lib  
2024-02-05T12:39:33.6998312Z -- Looking for Fortran cheev
2024-02-05T12:39:34.1690772Z -- Looking for Fortran cheev - not found
2024-02-05T12:39:34.2434860Z -- Looking for Fortran cheev
2024-02-05T12:39:35.0787250Z -- Looking for Fortran cheev - found
2024-02-05T12:39:35.0796885Z -- Found LAPACK: D:/bld/mumps_1707136264872/_h_env/Library/lib/lapack.lib;D:/bld/mumps_1707136264872/_h_env/Library/lib/blas.lib  
2024-02-05T12:39:35.0867781Z -- Configuring done (37.3s)
2024-02-05T12:39:35.3008197Z -- Generating done (0.2s)
2024-02-05T12:39:35.3310955Z -- Build files have been written to: D:/bld/mumps_1707136264872/work/build
2024-02-05T12:39:35.3921296Z ninja: error: '/WHOLEARCHIVE:Fortran_main.static.lib', needed by 'c_example.exe', missing and no known rule to make it
2024-02-05T12:39:37.2138372Z Traceback (most recent call last):

interestingly, the library installed by https://anaconda.org/conda-forge/libfortran-main is called Fortran_main.lib, not Fortran_main.static.lib as it seems that CMake is trying to use.

traversaro commented 7 months ago

interestingly, the library installed by https://anaconda.org/conda-forge/libfortran-main is called Fortran_main.lib, not Fortran_main.static.lib as it seems that CMake is trying to use.

Indeed, the problem is that with flang 18.1.0rc1 flang-new -### dummy.F outputs:

8047\work\build3>flang-new.exe -### ./test.F
flang-new version 18.1.0rc
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Users\straversaro\AppData\Local\miniforge3\conda-bld\mumps_1707139278047\_build_env\Library\bin
 "C:\\Users\\straversaro\\AppData\\Local\\miniforge3\\conda-bld\\mumps_1707139278047\\_build_env\\Library\\bin\\flang-new" "-fc1" "-triple" "x86_64-pc-windows-msvc19.29.30153" "-emit-obj" "-fcolor-diagnostics" "-mrelocation-model" "pic" "-pic-level" "2" "-target-cpu" "x86-64" "--dependent-lib=clang_rt.builtins-x86_64.lib" "-D_MT" "--dependent-lib=libcmt" "--dependent-lib=Fortran_main.static.lib" "--dependent-lib=FortranRuntime.static.lib" "--dependent-lib=FortranDecimal.static.lib" "-D_MSC_VER=1929" "-D_MSC_FULL_VER=192930153" "-D_WIN32" "-D_M_X64=100" "-mframe-pointer=none" "-o" "C:\\Users\\STRAVE~1\\AppData\\Local\\Temp\\test-3d1e5f.o" "-x" "f95-cpp-input" "./test.F"
 "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.29.30133\\bin\\Hostx64\\x64\\link.exe" "-out:a.exe" "-libpath:C:\\Users\\straversaro\\AppData\\Local\\miniforge3\\conda-bld\\mumps_1707139278047\\_build_env\\Library\\lib" "/WHOLEARCHIVE:Fortran_main.static.lib" "/subsystem:console" "-libpath:C:\\Users\\straversaro\\AppData\\Local\\miniforge3\\conda-bld\\mumps_1707139278047\\_build_env\\Library\\lib\\clang\\18\\lib\\windows" "-nologo" "C:\\Users\\STRAVE~1\\AppData\\Local\\Temp\\test-3d1e5f.o"

while flang 17 returned:

(flang) C:\Users\straversaro>flang-new -### ./dummy.F
flang-new version 17.0.6
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Users\straversaro\AppData\Local\miniforge3\envs\flang\Library\bin
 "C:\\Users\\straversaro\\AppData\\Local\\miniforge3\\envs\\flang\\Library\\bin\\flang-new" "-fc1" "-triple" "x86_64-pc-windows-msvc19.38.33135" "-emit-obj" "-fcolor-diagnostics" "-mrelocation-model" "pic" "-pic-level" "2" "-target-cpu" "x86-64" "-o" "C:\\Users\\STRAVE~1\\AppData\\Local\\Temp\\dummy-29bfa2.o" "-x" "f95-cpp-input" "./dummy.F"
 "C:\\Program Files\\Microsoft Visual Studio\\2022\\Professional\\VC\\Tools\\MSVC\\14.38.33130\\bin\\Hostx64\\x64\\link.exe" "-out:a.exe" "-libpath:C:\\Users\\straversaro\\AppData\\Local\\miniforge3\\envs\\flang\\Library\\lib" "Fortran_main.lib" "FortranRuntime.lib" "FortranDecimal.lib" "/subsystem:console" "-libpath:C:\\Users\\straversaro\\AppData\\Local\\miniforge3\\envs\\flang\\Library\\lib\\clang\\17\\lib\\windows" "-nologo" "C:\\Users\\STRAVE~1\\AppData\\Local\\Temp\\dummy-29bfa2.o"

traversaro commented 7 months ago

I guess that the Fortran_main.static.lib library was added in https://github.com/llvm/llvm-project/pull/70833, and indeed the library is built (from https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=868692&view=logs&jobId=a70f640f-cc53-5cd3-6cdc-236a1aa90802):

2024-02-01T18:33:16.3274953Z [599/635] Linking C static library lib\Fortran_main.dynamic.lib
2024-02-01T18:33:16.3275518Z [600/635] Linking C static library lib\Fortran_main.static.lib
2024-02-01T18:33:16.3276115Z [601/635] Building C object runtime\FortranMain\CMakeFiles\obj.Fortran_main.static_dbg.dir\Fortran_main.c.obj
2024-02-01T18:33:22.5790948Z [602/635] Building C object runtime\FortranMain\CMakeFiles\obj.Fortran_main.dynamic_dbg.dir\Fortran_main.c.obj
2024-02-01T18:33:22.5849854Z [603/635] Linking C static library lib\Fortran_main.static_dbg.lib
2024-02-01T18:33:22.5893492Z [604/635] Linking C static library lib\Fortran_main.dynamic_dbg.lib
2024-02-01T18:33:22.5937144Z [605/635] Linking C static library lib\Fortran_main.lib
2024-02-01T18:33:22.5946362Z [606/635] Building CXX object runtime\CMakeFiles\obj.FortranRuntime.dynamic_dbg.dir\unit.cpp.obj
2024-02-01T18:33:22.5989950Z [607/635] Linking CXX static library lib\FortranRuntime.dynamic_dbg.lib
2024-02-01T18:33:22.6078831Z [608/635] Linking CXX static library lib\FortranRuntime.lib
2024-02-01T18:33:23.8771005Z [609/635] Linking CXX executable bin\flang-new.exe

probably we need to add them in https://github.com/conda-forge/flang-feedstock/blob/rc/recipe/install_libfortran_main.bat#L3 . @h-vetinari are you interested in PRs on the rc branch of flang-feedstock, or are you still working on that so you prefer to handle that yourself? Thanks a lot in advance!

h-vetinari commented 7 months ago

@h-vetinari are you interested in PRs on the rc branch of flang-feedstock, or are you still working on that so you prefer to handle that yourself? Thanks a lot in advance!

Thanks for the analysis, would gladly take a PR! I've noted some relevant changes during the 18 cycle in an issue on the flang feedstock, but I haven't analysed all consequences. This looks like it's one of them.

traversaro commented 7 months ago

Actually, I am not sure about what I written, it seems that Fortran_main.static.lib is just installed in the flang package, while it could make sense to have it in libfortran-main, everything should work fine even if it is installed by flang.

traversaro commented 7 months ago

Reading better the ninja error, I think that in /WHOLEARCHIVE:Fortran_main.static.lib the problem is not the library name, but the /WHOLEARCHIVE: part.

traversaro commented 7 months ago

Manually removing the problematic dependency from build.ninja, the compilation fails with this error:

[232/912] Building Fortran object CMakeFiles\mumps_common.dir\src\double_linked_list.F.obj
FAILED: CMakeFiles/mumps_common.dir/src/double_linked_list.F.obj mumps_ddll.mod mumps_idll.mod
C:\Users\straversaro\AppData\Local\miniforge3\conda-bld\mumps_1707139278047\_build_env\Library\bin\flang-new.exe -IC:\Users\straversaro\AppData\Local\miniforge3\conda-bld\mumps_1707139278047\work\src -IC:\Users\straversaro\AppData\Local\miniforge3\conda-bld\mumps_1707139278047\work\include -IC:\Users\straversaro\AppData\Local\miniforge3\conda-bld\mumps_1707139278047\work\src -IC:\Users\straversaro\AppData\Local\miniforge3\conda-bld\mumps_1707139278047\work\PORD\include -IC:\Users\straversaro\AppData\Local\miniforge3\conda-bld\mumps_1707139278047\work\libseq -ffixed-line-length-72 -o CMakeFiles\mumps_common.dir\src\double_linked_list.F.obj -c CMakeFiles\mumps_common.dir\src\double_linked_list.F-pp.f
error: loc("C:\\Users\\straversaro\\AppData\\Local\\miniforge3\\conda-bld\\mumps_1707139278047\\work\\src\\double_linked_list.F":32:11): operation with block successors must terminate its parent block
error: verification of lowering to FIR failed
[233/912] Building Fortran object CMakeFiles\mumps_common.dir\src\front_data_mgt_m.F.obj
FAILED: CMakeFiles/mumps_common.dir/src/front_data_mgt_m.F.obj mumps_front_data_mgt_m.mod
C:\Users\straversaro\AppData\Local\miniforge3\conda-bld\mumps_1707139278047\_build_env\Library\bin\flang-new.exe -IC:\Users\straversaro\AppData\Local\miniforge3\conda-bld\mumps_1707139278047\work\src -IC:\Users\straversaro\AppData\Local\miniforge3\conda-bld\mumps_1707139278047\work\include -IC:\Users\straversaro\AppData\Local\miniforge3\conda-bld\mumps_1707139278047\work\src -IC:\Users\straversaro\AppData\Local\miniforge3\conda-bld\mumps_1707139278047\work\PORD\include -IC:\Users\straversaro\AppData\Local\miniforge3\conda-bld\mumps_1707139278047\work\libseq -ffixed-line-length-72 -o CMakeFiles\mumps_common.dir\src\front_data_mgt_m.F.obj -c CMakeFiles\mumps_common.dir\src\front_data_mgt_m.F-pp.f
error: loc("C:\\Users\\straversaro\\AppData\\Local\\miniforge3\\conda-bld\\mumps_1707139278047\\work\\src\\front_data_mgt_m.F":92:7): operation with block successors must terminate its parent block
error: verification of lowering to FIR failed
ninja: build stopped: subcommand failed.

h-vetinari commented 7 months ago

error: verification of lowering to FIR failed

sounds like flang isn't yet ready for certain syntax constructs (FIR is an intermediate representation during compilation).

Did mumps build with flang 17 or is this the first time you're attempting it? If it's the former, then it's a regression. If it's the latter, it's probably on someone's to-do list already. But in any case, it's probably not a bad idea to raise an issue upstream.

Krande commented 5 months ago

Hey @traversaro and @h-vetinari

Now that the "lowering to FIR" issue in conda-forge flang was resolved in https://github.com/conda-forge/flang-feedstock/issues/46 I thought I'd check this PR.

And it seems like with the fix for flang this PR successfully compiles with only a minor addition.

All I had to do was to add msvcrt.lib to LDFLAGS in the bld.seq.bat

set "LDFLAGS=%LDFLAGS% msvcrt.lib"

in order to resolve some linking issues

[907/913] Linking Fortran shared library zmumps.dll
FAILED: zmumps.dll zmumps.lib
...
mumps_common.lib(mumps_io.c.obj) : error LNK2019: unresolved external symbol __imp_fopen referenced in function mumps_dumprhsbinary_c_
mumps_common.lib(mumps_io_basic.c.obj) : error LNK2001: unresolved external symbol __imp_fopen
mumps_common.lib(mumps_io_err.c.obj) : error LNK2019: unresolved external symbol __imp_strncpy referenced in function mumps_io_error
mumps_common.lib(mumps_save_restore_C.c.obj) : error LNK2001: unresolved external symbol __imp_strncpy
mumps_common.lib(mumps_io_err.c.obj) : error LNK2019: unresolved external symbol __imp_strdup referenced in function mumps_io_sys_error
oldnames.lib(strdup.obi) : error LNK2001: unresolved external symbol __imp_strdup
mumps_common.lib(elapse.c.obj) : error LNK2019: unresolved external symbol __imp__ftime64 referenced in function mumps_elapse_
mumps_common.lib(mumps_io_basic.c.obj) : error LNK2019: unresolved external symbol __imp_fseek referenced in function mumps_io_do_write_block
mumps_common.lib(mumps_io_basic.c.obj) : error LNK2019: unresolved external symbol __imp_ferror referenced in function mumps_io_do_write_block
mumps_common.lib(mumps_io_basic.c.obj) : error LNK2019: unresolved external symbol __imp_fread referenced in function mumps_io_do_read_block
mumps_common.lib(mumps_io_basic.c.obj) : error LNK2019: unresolved external symbol __imp_ftell referenced in function mumps_compute_file_size
oldnames.lib(strdup.obi) : error LNK2001: unresolved external symbol __imp__strdup
zmumps.dll : fatal error LNK1120: 9 unresolved externals
ninja: build stopped: subcommand failed.

traversaro commented 5 months ago

Thanks @Krande , that is great! Feel free to edit the PR, I added you as a collaborator. Linking msvcrt sounds strange to me as I thought everything kind of migrated to ucrt, anyhow I am a bit short on time now, I will look in detail later.

traversaro commented 5 months ago

I wonder if there is some NODEFAULTLIB option being passed by someone? We could also test compiling with cl.exe instead of clang-cl.exe, for consistency with the rest of conda-forge

Krande commented 5 months ago

@conda-forge-admin, please rerender

Krande commented 5 months ago

@conda-forge-admin, please rerender

github-actions[bot] commented 5 months ago

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/mumps-feedstock/actions/runs/8708334918.

Krande commented 5 months ago

@traversaro If we want to build against metis and libscotch. I reckon we need to do something like scivision are doing in their FindMETIS.cmake?

traversaro commented 5 months ago

@traversaro If we want to build against metis and libscotch. I reckon we need to do something like scivision are doing in their FindMETIS.cmake?

Yes, I guess so.

h-vetinari commented 5 months ago

And it seems like with the fix for flang this PR successfully compiles with only a minor addition.

All I had to do was to add msvcrt.lib to LDFLAGS in the bld.seq.bat
set "LDFLAGS=%LDFLAGS% msvcrt.lib"

That's great news! 🥳

CC also @isuruf

h-vetinari commented 5 months ago

I wonder if there is some NODEFAULTLIB option being passed by someone?

We do set --dependent-lib=msvcrt for clang, but looking at this now, I'm pretty sure our flang activation when used together with msvc is lacking...

h-vetinari commented 5 months ago

We do set --dependent-lib=msvcrt for clang [...]

Note that flang only gained support for this option in LLVM 18, see https://github.com/conda-forge/flang-feedstock/issues/35

Krande commented 5 months ago

@conda-forge-admin, please rerender

h-vetinari commented 1 month ago

I rebased this PR in #119 for testing with the upcoming flang 19

traversaro commented 1 month ago

I rebased this PR in #119 for testing with the upcoming flang 19

Great! Let's close this PR as it was superseded by https://github.com/conda-forge/mumps-feedstock/pull/119 .

conda-forge / mumps-feedstock

[Draft] Try to build with flang 18 on Windows #111