search

firemodels / fds

Fire Dynamics Simulator
https://pages.nist.gov/fds-smv/
Other
629 stars 611 forks source link

Intel Fortran Compiler Classic (ifort) deprecation by the end of 2024 and flag issue in ifx #13132

Open KingCZE opened 1 week ago

KingCZE commented 1 week ago

Hello, As Intel Fortran Compiler Classic (ifort) is going to be deprecated by the end of 2024 (https://community.intel.com/t5/Blogs/Tech-Innovation/Tools/Deprecation-of-The-Intel-Fortran-Compiler-Classic-ifort/post/1541699), are you planning a transition to Intel Fortran Compiler (ifx)?

I managed to compile FDS in ifx; however, I encountered an issue with one of the compiler flags. /Qipo, calling -flto and -fuse-ld=lld, did not work in my case. It opened "lld-link.exe"; however, it ended up with the followng error message:

PLEASE submit a bug report to https://software.intel.com/en-us/support/priority-support and include the crash backtrace.
Stack dump:
0.      Program arguments: C:\\PROGRA~2\\Intel\\oneAPI\\compiler\\latest\\bin\\compiler\\lld-link @C:\\Users\\Patrik\\AppData\\Local\\Temp\\19380122arg3
Exception Code: 0xC0000005
 #0 0x00007ff6ee47493b (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x1fa493b)
 #1 0x00007ff6ed5a216f (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x10d216f)
 #2 0x00007ff6eceebe03 (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0xa1be03)
 #3 0x00007ff6eceebfdc (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0xa1bfdc)
 #4 0x00007ff6ee473570 (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x1fa3570)
 #5 0x00007ff6ee472c82 (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x1fa2c82)
 #6 0x00007ff6edfaa6ea (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x1ada6ea)
 #7 0x00007ff6edfab3f1 (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x1adb3f1)
 #8 0x00007ff6edfaafd2 (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x1adafd2)
 #9 0x00007ff6ec56fa7d (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x9fa7d)
#10 0x00007ff6ec534484 (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x64484)
#11 0x00007ff6ec505dbb (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x35dbb)
#12 0x00007ff6ec4fbdd3 (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x2bdd3)
#13 0x00007ff6ec4f8885 (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x28885)
#14 0x00007ff6ec4d1294 (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x1294)
#15 0x00007ff6ec4d17c3 (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x17c3)
#16 0x00007ff6ed74dd08 (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x127dd08)
#17 0x00007ffcd89c257d (C:\Windows\System32\KERNEL32.DLL+0x1257d)
#18 0x00007ffcd9d2af28 (C:\Windows\SYSTEM32\ntdll.dll+0x5af28)
ifx: error #10401: error running 'link for host dependence gathering'
make: *** [impi_intel_win_openmp] Error 1
Press any key to continue . . .

Somewhere, I found that I should use -fuse-ld=llvm-bin. This does not seem to be supported by ifx. When I compile FDS without -flto (or /Qipo), what performance impact does it have? Would you have a solution to this?

Overall, I find FDS compiled in ifx (even without /Qipo) way faster, but this may be caused by the additional flags I used during compilation.

marcosvanella commented 1 week ago

Hi, yes the -ipo flag will make the link phase take some time, in the order of minutes. It should give a speedup of 5 to 10% depending on the case and system.

How are you trying to compile the code? Are you using mpiifort -fc=ifx or mpiifx (ifx compiled mpi library)? This might be related to the windows version of OneAPI.

Using OneAPI 2024.1 in linux I was able to compile FDS with these two options:

1. mpiifort -m64 -fc=ifx -O2 -ipo -no-wrap-margin -fpp ... 2. mpiifx -m64 -fc=ifx -O2 -ipo -no-wrap-margin -fpp ...

       We are working on getting FDS to compile with ifx. There are still issues with ifx that need to be fixed upstream to be able to get our continuous integration migrated to it. Some of them are related to runtime checking debug flags, see below:

https://community.intel.com/t5/Intel-Fortran-Compiler/Errors-using-mpiifx-and-check-all-with-OneAPI-2024-1/td-p/1600662 Errors using mpiifx and -check all with OneAPI 2024.1https://community.intel.com/t5/Intel-Fortran-Compiler/Errors-using-mpiifx-and-check-all-with-OneAPI-2024-1/td-p/1600662 Hi, we've come across two errors compiling with mpiifx and -check all using the latest OneAPI 2024.1 Base and HPC toolkits. To reproduce these I have made a snippet code that essentially does a "Hello World" test with MPI calling some MPI routines. - First issue : comes trying to link the source to... community.intel.com

https://community.intel.com/t5/Intel-Fortran-Compiler/IFX-issue-when-building-check-all-or-check-uninit/td-p/1545825 [https://community.intel.com/t5/image/serverpage/image-id/49525i9D89CDCB7D371B64?v=v2&whitelist-exif-data=Orientation%2CResolution%2COriginalDefaultFinalSize%2CCopyright]https://community.intel.com/t5/Intel-Fortran-Compiler/IFX-issue-when-building-check-all-or-check-uninit/td-p/1545825 IFX issue when building -check all or -check uninithttps://community.intel.com/t5/Intel-Fortran-Compiler/IFX-issue-when-building-check-all-or-check-uninit/td-p/1545825 With the linux* version of ifx in the 2024.0.x release there is a known bug with the new LLVM Memory Sanitizer feature. This does not affect Windows users. A fix is coming in the 2024.1 Update Release. There are several workaround until that Update 1 release fixes the issue: Workaround: For now... community.intel.com

From: Patrik Král @.> Sent: Sunday, July 7, 2024 04:59 PM To: firemodels/fds @.> Cc: Subscribed @.***> Subject: [firemodels/fds] Intel Fortran Compiler Classic (ifort) deprecation by the end of 2024 and flag issue in ifx (Issue #13132)

Hello, As Intel Fortran Compiler Classic (ifort) is going to be deprecated by the end of 2024 (https://community.intel.com/t5/Blogs/Tech-Innovation/Tools/Deprecation-of-The-Intel-Fortran-Compiler-Classic-ifort/post/1541699), are you planning a transition to Intel Fortran Compiler (ifx)?

I managed to compile FDS in ifx; however, I encountered an issue with one of the compiler flags. /Qipo, calling -flto and -fuse-ld=lld, did not work in my case. It opened "lld-link.exe"; however, it ended up with the followng error message:

PLEASE submit a bug report to https://software.intel.com/en-us/support/priority-support and include the crash backtrace. Stack dump:

  1. Program arguments: C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link @C:\Users\Patrik\AppData\Local\Temp\19380122arg3 Exception Code: 0xC0000005

    0 0x00007ff6ee47493b (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x1fa493b)

    1 0x00007ff6ed5a216f (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x10d216f)

    2 0x00007ff6eceebe03 (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0xa1be03)

    3 0x00007ff6eceebfdc (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0xa1bfdc)

    4 0x00007ff6ee473570 (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x1fa3570)

    5 0x00007ff6ee472c82 (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x1fa2c82)

    6 0x00007ff6edfaa6ea (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x1ada6ea)

    7 0x00007ff6edfab3f1 (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x1adb3f1)

    8 0x00007ff6edfaafd2 (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x1adafd2)

    9 0x00007ff6ec56fa7d (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x9fa7d)

    10 0x00007ff6ec534484 (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x64484)

    11 0x00007ff6ec505dbb (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x35dbb)

    12 0x00007ff6ec4fbdd3 (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x2bdd3)

    13 0x00007ff6ec4f8885 (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x28885)

    14 0x00007ff6ec4d1294 (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x1294)

    15 0x00007ff6ec4d17c3 (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x17c3)

    16 0x00007ff6ed74dd08 (C:\PROGRA~2\Intel\oneAPI\compiler\latest\bin\compiler\lld-link.exe+0x127dd08)

    17 0x00007ffcd89c257d (C:\Windows\System32\KERNEL32.DLL+0x1257d)

    18 0x00007ffcd9d2af28 (C:\Windows\SYSTEM32\ntdll.dll+0x5af28)

    ifx: error #10401: error running 'link for host dependence gathering' make: *** [impi_intel_win_openmp] Error 1 Press any key to continue . . .

Somewhere, I found that I should use -fuse-ld=llvm-bin. This does not seem to be supported by ifx. When I compile FDS without -flto (or /Qipo), what performance impact does it have? Would you have a solution to this?

Overall, I find FDS compiled in ifx way faster, but this may be caused by the additional flags I used during compilation.

— Reply to this email directly, view it on GitHubhttps://github.com/firemodels/fds/issues/13132, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABY23VMECSO6YLJZ4JGSGJDZLGT23AVCNFSM6AAAAABKPVGMAGVHI2DSMVQWIX3LMV43ASLTON2WKOZSGM4TIMRRHE2TCOI. You are receiving this because you are subscribed to this thread.Message ID: @.***>

KingCZE commented 1 week ago

I find it a bit easier to compile it for Linux (will do it too), but I did this for Windows. Just changed the compiler from ifort to ifx. (As far as I know, you don't specify mpiifx for windows in the FDS build? I know oneAPI has mpiifx.bat though.)

There were a few more issues I needed to solve for the compilation with ifx to run:

  1. I needed to change this: GITINFO=-fpp $(GITINFO_BASE) -DBUILDDATE_PP=\""$(BUILD_DATE)\"" to this (the quotes and it cannot start with a digit): GITINFO=-fpp $(GITINFO_BASE) -DBUILDDATE_PP="\"d$(BUILD_DATE)\"" same for some other comment starting with a digit (I don't have my makefile with me now).

  2. I needed to set rigid links in the following line code (I don't know why ifx had a problem with these links) LFLAGSMKL_INTEL_OPENMP = "$(MKLLIBDIR)"\mkl_intel_lp64.lib "$(MKLLIBDIR)"\mkl_intel_thread.lib "$(MKLLIBDIR)"\mkl_core.lib "$(MKLLIBDIR)"\mkl_blacs_intelmpi_lp64.lib

mcgratta commented 1 week ago

Thanks for the information. To add to what Marcos said, we have been transitioning to ifx from ifort, but there have been several issues which have been reported to Intel. Not all have been resolved. A few are

  1. We cannot use mpiifx on Windows because there are blank spaces in the GIT compiler directives that Windows does not understand. I see you have noticed this kind of issue too. I reported to Intel, but they said that they are not going to fix the issue on Windows. So we have to use ifx instead of mpiifx and explicitly call out the various libraries. We do this already with ifort on Windows.
  2. To date, I have not been able to build FDS under linux using mpiifx because there is some issue with the MKL libraries. When I compile without the MKL libs, it works.
  3. There are various debugging "check" options that have not worked properly. There is a new version of oneAPI that we need to check to see if these have been resolved.
mcgratta commented 1 week ago

All this being said, we will look at what you have done to see if these issues can be resolved.

mcgratta commented 1 week ago

I just compiled FDS successfully on our new linux cluster with oneAPI 2024.1. I did not modify the makefile at all. I am now testing on Windows.

mcgratta commented 1 week ago

My compilation on Windows with ifx failed with the same errors posted at the top of this thread. Now I recall that this is the bug that I reported to Intel, and they acknowledged it as a bug.

@KingCZE can you explain what you mean here

I needed to set rigid links in the following line code (I don't know why ifx had a problem with these links)

LFLAGSMKL_INTEL_OPENMP = "$(MKLLIBDIR)"\mkl_intel_lp64.lib "$(MKLLIBDIR)"\mkl_intel_thread.lib "$(MKLLIBDIR)"\mkl_core.lib "$(MKLLIBDIR)"\mkl_blacs_intelmpi_lp64.lib