search

sandialabs / CACTUS

CACTUS (Code for Axial and Cross-flow TUrbine Simulation) is a turbine performance simulation code, based on a free wake vortex method, to study wind turbines and marine hydrokinetic (MHK) devices.
BSD 3-Clause "New" or "Revised" License
20 stars 13 forks source link

compilation with intel compiler on windows #54

Open rdamiani opened 2 years ago

rdamiani commented 2 years ago

Hello,

I know that the windows tips recommend using MinGW, but I was wondering whether there is any way to compile in windows with Intel Compiler (Intel OneAPI 2021.5.0.20211109). Cmake seems to run but gives some warnings including:

-- Could NOT find BLAS (missing: BLAS_LIBRARIES) -- Could NOT find LAPACK (missing: LAPACK_LIBRARIES) Reason given by package: LAPACK could not be found because dependency BLAS could not be found.

Then, trying to build from Visual Studio fails with the following errors:

Error fatal error LNK1120: 1 unresolved externals
Error error LNK2019: unresolved external symbol DGESV referenced in function WALLSYSTEM_mp_INVERT_INFLUENCE_MATRIX wallsystem.obj
Error error LNK2001: unresolved external symbol DGESV WSolnSetup.obj

to note that there are several warnings related to

Warning command line warning #10006: ignoring unknown option '/mkl' ifort

I then tried to include the MKL during compilation, and even though I get an executable, it does not run as expected. Issuing cactus.exe from the command prompt does not produce any result at all. I doubt the .exe is meaningful at all. Any help would be much appreciated.

whophil commented 2 years ago

Hi @rdamiani - just tried, and while it doesn't compile without warnings, it does work. On my machine, CMake is able to find LAPACK. Do you have MKL installed in your environment? It is an optional package of oneAPI's Base Kit.

(base) C:\Users\Remote\cactus\build>cmake ..
-- Building for: Visual Studio 16 2019
-- Selecting Windows SDK version 10.0.16299.0 to target Windows 10.0.19042.
-- The Fortran compiler identification is Intel 2021.5.0.20211109
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Determine Intel Fortran Compiler Implicit Link Path
-- Determine Intel Fortran Compiler Implicit Link Path - done
-- Check for working Fortran compiler: C:/Program Files (x86)/Intel/oneAPI/compiler/2022.0.3/windows/bin/intel64/ifort.exe - skipped
-- The C compiler identification is MSVC 19.29.30139.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Found Threads: TRUE
-- Looking for Fortran sgemm
-- Looking for Fortran sgemm - found
-- Found BLAS: C:/Program Files (x86)/Intel/oneAPI/mkl/latest/lib/intel64/mkl_intel_lp64_dll.lib;C:/Program Files (x86)/Intel/oneAPI/compiler/latest/windows/compiler/lib/ia32_win/libiomp5md.lib;C:/Program Files (x86)/Intel/oneAPI/mkl/latest/lib/intel64/mkl_intel_thread_dll.lib;C:/Program Files (x86)/Intel/oneAPI/mkl/latest/lib/intel64/mkl_core_dll.lib
-- Looking for Fortran cheev
-- Looking for Fortran cheev - found
-- Found LAPACK: C:/Program Files (x86)/Intel/oneAPI/mkl/latest/lib/intel64/mkl_intel_lp64_dll.lib;C:/Program Files (x86)/Intel/oneAPI/compiler/latest/windows/compiler/lib/ia32_win/libiomp5md.lib;C:/Program Files (x86)/Intel/oneAPI/mkl/latest/lib/intel64/mkl_intel_thread_dll.lib;C:/Program Files (x86)/Intel/oneAPI/mkl/latest/lib/intel64/mkl_core_dll.lib
-- Found OpenMP_C: -openmp (found version "2.0")
-- Found OpenMP_Fortran: -Qopenmp (found version "5.0")
-- Found OpenMP: TRUE (found version "2.0")
-- Configuring done
-- Generating done
-- Build files have been written to: C:/Users/Remote/cactus/build

Even with this working, the Windows + Intel build path needs some work. I get a lot of warnings about invalid flags. I also had to set -DOPENMP=OFF. Finally, it appears the resulting executable requires MKL runtimes. Perhaps it's better to link statically.

rdamiani commented 2 years ago

Hi Phil,

Thank you for your time, I have double checked and have got the MKL lib in the same locations as yours, yet cmake cannot find them, it also misses the Looking for Fortran sgemm checks, unsure why:

PS [...]\CACTUS\build> cmake .. -- Building for: Visual Studio 16 2019 -- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.22000. -- The Fortran compiler identification is Intel 2021.5.0.20211109 -- Detecting Fortran compiler ABI info -- Detecting Fortran compiler ABI info - done -- Determine Intel Fortran Compiler Implicit Link Path -- Determine Intel Fortran Compiler Implicit Link Path - done -- Check for working Fortran compiler: C:/Program Files (x86)/Intel/oneAPI/compiler/2022.0.3/windows/bin/intel64/ifort.exe - skipped -- Checking whether C:/Program Files (x86)/Intel/oneAPI/compiler/2022.0.3/windows/bin/intel64/ifort.exe supports Fortran 90 -- Checking whether C:/Program Files (x86)/Intel/oneAPI/compiler/2022.0.3/windows/bin/intel64/ifort.exe supports Fortran 90 - yes -- The C compiler identification is MSVC 19.29.30141.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: D:/Program Files/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped -- Detecting C compile features -- Detecting C compile features - done -- Looking for pthread.h -- Looking for pthread.h - not found -- Found Threads: TRUE -- Could NOT find BLAS (missing: BLAS_LIBRARIES) -- Could NOT find LAPACK (missing: LAPACK_LIBRARIES) Reason given by package: LAPACK could not be found because dependency BLAS could not be found.

-- Found OpenMP_C: -openmp (found version "2.0") -- Found OpenMP_Fortran: -Qopenmp (found version "5.0") -- Found OpenMP: TRUE (found version "2.0") -- Configuring done -- Generating done -- Build files have been written to: [...]/CACTUS/build``

I tried with and w/o a conda activated environment, same results. Any idea ? Is there something I need to set manually for the path? Thank you!

whophil commented 2 years ago

The conda environment should be inconsequential to this (but that's how I've installed CMake). Are you using the Intel oneAPI command prompt, the Visual Studio command prompt, or something else?

rdamiani commented 2 years ago

I have used the Windows Terminal, and both PowerShell and Command Prompt. Same results there.

whophil commented 2 years ago

I only have experience using the oneAPI command prompt, as that activates the required environment automatically. The same should be true if you are using oneAPI's setvars.bat in whatever terminal you are using. I think the important one is mkl - does your build environment have that?

   Initializing Visual Studio command-line environment...
   Visual Studio version 16.11.9 environment configured.
   "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\"
   Visual Studio command-line environment initialized for: 'x86'
:  compiler -- latest
:  debugger -- latest
:  dev-utilities -- latest
:  mkl -- latest
:  mpi -- latest
:  tbb -- latest
:: oneAPI environment initialized ::
rdamiani commented 2 years ago

Hi, thank you for checking some more into this.

I have tried launching the oneAPI command prompt, and I get what's below, which then should confirm that things are installed as expected. I then wiped out the folder build and tried rebuilding with cmake .. but with the same issues of blas not found. A bit lost.

I have then used WSL Ubuntu to compile CACTUS on another machine, and that went through ok. Yet I really would like to get the intel compiler and cmake to behave properly on my machine.

Not sure this is of interest here and to you, but, in pyCactus, had to port some print statements to python 3.10, and then had an issue with line 203 of CactusField.py that I changed to X = np.float32(np.reshape(x.to_numpy(), [nz, ny, nx])) (x was a pandas DF and did not get reshaped otherwise). Thank you!

:: initializing oneAPI environment... Initializing Visual Studio command-line environment... Visual Studio version 16.11.11 environment configured. "D:\Program Files\Microsoft Visual Studio\2019\Community\" Visual Studio command-line environment initialized for: 'x86' : advisor -- latest : compiler -- latest : dal -- latest : debugger -- latest : dev-utilities -- latest : dnnl -- latest : dpcpp-ct -- latest : dpl -- latest : inspector -- latest : intelpython -- latest : ipp -- latest : ippcp -- latest : itac -- latest : mkl -- latest : mpi -- latest : tbb -- latest : vpl -- latest : vtune -- latest :: oneAPI environment initialized ::

rdamiani commented 1 year ago

Dear Phil, I had managed to intel-compile a successful executable, but I had to reinstall windows recently due to hardware failure. I have now Microsoft Visual Studio Community 2022 Version 17.4.2 with Intel Libraries for oneAPI Package ID: w_oneAPI_2022.2.1.14974 I can compile without errors, but at runtime I get: forrtl: severe (170): Program Exception - stack overflow Image PC Routine Line Source cactus.exe 00007FF6964173D7 Unknown Unknown Unknown cactus.exe 00007FF6963CD3AF Unknown Unknown Unknown cactus.exe 00007FF6963F57AF Unknown Unknown Unknown cactus.exe 00007FF6964114AE Unknown Unknown Unknown cactus.exe 00007FF696417644 Unknown Unknown Unknown KERNEL32.DLL 00007FFE67CB26BD Unknown Unknown Unknown ntdll.dll 00007FFE686ADFB8 Unknown Unknown Unknown

It fails right at the beginning at CALL input(ErrFlag) (line 157 of Cactus.f90). I assume I need to set some settings for the compiler/linker but I am not sure what I need to do. Do you have any suggestions? Thank you!