Unable to compile GAMESS with Intel Compilers, fails for ddi_fortran.c

[shoubhikraj]

install.info.txt

• [x] Summary of the issue and expected results.

I tried to compile the GAMESS binary with Intel Compilers on Windows natively. I know that the precompiled executable is available, but it is not compiled with openMP, so RI-CCSD does not work. Besides, the Intel MKL is quite faster on Intel chips than PGI-BLAS, which is what the precompiled binary is built with.

I followed the instructions in gamess/machines/readme.microsoft.

I ran ./config then configured it for Windows with Intel compilers. Then I ran cd ddi; ./compddi. Here the compilation crashed:

Compiling FORTRAN wrappers for DDI
icl -DLINUX -DWINDOWS -DWINDOWS64 -DWINTEL -DOLDDDITIMER -Od -I./include -IG:\MPISDK\MPI\Include\ -DDDI_MPI -DMAX_SMP_PROCS=32 -DMAX_NODES=1024 -DINT_SIZE=__int64 -D_UNDERSCORES=0 -DF77_UPPERCASE -Dgetarg_=GETARG -Diargc_=IARGC -o ddi_fortran.o -c ddi_fortran.c
Intel(R) C++ Intel(R) 64 Compiler Classic for applications running on Intel(R) 64, Version 2021.1 Build 20201112_000000
Copyright (C) 1985-2020 Intel Corporation.  All rights reserved.

ddi_fortran.c
G:\gamess\ddi\src\mysystem.h(157): warning #47: incompatible redefinition of macro "max" (declared at line 1289 of "C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\stdlib.h")
   # define max(a,b) ((a) > (b) ? (a) : (b))
            ^

G:\gamess\ddi\src\mysystem.h(158): warning #47: incompatible redefinition of macro "min" (declared at line 1290 of "C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\stdlib.h")
   # define min(a,b) ((a) < (b) ? (a) : (b))
            ^

G:\gamess\ddi\src\mysystem.h(199): warning #47: incompatible redefinition of macro "ELOOP" (declared at line 99 of "C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\errno.h")
   # define ELOOP          31
            ^

G:\gamess\ddi\src\mysystem.h(207): warning #47: incompatible redefinition of macro "ETXTBSY" (declared at line 124 of "C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt\errno.h")
   # define ETXTBSY        16
            ^

ddi_fortran.c(378): warning #810: conversion from "void (*)(int_f77={__int64} *, int_f77={__int64} *, int_f77={__int64} *, int_f77={__int64} *, int_f77={__int64} *, void *)" to "int" may lose significant bits
        Scattered.oper   = DDI_ACC;
                         ^

ddi_fortran.c(388): error: expected an identifier
     void F77_GetAcc(int_f77 *handle, int_f77 *ilo, int_f77 *ihi,
          ^

compilation aborted for ddi_fortran.c (code 2)
unset echo
Error compiling: ddi_fortran.o

DDI compilation did not finish correctly, please fix, and try again

rm: No match.
rm: No match.
rm: No match.
Mon, Mar  8, 2021 12:21:56 PM

• [x] Description of the run environment:

  • operating system : Windows 10

  • compiler : Intel C++ compiler classic (icl.exe), Intel Fortran compiler classic (ifort.exe) provided by Intel oneAPI v2021.1

  • math library : Intel MKL (v2021.1)

  • communication library : MS-MPI

• A copy of your install.info file : attached

[saromleang]

Windows support trails behind that of Linux. As of right now, Windows support PGI 19.4 and Intel 2011.

So in order to compile GAMESS on Windows with the latest Intel OneAPI compiler you will need to put in some effort on your own.

[saromleang]

I will also add that we have pre-compiled Windows binaries available (unified binaries built with PGI 19.4).

[shoubhikraj]

@saromleang

I was able to make the compilation work by making several changes.

First of all, the compilation of ddi/src/ddi_fortran.c runs into an error due to namespace collision of the DDI operations and fortran method names. I can see from the ddi/src/ddi.h file that several other operations like DDI_ACC are already protected by an _OP suffix. The DDI_GETACC operation has to be also protected by this method (I am not sure why this protection is done with an #if defined WINTEL because the protected operation names should work on other compilers too, while the original names don't work on windows with ifort).

This means the operation name has to be changed (so that on Windows with ifort, the name DDI_GETACC is replaced with DDI_GETACC_OP) in other files too, in ddi/src/ddi_getacc.c (line 78), ddi_server.c (line 212) and ddi/src/ds_thread.c (line 287). These also have to be made conditional with #if defined WINTEL, if the ddi.h file uses that.

In ddi/src/ddi_fortran.c (line 378) Scattered.oper = DDI_ACC;. This line has to be changed so that when #if defined WINTEL the line is Scattered.oper = DDI_ACC_OP (similar to the previous changes).

This allows the ddi compilation to run successfully.

There are many other issues in the compile scripts, which has to be fixed for the newer version of the compilers:

1) In compall, comp and lked scripts, all instances of the string "cygwin" has to be replaced with "cygwin64". This is because the newer versions of cygwin's default installation location is C:\cygwin64\ on 64-bit Windows machines. The machines/readme.microsoft file mentions that the gamess source files have to be in C:\gamess. But the scripts assume that the source files are in C:\cygwin\gamess, so the readme file has to be updated.

2) In lked script, after line 768, set MATHLIBS="" this extra line has to be added, otherwise the final linking command fails as $MATHLIBS isn't defined. In line 778, change to set EXTRA_LINK_FLAGS="". The Intel Fortran compiler can use its own linker (xilink.exe) which finds the msvc linker (link.exe) automatically, so there is no need to point to the visual studio directory.

3) In lked, line 1069 change it to set MSG_LIBRARIES="c:/cygwin64$GMS_BUILD_DIR/ddi/libddi.a $MPILIBS $EXTRA_MPI_LIB_FLAGS"

4) The comp script prevents compilation of modules with openMP on windows, because that option is not included in the script. I have modified this so that now those modules (and other modules) can be compiled with openMP. After line 2766 in comp, the following has to be added

         if ($GMS_OPENMP == true)  then
             set OPT="$OPT -Qopenmp"
            #
             switch ($MODULE)
                 case omp*:
                 case riccutils:
                 case riccints:
                 case riccdiag:
                 case riccrhf:
                 case rimp2grd:
                 case rimp2omp:
                 case libxc:
                    breaksw
                 default:
                    $GMS_BUILD_DIR/tools/addomp.sh $MODULE_F
                    breaksw
              endsw
         endif

4) In comp, line 2787 the string -Qvec-report0 has to be replaced with -Qopt-report:0. The -Qvec-report argument is no longer supported by Intel fortran. Then, line 2785 has to be changed to set EXTRAOPT='-fpp', because the Fortran preprocessor is necessary for the openMP enabled source codes.

5) After compilation, the stacksize of the executable has to be increased (at least to 10MB), with editbin /stack:10000000 gamess.00.exe (using the editbin utility of the visual studio). This is necessary if the openMP threaded parts of the program, like RICC has to be used.

After all of this, GAMESS should compile successfully without any problem with the recent Intel Compilers. I have not been able to run all of the tests, but I have ran a few tests from rhf, uhf, rimp2grd-mpiomp and ricc-mpiomp folders. All of the tests so far have ran succesfully. (Note: Intel's openMP library libiomp5md.dll has to be in the same folder as the gamess executable to run the binary. For windows x64 and Intel oneAPI kit, the dll file is located in C:\Program Files (x86)\Intel\oneAPI\compiler\latest\windows\redist\intel64_win\compiler\ )

As far as I can tell, the changes I have made in the files should not interfere with compilation with any other compiler and/or any other platform. I don't have access to the main gamess repository, but I hope that one of the developers would be able to fix these issues in the main source code. I understand that most users of computational code users use linux, but there are a lot of windows users too, and if the code can run on windows, it should.

A copy of the modified versions of the files are here (.txt extension added because github does not support the normal extensions) : compall.txt comp.txt lked.txt ddi_fortran.c.txt ddi.h.txt ddi_getacc.c.txt ddi_server.c.txt ds_thread.c.txt

[saromleang]

Thank you very much for this info and the updates for Intel support on Windows!

[saromleang]

Could you please provide your full name and affiliation so that I can credit you for your effort in the GAMESS release notes?

[shoubhikraj]

@saromleang

Thanks! My name is Shoubhik R Maiti and I am a student in the University of Sheffield.

Another thing, I have also managed to make the libxc interface of GAMESS work on windows with Intel compilers. If I write it here, would that be helpful?

[saromleang]

@ShoubhikRaj yes! that would be very helpful. Thank you.