Installation does not work with icc and ifort

[ftherrien]

While we are at it, this is another issue we came accross while installing pylada on the new system. Pylada installs smoothly with gcc and gfortran, but when trying to install it using ifort and icc (i.e. they are the default compiler found by cmake when the intel modules are loaded) we get the following error:

/home/felixt/src/pylada-impi/ewald/erfc.cc(26): error: identifier "std_erfc" is undefined

      extern "C" double FC_GLOBAL_(std_erfc, STD_ERFC)( const double *const _in )
                                   ^
    /home/felixt/src/pylada-impi/ewald/erfc.cc(26): error: expected a ")"

      extern "C" double FC_GLOBAL_(std_erfc, STD_ERFC)( const double *const _in )
                                           ^
    /home/felixt/src/pylada-impi/ewald/erfc.cc(26): error: expected a ";"

      extern "C" double FC_GLOBAL_(std_erfc, STD_ERFC)( const double *const _in )
                                                      ^
    compilation aborted for /home/felixt/src/pylada-impi/ewald/erfc.cc (code 2)

This is because FC_GLOBAL (and FC_MODULE) is not defined in build/FCMangle.h as it should. This happens when Cmake is unable to interface C and Fortran. One way to check this is to add:

FortranCInterface_VERIFY()

in CMakeLists.txt. After doing so, we get the following error:

Call Stack (most recent call first):

     /home/felixt/.conda-envs/testenv/share/cmake-3.11/Modules/FortranCInterface/Verify/CMakeLists.txt:16 (FortranCInterface_HEADER)

    This warning is for project developers.  Use -Wno-dev to suppress it.

    -- Verifying Fortran/C Compiler Compatibility - Failed

    CMake Error at /home/felixt/.conda-envs/testenv/share/cmake-3.11/Modules/FortranCInterface.cmake:383 (message):

      The Fortran compiler:

        /nopt/intel/psxe2017u2/compilers_and_libraries_2017.5.239/linux/bin/intel64/ifort

      and the C compiler:

        /nopt/intel/psxe2017u2/compilers_and_libraries_2017.5.239/linux/bin/intel64/icc

      failed to compile a simple test project using both languages.

I am not sure how to proceed to solve this error... It might be independant of pylada...

[mdavezac]

I'll see if I can find a platform with intel. But it does look like an issue with cmake and ifort.

[TomKasel]

I have experienced the same issue. I went through and attempted to compile Pylada using different versions of Cmake (2.8, 3.0, 3.9, 3.11, and 3.12 specifically). Anything below 3.9 wasn't able to begin compiling, while everything 3.9 or above returns the same interface error. I found somewhere that this issue was well known in Cmake 3.9, but haven't been able to find anything on later version, but I can only assume the problem hasn't truly been fixed since 3.11 nor 3.12 were able to successfully compile. I'm interested in seeing if any progress has been made on solving this issue, or perhaps if more information is known.

[mdavezac]

no news yet I'm afraid.

[TomKasel]

Many websites claimed CMake version 3.8.2 would fix the problem, however I recently found this to not be true. Unlike @ftherrien, I was unable to compile Pylada-Light using gcc and gfortran as it resulted in the same "-Detecting Fortran/C Interface - Failed to compile" error. If I may ask, what versions of gcc, gfortran, and CMake resulted in a successful installation?

[ftherrien]

We used gcc 7.2.0 and cmake 3.11.1. Can you show us the full error message?

[TomKasel]

Here is the full output of an installation attempt. Also attached are CMakeError.log and CMakeOutput.log. It should be noted that I recently upgraded to macOS Mojave, c++/ifort 2019, and gcc/gfortran 8.1 in an attempt to solve this issue. I am rather novice at this so perhaps I missed something fundamental or simple, which would be welcome as fixing it should hopefully be trivial! Also this is run out of a conda environment to isolate the package. CMakeError.log CMakeOutput.log

pylada27) hms2:pylada-light-master hms2$ python setup.py install running install running build creating /Users/hms2/bin/pylada-light-master/build CMake: configuring with variables in /Users/hms2/bin/pylada-light-master/build/Variables.cmake /Users/hms2/anaconda2/envs/pylada27/bin/cmake -C/Users/hms2/bin/pylada-light-master/build/Variables.cmake /Users/hms2/bin/pylada-light-master loading initial cache file /Users/hms2/bin/pylada-light-master/build/Variables.cmake -- The C compiler identification is GNU 8.1.0 -- The CXX compiler identification is GNU 8.1.0 -- The Fortran compiler identification is GNU 8.1.0 -- Checking whether C compiler has -isysroot -- Checking whether C compiler has -isysroot - yes -- Checking whether C compiler supports OSX deployment target flag -- Checking whether C compiler supports OSX deployment target flag - yes -- Check for working C compiler: /usr/local/bin/gcc -- Check for working C compiler: /usr/local/bin/gcc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Checking whether CXX compiler has -isysroot -- Checking whether CXX compiler has -isysroot - yes -- Checking whether CXX compiler supports OSX deployment target flag -- Checking whether CXX compiler supports OSX deployment target flag - yes -- Check for working CXX compiler: /usr/local/bin/g++ -- Check for working CXX compiler: /usr/local/bin/g++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Checking whether Fortran compiler has -isysroot -- Checking whether Fortran compiler has -isysroot - yes -- Checking whether Fortran compiler supports OSX deployment target flag -- Checking whether Fortran compiler supports OSX deployment target flag - yes -- Check for working Fortran compiler: /usr/local/bin/gfortran -- Check for working Fortran compiler: /usr/local/bin/gfortran -- works -- Detecting Fortran compiler ABI info -- Detecting Fortran compiler ABI info - done -- Checking whether /usr/local/bin/gfortran supports Fortran 90 -- Checking whether /usr/local/bin/gfortran supports Fortran 90 -- yes -- [GreatCMakeCookOff] not found. Will attempt to clone it. -- Found Git: /Users/hms2/anaconda2/envs/pylada27/bin/git (found version "2.19.1") -- [GreatCMakeCookOff] downloaded to /Users/hms2/bin/pylada-light-master/build/external/src/GreatCMakeCookOff -- Detecting Fortran/C Interface -- Detecting Fortran/C Interface - Failed to compile -- Performing Test has_std_cpp11 -- Performing Test has_std_cpp11 - Success -- Performing Test has_std_cpp0x -- Performing Test has_std_cpp0x - Success -- Found Eigen3 -- Found PythonInterp: /Users/hms2/anaconda2/envs/pylada27/bin/python (found version "2.7.3") -- Found pytest: /Users/hms2/anaconda2/envs/pylada27/lib/python2.7/site-packages (found version "3.9.1") -- Python module nbconvert could not be found. (missing: nbconvert_LOCATION) -- Found nbformat: /Users/hms2/anaconda2/envs/pylada27/lib/python2.7/site-packages/nbformat (found version "4.4.0") -- Found pytest_bdd: /Users/hms2/anaconda2/envs/pylada27/lib/python2.7/site-packages/pytest_bdd (found version "3.0.0") -- Found mpi4py: /Users/hms2/anaconda2/envs/pylada27/lib/python2.7/site-packages/mpi4py (found version "2.0.0") -- Found IPython: /Users/hms2/anaconda2/envs/pylada27/lib/python2.7/site-packages/IPython (found version "5.8.0") -- Found numpy: /Users/hms2/anaconda2/envs/pylada27/lib/python2.7/site-packages/numpy (found version "1.15.3") -- Found quantities: /Users/hms2/anaconda2/envs/pylada27/lib/python2.7/site-packages/quantities (found version "0.12.2") -- Found f90nml: /Users/hms2/anaconda2/envs/pylada27/lib/python2.7/site-packages/f90nml-1.0.2-py2.7.egg/f90nml (found version "1.0.2") -- Found six: /Users/hms2/anaconda2/envs/pylada27/lib/python2.7/site-packages (found version "1.11.0") -- Found traitlets: /Users/hms2/anaconda2/envs/pylada27/lib/python2.7/site-packages/traitlets (found version "4.3.2") -- Found cython: /Users/hms2/anaconda2/envs/pylada27/lib/python2.7/site-packages (found version "0.29") -- Found NumpyLibrary: /Users/hms2/anaconda2/envs/pylada27/lib/python2.7/site-packages/numpy/core/include (found version "1.15.3") -- Python install path (PYTHON_PKG_DIR): /Users/hms2/anaconda2/envs/pylada27/lib/python2.7/site-packages -- Configuring done -- Generating done -- Build files have been written to: /Users/hms2/bin/pylada-light-master/build CMake: building in /Users/hms2/bin/pylada-light-master/build /Users/hms2/anaconda2/envs/pylada27/bin/cmake --build /Users/hms2/bin/pylada-light-master/build Scanning dependencies of target lookup_dependencies [ 0%] Built target lookup_dependencies Scanning dependencies of target pylada_error [ 0%] Built target pylada_error Scanning dependencies of target cutilities [ 2%] Building CXX object crystal/CMakeFiles/cutilities.dir/smith_normal_form.cc.o [ 5%] Building CXX object crystal/CMakeFiles/cutilities.dir/gruber.cc.o [ 8%] Building CXX object crystal/CMakeFiles/cutilities.dir/noopt.cc.o [ 11%] Linking CXX static library libcutilities.a [ 11%] Built target cutilities [ 14%] Generating c/c++ source cutilities.pyx with cython () [ 17%] Building CXX object crystal/CMakeFiles/pylada.crystal.cutilities-cython.dir/cython_cutilities.cc.o [ 20%] Linking CXX shared module ../../pkg_install/pylada/crystal/cutilities.so [ 20%] Built target pylada.crystal.cutilities-cython [ 22%] Generating c/c++ source _primitive.pyx with cython () [ 25%] Building CXX object crystal/CMakeFiles/pylada.crystal._primitive-cython.dir/cython__primitive.cc.o [ 28%] Linking CXX shared module ../../pkg_install/pylada/crystal/_primitive.so [ 28%] Built target pylada.crystal._primitive-cython [ 31%] Generating c/c++ source _map_sites.pyx with cython () [ 34%] Building CXX object crystal/CMakeFiles/pylada.crystal._map_sites-cython.dir/cythonmap_sites.cc.o [ 37%] Linking CXX shared module ../../pkg_install/pylada/crystal/_map_sites.so [ 37%] Built target pylada.crystal._map_sites-cython [ 40%] Generating c/c++ source _space_group.pyx with cython () [ 42%] Building CXX object crystal/CMakeFiles/pylada.crystal._space_group-cython.dir/cythonspace_group.cc.o [ 45%] Linking CXX shared module ../../pkg_install/pylada/crystal/_space_group.so [ 45%] Built target pylada.crystal._space_group-cython [ 48%] Generating c/c++ source _coordination_shells.pyx with cython () [ 51%] Building CXX object crystal/CMakeFiles/pylada.crystal._coordination_shells-cython.dir/cython__coordination_shells.cc.o [ 54%] Linking CXX shared module ../../pkg_install/pylada/crystal/_coordination_shells.so [ 54%] Built target pylada.crystal._coordination_shells-cython Scanning dependencies of target pylada.crystal [ 54%] Built target pylada.crystal Scanning dependencies of target third_order [ 57%] Building CXX object crystal/defects/CMakeFiles/third_order.dir/third_order.cc.o [ 60%] Linking CXX static library libthird_order.a [ 60%] Built target third_order [ 62%] Generating c/c++ source cutilities.pyx with cython () [ 65%] Building CXX object crystal/defects/CMakeFiles/pylada.crystal.defects.cutilities-cython.dir/cython_cutilities.cc.o [ 68%] Linking CXX shared module ../../../pkg_install/pylada/crystal/defects/cutilities.so [ 68%] Built target pylada.crystal.defects.cutilities-cython Scanning dependencies of target pylada.crystal.defects [ 68%] Built target pylada.crystal.defects Scanning dependencies of target pylada.crystal.tests [ 68%] Built target pylada.crystal.tests Scanning dependencies of target ewaldf90 [ 71%] Building Fortran object ewald/CMakeFiles/ewaldf90.dir/ep_com.f90.o [ 74%] Building CXX object ewald/CMakeFiles/ewaldf90.dir/erfc.cc.o /Users/hms2/bin/pylada-light-master/ewald/erfc.cc:26:30: warning: ‘FCGLOBAL’ initialized and declared ‘extern’ extern "C" double FCGLOBAL(std_erfc, STD_ERFC)( const double const _in ) ^~~~ /Users/hms2/bin/pylada-light-master/ewald/erfc.cc:26:30: error: ‘std_erfc’ was not declared in this scope /Users/hms2/bin/pylada-light-master/ewald/erfc.cc:26:30: note: suggested alternative: ‘erfc’ extern "C" double FCGLOBAL(std_erfc, STD_ERFC)( const double const _in ) ^~~~ erfc /Users/hms2/bin/pylada-light-master/ewald/erfc.cc:26:40: error: ‘STD_ERFC’ was not declared in this scope extern "C" double FCGLOBAL(std_erfc, STD_ERFC)( const double const _in ) ^~~~ /Users/hms2/bin/pylada-light-master/ewald/erfc.cc:26:40: note: suggested alternative: ‘SIG_ERR’ extern "C" double FCGLOBAL(std_erfc, STD_ERFC)( const double const _in ) ^~~~ SIG_ERR /Users/hms2/bin/pylada-light-master/ewald/erfc.cc:26:48: error: expression list treated as compound expression in initializer [-fpermissive] extern "C" double FCGLOBAL(std_erfc, STD_ERFC)( const double *const _in ) ^ make[2]: [ewald/CMakeFiles/ewaldf90.dir/erfc.cc.o] Error 1 make[1]: [ewald/CMakeFiles/ewaldf90.dir/all] Error 2 make: *** [all] Error 2 error: command '/Users/hms2/anaconda2/envs/pylada27/bin/cmake' failed with exit status 2

[ftherrien]

Cmake can't find the gcc libraries (lgcc_s) which seems to be related to Mojave. Other people are having similar issues: https://github.com/Homebrew/homebrew-core/issues/31483

These posts are particularly useful: https://github.com/neovim/neovim/issues/9050#issuecomment-424441062 https://github.com/neovim/neovim/issues/9050#issuecomment-429674155

Try this before installing: export MACOSX_DEPLOYMENT_TARGET=10.14

It seems to have worked for other people on Mojave.

[mdavezac]

As far as the fortran bindings are concerned, it should be possible to use fortran 2003's "bind(c)" and avoid FC_GLOBAL entirely. That should remove any dependency on cmake figuring out how to bind C and Fortran compilers.

[TomKasel]

@ftherrien That worked perfectly, thanks! I presumed I more than likely shot myself in the foot a bit when upgrading my OS.

[TomKasel]

@ftherrien I was able to install Pylada today stipulating ifort and icc using Python3.7. The issue I run into however, on all Python3 versions except 3.6.4, is a segmentation fault :11 when attempting to import Pylada. 3.6.4 itself is finicky about package versions, and any changes seems to result in an identical segmentation fault. Below are the errors resulting from segmentation fault and the initial installation lines stipulating which C, CXX and Fortran were used:

import faulthandler faulthandler.enable() import pylada Fatal Python error: Segmentation fault Current thread 0x000000011c8c25c0 (most recent call first): File "", line 219 in _call_with_frames_removed File "", line 1043 in create_module File "", line 583 in module_from_spec File "", line 670 in _load_unlocked File "", line 967 in _find_and_load_unlocked File "", line 983 in _find_and_load File "/Users/hms2/bin/pylada-light-master/pkg_install/pylada/crystal/init.py", line 32 in File "", line 219 in _call_with_frames_removed File "", line 728 in exec_module File "", line 677 in _load_unlocked File "", line 967 in _find_and_load_unlocked File "", line 983 in _find_and_load File "", line 219 in _call_with_frames_removed File "", line 1035 in _handle_fromlist File "/Users/hms2/bin/pylada-light-master/pkg_install/pylada/init.py", line 131 in File "", line 219 in _call_with_frames_removed File "", line 728 in exec_module File "", line 677 in _load_unlocked File "", line 967 in _find_and_load_unlocked File "", line 983 in _find_and_load File "", line 1 in Segmentation fault: 11

-- The C compiler identification is Intel 19.0.0.20180804 -- The CXX compiler identification is GNU 8.1.0 -- The Fortran compiler identification is Intel 19.0.0.20180804

[ftherrien]

It might be because your C and Fortran compilers are intel and your C++ compiler is GNU. Were you not able to install Pylada using GNU compilers? Assuming you installed all of the following, try:

export MACOSX_DEPLOYMENT_TARGET=10.14
export CC=gcc
export CCX=g++
export FC=gfortran

[TomKasel]

Originally I had stipulated CC and FC to be GNU based through addition to CMakelist.txt:

SET( CMAKE_C_COMPILER "gcc" ) SET( CMAKE_Fortran_COMPILER "gfortran" )

And CCX fell into line being GNU based. This however results in the same segmentation fault error regardless. So I'm fairly certain it is independent of the CCX compiler. However, I was just able to compile using intel version of CC, CCX, and FC. This however has raised an ImportError due to what appears to be a failure to load a DYLD.

import pylada Traceback (most recent call last): File "", line 1, in File "/pylada-light-master/pkg_install/pylada/init.py", line 131, in from . import error, crystal, physics, misc, tools, ewald, decorations, periodic_table, vasp, \ File "/pylada-light-master/pkg_install/pylada/crystal/init.py", line 32, in from ._space_group import space_group, cell_invariants ImportError: dlopen(/pylada-light-master/pkg_install/pylada/crystal/_space_group.so, 2): __dyld section not supported in /pylada-light-master/pkg_install/pylada/crystal/_space_group.so

[mdavezac]

@TomKasel , if you want to try the latest from the master branch, it doesn't require fortran name mangling anymore and should take care of the "FC_GLOBAL" issue with the intel suite. Nominally, it should be possible to use any set of c++, c, and fortran compilers, though it might be best to stick to one suite at a time, e.g. all intel, all gnu, or clang + gfortran.

[TomKasel]

@mdavezac I attempted building with the latest master branch, however, building now fails when specifying CXX as intel. I went through and tested the different combinations of C, CXX, and Fortran in CMakeList.txt, and all build fine regardless of what C and Fortran are specified as, but will always fail with CXX specified as intel. When it does compile, it returns the same segmentation error mentioned before. The error when compiling using CXX as intel:

[ 57%] Building CXX object crystal/defects/CMakeFiles/third_order.dir/third_order.cc.o /pylada-light-master/crystal/defects/third_order.cc(56): error #303: explicit type is missing ("int" assumed) auto const matrix = types::rMatrix3d::Map(_matrix).transpose(); ^ /pylada-light-master/crystal/defects/third_order.cc(56): error: no suitable conversion function from "const Eigen::Transpose<const Eigen::Map<const Eigen::Matrix<double, 3, 3, 0, 3, 3>, 0, Eigen::Stride<0, 0>>>" to "const int" exists auto const matrix = types::rMatrix3d::Map(_matrix).transpose(); ^ /pylada-light-master/crystal/defects/third_order.cc(57): error: no instance of overloaded function "pylada::third_order" matches the argument list argument types are: (const int, pylada::types::t_int) return third_order(matrix, _n); ^ /pylada-light-master/crystal/defects/third_order.cc(29): note: this candidate was rejected because at least one template argument could not be deduced template types::t_real third_order(Eigen::MatrixBase const &_matrix, types::t_int _n) { ^ /pylada-light-master/crystal/defects/third_order.cc(55): note: this candidate was rejected because arguments do not match types::t_real third_order(types::t_real const *_matrix, types::t_int _n) { ^ compilation aborted for /pylada-light-master/crystal/defects/third_order.cc (code 2) make[2]: [crystal/defects/CMakeFiles/third_order.dir/third_order.cc.o] Error 2 make[1]: [crystal/defects/CMakeFiles/third_order.dir/all] Error 2 make: *** [all] Error 2 error: command '/pylada/bin/cmake' failed with exit status 2

[mdavezac]

For intel, you will need the latest greatest cmake. Cmake only recently added c++11 detection for the compiler suite.

[ftherrien]

@TomKasel I am a little confused on what you were able to compile or not. Do you have at least one working build of pylada?

@mdavezac I just tried the lastest version and it solves the issue I originaly had. Should we close this issue?

[TomKasel]

The issues seemed to change depending on whether Ifort/c++ or gfortran/gcc was used. However, I did some poking around and attempted to compile something completely unrelated and discovered that my ifort, c++, gfortran, and gcc have issues, and have decided to get a fresh start and reinstall everything since "upgrading" to macOS Mojave. So once everything is put back in its place, I suspect several of my issues with be fixed, if not, its an issue with using packages/compilers that are "too up to date," in which case a system rollback will most likely solve the problem once and for all.

[mdavezac]

@TomKasel, we'll close this issue since compiling with intel seems to be solved. Please open another if you run into issues. For what it is worth, the latest error you posted seems to come from older cmake versions not passing c++11 flags to intel. You could try and run the whole process from scratch with CXXFLAGS=-std=c++11.