Fail to install MESA(version=r15140) in AMUSE

[adamparkosidis]

Describe the bug

After installing amuse as a development build (pip install -e . and python setup.py develop_build) I tried to install MESA (make mesa.code). Amuse supports two versions, i.e. MESA(version=r2208) and MESA(version=r15140), but the installation of the newer version fails. On the other hand, the installation of the older version is successful. I want to install the newer version.

To Reproduce

pip install -e .
python setup.py develop_build
make mesa.code

Expected behavior

Installing MESA(version=r15140)

Logs

build.log

Environment (please complete the following information):

• OS and version: Ubuntu 20.04.5 LTS
• Compiler: gcc 12.1.0

[rjfarmer]

Can you try:

rm -rf /home/adam/My_Environment/Projects/MSc_Thesis/amuse/src/amuse/community/mesa_r15140/src/

and try again. This way we do a full clean of the download folder.

[adamparkosidis]

I tried the command above and then proceed with:

make mesa_r15140.code DOWNLOAD_CODES=1

I am still not able to install this version, but now the error seems to be different.

build.log

I also tried to to start over:

make distclean
pip install -e .
python setup.py develop_build (<--- at this point the mesa_r2208 version is successfully build but not the version I want)
make mesa_r15140.code DOWNLOAD_CODES=1

and got the same error showing in the build.log

[rjfarmer]

/home/adam/My_Environment/Projects/MSc_Thesis/amuse/src/amuse/community/mesa_r15140/src/hdf5-1.12.0/fortran/testpar/hyper.f90:97: undefined reference to `h5pset_fapl_mpio_f_'

So the problem is happening when we try to build, link, and test HDF5.

@rieder do you have much experience with anaconda? It might be a issue with compilers/libraries getting mixed between the system and anaconda. As @adamparkosidis has a fortran compiler from:

Fortran Compiler: /usr/bin/mpif90

but seem it be pulling some things (like the linker) from anaconda:

Archiver: /home/adam/anaconda3/envs/amuse/bin/x86_64-conda-linux-gnu-ar

[rieder]

@rjfarmer yes I'm sure mixing system/anaconda libs is the problem here. It's why I can't recommend using AMUSE with Anaconda. We really need some help in making working instructions for AMUSE+Anaconda (see #235 and #525). For now, the solution is to deactivate the anaconda environment, then set up an AMUSE Python virtual environment and install AMUSE there.

[adamparkosidis]

So I followed the aforementioned instructions. I made a python environment (outside of anaconda) named amuse_env

python3 -m venv amuse_env
. amuse_env/bin/activate
pip install --upgrade pip
pip install numpy docutils mpi4py h5py wheel
pip install scipy astropy jupyter pandas seaborn matplotlib

next I cloned the amuse repository and made a developer's build

pip install -e .
python setup.py develop_build

A number of codes build. More specifically, 49 out of 55 codes, but once again the mesa_r15140 failed. In contrast mesa_r2208 succesfully build.

Here is the:

build.log

[rjfarmer]

Your PATH variable still has anaconda in it, as well as the mesasdk. You probably don't want either otherwise things can get confused

libtool: finish: PATH="/home/adam/amuse_env/bin:/home/adam/anaconda3/condabin:/home/adam/My_Environment/Projects/MSc_Courses/Structure_and_Evolution_of_Stars/mesasdk/bin:/home/adam/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/adam/My_Environment/Projects/MSc_Courses/Astrovaria/splash/bin:/sbin" ldconfig -n /home/adam/My_Environment/Projects/MSc_Thesis/amuse/src/amuse/community/mesa_r15140/src/mesa-r15140/lib

try:

export PATH="/home/adam/amuse_env/bin:/home/adam/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/adam/My_Environment/Projects/MSc_Courses/Astrovaria/splash/bin:/sbin"

before building amuse

[adamparkosidis]

I also have a MESA version installed in an anaconda environment, hence I have exported the paths in my bashrc file. I thought that they should not interact, but probably they do. I will remove the paths from the bashrc and I will install amuse again to see if that will solve the problem

[adamparkosidis]

After removing the paths in the bashrch, I installed amuse again and run the build command. Mesa_r15140 did not build, hence I tried the command above and then to build amuse again. It still fails to install the new version but the error is different. There are actually some instructions printed

build.log

I followed them but it still fails. Thats the build log after following the instructions

build.log

[rjfarmer]

Progress, those instructions should actually be ignored (i should get round to just deleting them). But it is telling us that one of the dependencies didn't get built properly (but you shouldn't be trying to build it yourself either).

Can you try deleting the mesa_r15140 src folder again?

[adamparkosidis]

I see, okey. I deleted the file rm -rf /home/adam/My_Environment/Projects/MSc_Thesis/amuse/src/amuse/community/mesa_r15140/src/ and tried to build the code again make mesa_r15140.code DOWNLOAD_CODES=1 It's being three hours now and the codes still tries to build. I know MESA is big but this does not seem normal

[rjfarmer]

What do the last few lines of the terminal output say? As long its changing (even if slowly) then its making progress

[adamparkosidis]

the last line in the terminal is 'inflating: crmath-1.2/crmath_r_sp.f90' and that the build.log until now

[adamparkosidis]

It took a while to complete, but again produced an error. The build failed and here is the log file

build.log

[rjfarmer]

The lines that matter:

/usr/bin/ld: ../../../lib/libhdf5.a(H5PLint.o): undefined reference to symbol 'dlclose@@GLIBC_2.2.5'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libdl.so: error adding symbols: DSO missing from command line

[adamparkosidis]

I am not exactly sure how to interpret this error. Is any module outdated?

[rjfarmer]

Looks like we just need to pass -ldl in the hdf5 link flags.

If you go to the file $AMUSE_DIR/src/amuse/community/mesa_r15140/Makefile and find the build_hdf5 target. Then replace the line:

./configure --enable-fortran --enable-fortran2003 --enable-parallel --prefix=$(MESA_DIR) --enable-shared=no;\

with

./configure --enable-fortran --enable-fortran2003 --enable-parallel --prefix=$(MESA_DIR) --enable-shared=no LIBS="-ldl";\

Then remove the src folder (again) and try rebuilding the dependencies.

[adamparkosidis]

I replaced the line and tried to build the dependencies, but the process failed again. That's the build.log

[adamparkosidis]

@rjfarmer I realized something that may be a problem. Even though I deactivate anaconda and made an installation in a python environment, I try to check the paths env | fgrep -i conda and I see that:

CONDA_EXE=/home/adam/anaconda3/bin/conda
_CE_CONDA=
CONDA_SHLVL=0
CONDA_PYTHON_EXE=/home/adam/anaconda3/bin/python
PATH=/home/adam/amuse_env/bin:/home/adam/anaconda3/condabin:/home/adam/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin

It seems that anaconda is somehow still in the PATH, could this explain the problem?

[rieder]

I don't think it is the problem. The condabin directory in your path only contains the conda executable. Also, the problem seems to be with libraries, not with executables.

[adamparkosidis]

I tried to install amuse in a clean python venv from scratch, but this time not the developers build.

pip install amuse

After the successful installation I tried to built mesa_r15140 by

cd src/amuse/community/mesa_r15140
make

but after ~15min it failed again. Now the error is different

make -C . build_crmath
make[2]: Entering directory '/home/adam/amuse/src/amuse/community/mesa_r15140'
/bin/sh: 1: makedepf90: not found
cd /home/adam/amuse/src/amuse/community/mesa_r15140/src/crmath-1.2;\
make;\

make[3]: Entering directory '/home/adam/amuse/src/amuse/community/mesa_r15140/src/crmath-1.2'
FC crmath.f90
FC crmath_r_sp.f90
FC crmath_c_sp.f90
FC crmath_r_dp.f90
FC crmath_c_dp.f90
crmath_c_dp.f90:25:39:

     log_x = CMPLX(log(abs(x)), atan2(x%im, x%re), DP)
                                       1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:39:45:

     log1p_x = CMPLX(log(abs(x1p)), atan2(x1p%im, x1p%re), DP)
                                             1
Error: Unexpected ‘%’ for nonderived-type variable ‘x1p’ at (1)
crmath_c_dp.f90:49:18:

     exp_x = exp(x%re)*CMPLX(cos(x%im), sin(x%im), DP)
                  1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:73:10:

     if (x%im == 0._DP) then
          1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:75:13:

        if (x%re < 0._DP) then
             1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:77:45:

           sqrt_x = CMPLX(0._DP, SIGN(SQRT(-x%re), x%im), DP)
                                             1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:79:11:

        else
           1
Error: Unexpected ELSE statement at (1)
crmath_c_dp.f90:81:32:

           sqrt_x = CMPLX(SQRT(x%re), SIGN(0._DP, x%im), DP)
                                1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:83:10:

        endif
          1
Error: Expecting END FUNCTION statement at (1)
crmath_c_dp.f90:85:14:

     elseif (x%re == 0._DP) then
              1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:87:29:

        r = SQRT(0.5_DP*ABS(x%im))
                             1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:89:35:

        sqrt_x = CMPLX(r, SIGN(r, x%im), DP)
                                   1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:91:8:

     else
        1
Error: Unexpected ELSE statement at (1)
crmath_c_dp.f90:93:19:

        d = hypot(x%re, x%im)
                   1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:95:13:

        if (x%re > 0._DP) then
             1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:96:39:

           r = SQRT(0.5_DP*d + 0.5_DP*x%re)
                                       1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:97:23:

           s = 0.5_DP*x%im/r
                       1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:98:11:

        else
           1
Error: Unexpected ELSE statement at (1)
crmath_c_dp.f90:99:39:

           s = SQRT(0.5_DP*d - 0.5_DP*x%re)
                                       1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:100:27:

           r = ABS(0.5_DP*x%im/s)
                           1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:101:10:

        endif
          1
Error: Expecting END FUNCTION statement at (1)
crmath_c_dp.f90:103:35:

        sqrt_x = CMPLX(r, SIGN(s, x%im), DP)
                                   1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:105:7:

     endif
       1
Error: Expecting END FUNCTION statement at (1)
crmath_c_dp.f90:115:20:

     abs_x = hypot(x%re, x%im)
                    1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:125:24:

     cos_x = CMPLX(cos(x%re)*cosh(x%im), -sin(x%re)*sinh(x%im), DP)
                        1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:135:24:

     sin_x = CMPLX(sin(x%re)*cosh(x%im), cos(x%re)*sinh(x%im), DP)
                        1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:148:15:

     rt = tan(x%re)
               1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:149:16:

     it = tanh(x%im)
                1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:191:26:

     cosh_x = CMPLX(cosh(x%re)*cos(x%im), sinh(x%re)*sin(x%im), DP)
                          1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:201:26:

     sinh_x = CMPLX(sinh(x%re)*cos(x%im), cosh(x%re)*sin(x%im), DP)
                          1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:214:16:

     rt = tanh(x%re)
                1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
crmath_c_dp.f90:215:15:

     it = tan(x%im)
               1
Error: Unexpected ‘%’ for nonderived-type variable ‘x’ at (1)
Makefile:14: recipe for target 'crmath_c_dp.o' failed
make[3]: *** [crmath_c_dp.o] Error 1
make[3]: Leaving directory '/home/adam/amuse/src/amuse/community/mesa_r15140/src/crmath-1.2'
Makefile:82: recipe for target 'build_crmath' failed
make[2]: *** [build_crmath] Error 2
make[2]: Leaving directory '/home/adam/amuse/src/amuse/community/mesa_r15140'
Makefile:50: recipe for target 'download' failed
make[1]: *** [download] Error 2
make[1]: Leaving directory '/home/adam/amuse/src/amuse/community/mesa_r15140'
Makefile:47: recipe for target '/install' failed
make: *** [/install] Error 2

is that helpful?

[rieder]

That method will not work - the pip version of AMUSE does not contain MESA, if you install it Python will not be able to find the 'developer' AMUSE in which you're attempting to build MESA. As for the error, I'm not sure... I think /bin/sh: 1: makedepf90: not found may be the problem here. @rjfarmer?

[adamparkosidis]

So I want to ask: 1) Is it only via development build possible to use MESA within amuse? 2) Is there anyone that is able to use MESA version = 15140 within amuse?

[rieder]

1) yes, that's the only way at the moment 2) yes, it works for me at least...

[rieder]

One more thing you could try - and it's very much a workaround - is to install a virtual ubuntu machine, in which you only install AMUSE. E.g. you could do so with https://multipass.run. This is not ideal (though it can work quite well), but it is tested and working.

[adamparkosidis]

I see maybe I can give it a try, but before that could you provide me the pip list of your environment in which MESA version = 15140 works? I will try to install the developer build once again in a clean venv.

[rieder]

I think the problem is not in the python environment, but rather somewhere in the prerequisites. Could you attach the config.mk file from your AMUSE root directory here?

[adamparkosidis]

Is this the one you refer to config.txt ?

[rieder]

Yes. I can't see any obvious problems there though. Since this is proving difficult to solve, could you try installing AMUSE in a virtual machine with Multipass? This may not be the ideal solution, but it's one that will almost certainly work. I can help out with this if needed.

[rieder]

Hi @adamparkosidis, can you confirm if the last change fixes your issue?

[adamparkosidis]

Hi all, yes indeed. mesa_r15140 successfully build after changing the order of the libraries

[rieder]

Great! Thanks to Martin Heemskerk for finding the issue.