search

amusecode / amuse

Astrophysical Multipurpose Software Environment. This is the main repository for AMUSE
http://www.amusecode.org
Apache License 2.0
158 stars 99 forks source link

pip installing issues #711

Closed broulston closed 2 years ago

broulston commented 3 years ago

I've been trying to install amuse on my mac for a while now and always seem to have problems that I havn't been able to figure out. I believe I have all the prerequisites installed, and pip install amuse-framework installed fine with no warning or issues at all.

Now when I try pip install amuse it builds for a while and then reports with a large error message (I can paste the full error message but didn't want to spam).

I ran amusifier --get-amuse-configmk and this is the output if it helps at all.

I'm using a conda environment that has my python installation and most packages installed that way.

Does anyone of any ideas what might be causing the install issues or experienced similar problems?

# amuse configuration generated by ./configure from config.mk.in

PYTHON=/Users/benjaminroulston/anaconda3/envs/amuse-env/bin/python3.8
CODE_GENERATOR=amusifier

#
# PYTHON embedding
#
PYTHON_DEV=no
CYTHON=/Users/benjaminroulston/anaconda3/bin/cython
PYTHONCONFIG=/usr/bin/python-config
PYTHONDEV_CFLAGS=
PYTHONDEV_LDFLAGS=

#
# Compilers, for compiling without MPI
# these compilers must be the same or compatible with
# the mpi compilers
#
CXX=x86_64-apple-darwin13.4.0-clang++
CC=gcc
FC=x86_64-apple-darwin13.4.0-gfortran
GFORTRAN_VERSION=
IFORT_VERSION=
FC_ISO_C_AVAILABLE=no

#
# Default flags, append to these in the makefiles
#
CXXFLAGS=-march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -std=c++14 -fmessage-length=0 -isystem /Users/benjaminroulston/anaconda3/envs/amuse-env/include -fPIC
CFLAGS=-march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/benjaminroulston/anaconda3/envs/amuse-env/include -fPIC
FCFLAGS= -fPIC
LDFLAGS=-Wl,-pie -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs -Wl,-rpath,/Users/benjaminroulston/anaconda3/envs/amuse-env/lib -L/Users/benjaminroulston/anaconda3/envs/amuse-env/lib

#
# MPI Compilers
#
MPI_ENABLED=yes
MPICXX=mpicxx
MPICC=mpicc
MPIFC=mpif90
MPIEXEC=/Users/benjaminroulston/anaconda3/envs/amuse-env/bin/mpiexec

MPI_CFLAGS=-I/Users/benjaminroulston/anaconda3/envs/amuse-env/include
MPI_CXXFLAGS=-I/Users/benjaminroulston/anaconda3/envs/amuse-env/include
MPI_FCFLAGS=-Wl,-flat_namespace -Wl,-commons,use_dylibs -I/Users/benjaminroulston/anaconda3/envs/amuse-env/include -I/Users/benjaminroulston/anaconda3/envs/amuse-env/lib
MPI_CLIBS=-L/Users/benjaminroulston/anaconda3/envs/amuse-env/lib -Wl,-rpath,/Users/benjaminroulston/anaconda3/envs/amuse-env/lib -L/Users/benjaminroulston/anaconda3/envs/amuse-env/lib -lmpi
MPI_CXXLIBS=-L/Users/benjaminroulston/anaconda3/envs/amuse-env/lib -Wl,-rpath,/Users/benjaminroulston/anaconda3/envs/amuse-env/lib -L/Users/benjaminroulston/anaconda3/envs/amuse-env/lib -lmpi
MPI_FCLIBS=-Wl,-commons,use_dylibs -I/Users/benjaminroulston/anaconda3/envs/amuse-env/include -I/Users/benjaminroulston/anaconda3/envs/amuse-env/lib -L/Users/benjaminroulston/anaconda3/envs/amuse-env/lib -Wl,-rpath,/Users/benjaminroulston/anaconda3/envs/amuse-env/lib -L/Users/benjaminroulston/anaconda3/envs/amuse-env/lib -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi

#
# Java
#
JAVA_ENABLED=yes
JAVA=/usr/bin/java
JAVAC=/usr/bin/javac
JAR=/usr/bin/jar
JAVA_VERSION=9

#
# OpenMP
#

OPENMP_ENABLED=yes
OPENMP_FCFLAGS=
OPENMP_CFLAGS=-fopenmp

#
# Libraries: FFTW, GSL, HDF5, CUDA, SAPPORO
#

FOUND_FFTW=yes
FFTW_FLAGS=-I/usr/local/Cellar/fftw/3.3.8_2/include
FFTW_LIBS=-L/usr/local/Cellar/fftw/3.3.8_2/lib -lfftw3 -lfftw3_threads

FOUND_GSL=yes
GSL_FLAGS=-I/usr/local/include
GSL_LIBS=-L/usr/local/lib -lgsl -lgslcblas -lm

FOUND_GMP=yes
GMP_FLAGS=
GMP_LIBS=-L/usr/local/Cellar/gmp/6.2.1/lib -lgmp

FOUND_MPFR=yes
MPFR_FLAGS=
MPFR_LIBS=-L/usr/local/Cellar/mpfr/4.1.0/lib -lmpfr -lgmp

HDF5_FLAGS=-I/Users/benjaminroulston/anaconda3/envs/amuse-env/include
HDF5_LIBS= -L/Users/benjaminroulston/anaconda3/envs/amuse-env/lib -lhdf5_hl -lhdf5  -lcrypto -lcurl -lpthread -lz -ldl -lm

NETCDF_PREFIX=.
NETCDF_FLAGS=
NETCDF_LIBS=
NETCDFF_PREFIX=.
NETCDFF_FLAGS=
NETCDFF_LIBS=

CUDA_ENABLED=no
NVCC=
NVCC_FLAGS=
CUDA_TK=/NOCUDACONFIGURED
CUDA_LIBS=-L/NOCUDACONFIGURED cuda cudart

FOUND_CL=no
CL_LIBS=
CL_FLAGS=

SAPPORO_VERSION=light
SAPPORO_LIBS=-L${AMUSE_DIR}/lib/sapporo_light -lsapporo 

FS_FLAGS=
FS_LIBS=

SC_FLAGS=-I$(AMUSE_DIR)/lib/stopcond
SC_CLIBS=-L$(AMUSE_DIR)/lib/stopcond -lstopcond
SC_FCLIBS=-L$(AMUSE_DIR)/lib/stopcond -lstopcond
SC_MPI_CLIBS=-L$(AMUSE_DIR)/lib/stopcond -lstopcondmpi
SC_MPI_FCLIBS=-L$(AMUSE_DIR)/lib/stopcond -lstopcondmpi

export PYTHON CXX CC FC CXXFLAGS CFLAGS FCFLAGS MPICXX MPICC MPIFC LDFLAGS
export JAVA JAVAC JAVAH JAR JAVA_FLAGS
export FS_FLAGS FS_LIBS SC_FLAGS SC_CLIBS SC_FCLIBS SC_MPI_CLIBS SC_MPI_FCLIBS
export FFTW_FLAGS FFTW_LIBS
export GSL_FLAGS GSL_LIBS
export HDF5_FLAGS HDF5_LIBS
export OPENMP_FCFLAGS OPENMP_CFLAGS
export NVCC CUDA_TK CUDA_LIBS CUDA_ENABLED
export SAPPORO_VERSION SAPPORO_LIBS
export MPICFLAGS MPICCFLAGS MPIFCFLAGS
export IFORT_VERSION
export NVCC_FLAGS
export PYTHONDEV_LIBS PYTHONDEV_CFLAGS PYTHONDEV_LIBS
export FC_ISO_C_ENABLED

ifdef AMUSE_DIR
export AMUSE_DIR
endif
broulston commented 3 years ago

So I switched from using pip and used the source from GitHub. I did ./configure resulting in the output below

(amuse-env) benjaminroulston:./configure
checking for x86_64-apple-darwin13.4.0-gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether we are using the GNU C++ compiler... yes
checking whether x86_64-apple-darwin13.4.0-clang++ accepts -g... yes
checking for x86_64-apple-darwin13.4.0-gfortran... no
checking for x86_64-apple-darwin13.4.0-g95... no
checking for x86_64-apple-darwin13.4.0-xlf95... no
checking for x86_64-apple-darwin13.4.0-f95... no
checking for x86_64-apple-darwin13.4.0-fort... no
checking for x86_64-apple-darwin13.4.0-ifort... no
checking for x86_64-apple-darwin13.4.0-ifc... no
checking for x86_64-apple-darwin13.4.0-efc... no
checking for x86_64-apple-darwin13.4.0-pgfortran... no
checking for x86_64-apple-darwin13.4.0-pgf95... no
checking for x86_64-apple-darwin13.4.0-lf95... no
checking for x86_64-apple-darwin13.4.0-ftn... no
checking for x86_64-apple-darwin13.4.0-nagfor... no
checking for gfortran... gfortran
checking whether we are using the GNU Fortran compiler... yes
checking whether gfortran accepts -g... yes
checking whether the Fortran 90 compiler (gfortran -g -O2 -Wl,-pie -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs -Wl,-rpath,/Users/benjaminroulston/anaconda3/envs/amuse-env/lib -L/Users/benjaminroulston/anaconda3/envs/amuse-env/lib) works... yes
checking if the fortran compiler supports iso c binding...
checking for a Python interpreter with version >= 3.5... python
checking for python... /Users/benjaminroulston/anaconda3/envs/amuse-env/bin/python
checking for python version... 3.8
checking for python platform... darwin
checking for python script directory... ${prefix}/lib/python3.8/site-packages
checking for python extension module directory... ${exec_prefix}/lib/python3.8/site-packages
checking for /Users/benjaminroulston/anaconda3/envs/amuse-env/bin/python... (cached) /Users/benjaminroulston/anaconda3/envs/amuse-env/bin/python
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for python-config... /usr/bin/python-config
checking for cython... /Users/benjaminroulston/anaconda3/bin/cython
checking Python.h usability... no
checking Python.h presence... no
checking for Python.h... no
configure: WARNING: Cannot find headers (Python.h)
checking for java... /usr/bin/java
checking for javac... /usr/bin/javac
checking for jar... /usr/bin/jar
checking whether Java compiler works... yes
checking whether Java interpreter works... yes
checking Java environment... /Library/Java/JavaVirtualMachines/jdk-9.0.1.jdk/Contents/Home
checking Java version... 9
checking for mpiexec... /Users/benjaminroulston/anaconda3/envs/amuse-env/bin/mpiexec
checking for mpicc... mpicc
checking checking MPI C flags... flags found
checking for MPI_Init... yes
checking for mpi.h... yes
checking for mpicxx... mpicxx
checking checking MPI C++ flags... flags found
checking for mpi.h... yes
checking for mpicc... (cached) mpicc
checking checking MPI C flags... flags found
checking for mpi.h... yes
checking for mpif90... mpif90
checking checking MPI Fortran flags... flags found
checking for mpif.h... no
checking for OpenMP flag of C compiler... -fopenmp
checking for OpenMP flag of Fortran compiler... -fopenmp
checking if gfortran accepts -dumpversion option... yes
checking gfortran version... 9.3.0
checking for a sed that does not truncate output... /usr/bin/sed
checking whether we are using the Microsoft C compiler... no
checking CL/cl.h usability... no
checking CL/cl.h presence... no
checking for CL/cl.h... no
checking OpenCL/cl.h usability... yes
checking OpenCL/cl.h presence... yes
checking for OpenCL/cl.h... yes
checking windows.h usability... no
checking windows.h presence... no
checking for windows.h... no
checking for OpenCL library... no
checking for x86_64-apple-darwin13.4.0-pkg-config... no
checking for pkg-config... /usr/local/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking for FFTW... yes
checking for GMP... yes
checking for MPFR... yes
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking for h5pcc... no
checking for h5cc... /Users/benjaminroulston/anaconda3/envs/amuse-env/bin/h5cc
checking for HDF5 type... serial
checking for HDF5 libraries... yes (version 1.10.6)
checking hdf5.h usability... yes
checking hdf5.h presence... yes
checking for hdf5.h... yes
checking for H5Fcreate in -lhdf5... yes
checking for main in -lhdf5_hl... yes
checking for matching HDF5 Fortran wrapper... no
checking for nc-config... no
checking for NetCDF4 libraries... no
configure: WARNING:

Unable to locate NetCDF4 compilation helper script 'nc-config'.
Please specify --with-netcdf4=<LOCATION> as the full path prefix
where NetCDF4 has been installed.
NetCDF4 support is being disabled (equivalent to --with-netcdf4=no).

checking for gcc option to produce PIC... -fPIC
checking for GSL... yes
checking for python module numpy... found (1.19.4)
checking for python module mpi4py... found (3.0.3)
checking for python module h5py... found (3.1.0)
checking for python module pytest... found (6.2.0)
checking for python module docutils... found (0.16)
checking for python module zlib... found (1.0)
checking for python module matplotlib... found
configure: creating ./config.status
config.status: creating config.mk
config.status: creating build.py
config.status: creating bin/amusifier
configure: Configuration done. Configuration written to config.mk

I then ran make DOWNLOAD_CODES=1 which then resulted in only 9 of the codes being built. I attach the build.log file since it is quite long. It seems there are a lot of issues with compiling the C and Fortran codes and MPI. But I am not totally sure how to fix this. 

(amuse-env) benjaminroulston:make DOWNLOAD_CODES=1
/Users/benjaminroulston/anaconda3/envs/amuse-env/bin/python setup.py generate_main
running generate_main
/Users/benjaminroulston/anaconda3/envs/amuse-env/bin/python setup.py build_codes --inplace
running build_codes
building libraries and community codes
build, for logging, see 'build.log'
[17:32:42] building amuse_mpi-library
[17:32:43] building forsockets-library
[17:32:43] building g6-library
[17:32:43] building sapporo_2-library
[17:32:43] building sapporo_light-library
[17:32:43] building simple_hash-library
[17:32:46] building stopcond-library
[17:32:46] building aarsethzare
[17:32:49] building adaptb
[17:32:51] building asterisk
[17:32:57] building athena
[17:33:02] building bhtree
[17:33:03] building bonsai
[17:33:04] building bonsai2
[17:33:04] building brutus
[17:33:06] building bse
[17:33:11] building capreole
[17:33:11] building distributed
[17:33:17] building etics
[17:33:17] building evtwin
[17:34:16] building fastkick
[17:34:18] building fi
[17:34:39] building flash
[17:34:40] building fractalcluster
[17:34:41] building gadget2
[17:34:46] building galactics
[17:34:50] building galaxia
[17:34:52] building hacs64
[17:34:54] building halogen
[17:34:56] building hermite
[17:34:59] building higpus
[17:35:00] building hop
[17:35:04] building huayno
[17:35:16] building kepler
[17:35:18] building kepler_orbiters
[17:35:19] building krome
[17:35:21] building mameclot
[17:35:22] building mercury
[17:35:22] building mesa
[17:44:24] building mi6
[17:44:27] building mikkola
[17:44:28] building mmams
[17:44:38] building mmc
[17:44:39] building mobse
[17:44:45] building mocassin
[17:45:18] building mosse
[17:45:21] building mpiamrvac
[17:45:29] building nbody6xx
[17:45:30] building octgrav
[17:45:30] building petar
[17:45:57] building ph4
[17:45:59] building phigrape
[17:46:00] building pikachu
[17:46:00] building ramses
[17:46:44] building rebound
[17:47:00] building sakura
[17:47:01] building seba
[17:47:14] building secularmultiple
[17:47:24] building sei
[17:47:26] building simplex
[17:47:27] building smalln
[17:47:29] building sphray
[17:47:36] building sse
[17:47:39] building symple
[17:47:41] building tupan
[17:47:46] building twobody
Community codes not built (because of errors):
================================================================================
 * aarsethzare
 * adaptb
 * athena
 * bhtree
 * brutus
 * bse
 * capreole
 * evtwin
 * fastkick
 * fi
 * flash
 * fractalcluster
 * galactics
 * galaxia
 * hacs64
 * halogen
 * hermite
 * kepler
 * kepler_orbiters
 * krome
 * mercury
 * mesa
 * mi6
 * mikkola
 * mmams
 * mmc
 * mobse
 * mocassin
 * mosse
 * mpiamrvac
 * nbody6xx
 * petar
 * ph4
 * phigrape
 * ramses
 * sakura
 * seba
 * secularmultiple
 * sei
 * simplex
 * smalln
 * sphray
 * sse
 * symple
 * tupan
Optional builds failed, need special libraries:
 * asterisk - java
 * gadget2 - periodic, normal, periodic_nogravity, nogravity
 * higpus - gpu
 * huayno - cl, mp.so
Optional builds failed, need CUDA/GPU libraries:
 * sapporo_light
 * bonsai
 * bonsai2
 * etics
 * octgrav
 * pikachu
================================================================================
Community codes built
================================================================================
* asterisk
* distributed
* gadget2
* higpus
* hop
* huayno (mp)
* mameclot
* rebound
* twobody
================================================================================
9 out of 54 codes built, 5 out of 6 libraries built
rieder commented 3 years ago

This looks like a compiler issue. You seem to use a combination of Anaconda and homebrew, is that correct?

broulston commented 3 years ago

Correct, I use anaconda to make my environment and setup most of my Python modules. I used brew for most of the AMUSE prerequisites since they didn't seem to be on anaconda.

rieder commented 3 years ago

Ok, this is likely to cause the problem. The reason these codes don't build is often because compiler and python libraries don't match. You would have to use either homebrew or anaconda, mixing the two will cause issues like this. I would recommend homebrew over anaconda. Also, I would highly recommend using a python virtual environment, which helps in making sure you use the correct binaries and libraries.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 28 days if no further activity occurs. Thank you for your contributions.