Installation issue

[Hassankh92]

Dear Zach, I am trying to install the package, and everything is going on without error, but when I am trying to import any of the codes in Python, I am getting gsl-related errors like (undefined symbol: gsl_odeiv2_control_free). This is true for the bhpwave as well. I guess it has something to do with the compiling part?! I would appreciate it if you could give me a hint on that. Thank you very much.

[znasipak]

Hi Hassan, I am sorry that you are having installation issues. What OS are you using (Mac? Linux? Windows?) and are you using a virtual environment through Anaconda?

[Hassankh92]

Hi Zach, Thank you for your response. I am using Linux (Ubuntu), and yes I have a separate conda environment for this code. When I am using pip to install it, it is going through with no error. But it seems that something is not quite well compiled. The only thing is that with python 3.7, the pip is not working and I had to change it to Python 3.9! Could that be the problem?

[znasipak]

Interesting. I don't think Python 3.9 should be an issue. Yes this seems to be a problem that when the code is compiled, it is not linking to the proper include directory for the gsl library. First, double check that conda actually downloaded and stored the GSL library properly. If it did, you should be able to find it under /PATH/TO/ANACONDA/envs/pybhpt-env/include/gsl.

If that exists, then there is a linking issue. Run echo $CC and tell me what you get. Additionally, check if the $PATH variable points to the gsl library

[Hassankh92]

Thank you very much. You are right. Because when I want to compile my c++ codes the gsl and lapack would not be detected automatically. I forgo to mention that I am using the hpc (symmetry at PI). The gsl lib is stored in the mentioned path. ->> /envs/pybhpt-env/include/gsl.

for the CC -->> /home/hkhalvati/.conda/envs/pybhpt-env/bin/x86_64-conda-linux-gnu-cc

The gsl path does not exist in $PATH. Should I export the gsl lib path above to the $PATH?

I am going to try installing it on my local Ubuntu as well, I guess it would work.

[znasipak]

Okay that all looks correct, so I am assuming the PATH variable is fine. It only needs to point to the environments /bin directory. Usually by having everything in the conda environment, the compiler is able to locate and link against all of the appropriate libraries. Let me know how the install on your local Ubuntu goes. I will try compiling on my hpc and see if I run into similar errors. So far I have only stress tested install on macOS. So let's see if this is an issue with compiling on linux or an issue with the directory structure on the hpc.

[znasipak]

Try replacing the setup.py file with the following code:

import os
from Cython.Build import cythonize
from setuptools import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
import numpy as np
from subprocess import run
import sys

compiler_flags = ["-std=c++11", "-fopenmp", "-march=native"]
libraries = ["gsl", "gslcblas", "boost_filesystem"]

# set some flags and libraries depending on the system platform
if sys.platform.startswith('win32'):
    compiler_flags.append('/Od')
    libraries.append('gomp')
elif sys.platform.startswith('darwin'):
    compiler_flags.append('-O2')
    libraries.append('omp')
elif sys.platform.startswith('linux'):
    compiler_flags.append('-O2')
    libraries.append('gomp')

CFLAGS = os.getenv("CFLAGS")
if CFLAGS is None:
    CFLAGS = ""
else:
    CFLAGS = str(CFLAGS)
os.environ["CFLAGS"] = CFLAGS
os.environ["CFLAGS"] += " "
os.environ["CFLAGS"] += ' '.join(compiler_flags)
base_path = sys.prefix

utils_dependence = ["cpp/src/utils.cpp"]
specialfunc_dependence = ["cpp/src/specialfunc.cpp"]
kerr_dependence = ["cpp/src/kerr.cpp", *specialfunc_dependence]
geo_dependence = ["cpp/src/geo.cpp", *specialfunc_dependence]
cf_dependence = ["cpp/src/cf.cpp", *utils_dependence]
swsh_dependence = ["cpp/src/swsh.cpp", *specialfunc_dependence]
monodromy_dependence = ["cpp/src/monodromy.cpp", *specialfunc_dependence]
hyperf_dependence = ["cpp/src/hypergeo_f.cpp", *specialfunc_dependence]
bessel_dependence = ["cpp/src/bessel.cpp", *hyperf_dependence, *specialfunc_dependence]
hyperu_dependence = ["cpp/src/hypergeo_u.cpp", *cf_dependence, *bessel_dependence, *specialfunc_dependence]
nusolver_dependence = ["cpp/src/nusolver.cpp", *monodromy_dependence, *cf_dependence, *swsh_dependence]
mst_dependence = ["cpp/src/mst.cpp", *nusolver_dependence, *hyperu_dependence, *hyperf_dependence]
gsn_dependence = ["cpp/src/gsn_asymp.cpp", *specialfunc_dependence]
radial_dependence = ["cpp/src/radialsolver.cpp", *mst_dependence, *gsn_dependence]
sourceint_dependence = ["cpp/src/sourceintegration.cpp", *geo_dependence, *swsh_dependence, *radial_dependence, "cpp/src/specialfunc.cpp"]
teuk_dependence = ["cpp/src/teukolsky.cpp", *sourceint_dependence, *radial_dependence]
hertz_dependence = ["cpp/src/hertz.cpp", *teuk_dependence]
metriccoeffs_dependence = ["cpp/src/metriccoeffs.cpp", *geo_dependence, *utils_dependence]
fluxes_dependence = ["cpp/src/fluxes.cpp", *teuk_dependence]
metric_dependence = ["cpp/src/metric.cpp", *hertz_dependence, *utils_dependence, *metriccoeffs_dependence, *kerr_dependence]
redshift_dependence = ["cpp/src/redshift.cpp", *metric_dependence]
unit_dependence = ["cpp/src/unit_test.cpp", *fluxes_dependence]

full_dependence = [*geo_dependence, *teuk_dependence, *radial_dependence, *hertz_dependence, *metriccoeffs_dependence, *fluxes_dependence, *redshift_dependence, *unit_dependence]

# create library of all the c++ files that get linked at the end in setup so that we are
# not duplicating c++ file compilation across the different extensions
# lib_extension = dict(
#     sources = [*set(full_dependence)],
#     libraries=libraries,
#     language='c++',
#     include_dirs = ["cpp/include", base_path + "/include"],
# )

# cppbhpt = ['cppbhpt', lib_extension]

cpu_extension = dict(
    libraries=libraries,
    language='c++',
    # extra_compile_args=["-Xclang", "-fopenmp", "-O2"], 
    # extra_compile_args=["-Xclang", "-fopenmp", "-O0", "-ggdb"], # use to speed up compilation during development
    include_dirs=["cpp/include", np.get_include()],
)

teuk_ext = Extension(
    "cybhpt_full", 
    sources=["cython/redshift_wrap.pyx", *set(full_dependence)], 
    **cpu_extension,
)

ext_modules = [teuk_ext]

setup(
    name="pybhpt",
    author="Zach Nasipak",
    author_email="znasipak@gmail.com",
    version = "0.0.1",
    description = "Black Hole Perturbation Theory and Self-Force Algorithms in Python",
    ext_modules = cythonize(ext_modules, language_level = "3"),
    packages=["pybhpt"],
    py_modules=["pybhpt.geo", "pybhpt.swsh", "pybhpt.radial", "pybhpt.teuk", "pybhpt.hertz", "pybhpt.metric"],
    classifiers=[
        "Programming Language :: Python :: 3",
        "License :: OSI Approved :: GNU General Public License (GPL)",
        "Natural Language :: English",
        "Programming Language :: C++",
        "Programming Language :: Cython",
    ],
#    libraries = [cppbhpt],
    cmdclass = {'build_ext': build_ext},
    zip_safe=False
)

Recompile with this new setup install file and let me know if things work. This got everything working with our hpc

[Hassankh92]

Alrigh. I am using the ubuntu on my laptop, the same issue yet. Even with replacing the new setup file for the hpc.

Am I missing something?

[znasipak]

Did you pip uninstall pybhpt and remove the build/ directory? If not, try those and re-install with the new setup.py script and let me know if that works.

[Hassankh92]

I see! Perfect. It worked on my local and the hpc both. Thank you very much for your time and super quick responses.

[znasipak]

No problem! I've made some changes to the code to use the new setup.py. So if you need to clone the repository or pull again, everything should work. Thank you for identifying this issue! Please report any issues that you run into! It is really helpful to me. Also note that I am still developing this code quite a bit, so expect more tutorials/documentation and possible API changes in the coming months.

[znasipak]

Installation issues were resolved with 63479b0