etal / cnvkit

Copy number variant detection from targeted DNA sequencing
http://cnvkit.readthedocs.org
Other
520 stars 163 forks source link

Installing via pip on CentOS and finding atlas libraries #684

Closed astrophys closed 1 year ago

astrophys commented 2 years ago

I am trying to install this software on CentOS 8.1 using pip from within a virtual environment. I had previously tried installing it using conda but after 3h, it failed with

UnsatisfiableError: The following specifications were found to be incompatible with the existing python installation in your environment:

followed by about 10k lines of output. So I tried something similar. I created a virtual environment, activated the environment and loaded some modules. We use lmod modules (which are written in lua) to manage software. The module files basically modify the user's environment (e.g. PATH and LD_LIBRARY_PATH) to find the executables and runtime libraries. So I ran :

(cnvkit) r1pl-hpcf-log01: venv-example$ ml load GCC/8.2.0-2.31.1
(cnvkit) r1pl-hpcf-log01: venv-example$ ml load LAPACK/3.9.0
(cnvkit) r1pl-hpcf-log01: venv-example$ pip install cnvkit

Collecting cnvkit
  Using cached https://files.pythonhosted.org/packages/93/d9/ee1f7b814a8a019225f97dfcc3352a3667f60b653cf9671dd7618807bfbe/CNVkit-0.9.9.tar.gz
Collecting biopython>=1.62 (from cnvkit)
  Using cached https://files.pythonhosted.org/packages/29/3b/a892a2782334544c379b89943b1d548b7b26b7bb4730561ceb942076b2a7/biopython-1.79-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl
Collecting matplotlib>=1.3.1 (from cnvkit)
  Using cached https://files.pythonhosted.org/packages/22/1c/d5e535b36c1de4eef4205656e76ac993c6d01b62cfdcac579edb63cd82e0/matplotlib-3.5.1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl
Collecting numpy>=1.9 (from cnvkit)
  Using cached https://files.pythonhosted.org/packages/50/e1/9b0c184f04b8cf5f3c941ffa56fbcbe936888bdac9aa7ba6bae405ac752b/numpy-1.22.0.zip
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting pandas>=0.23.3 (from cnvkit)
  Using cached https://files.pythonhosted.org/packages/99/f0/f99700ef327e51d291efdf4a6de29e685c4d198cbf8531541fc84d169e0e/pandas-1.3.5.tar.gz
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting pomegranate>=0.9.0 (from cnvkit)
  Using cached https://files.pythonhosted.org/packages/de/0e/8db1be7b4c6b90a7f576c341488527ba158f9d6b7326b7e94563fea132af/pomegranate-0.14.7.tar.gz
  Installing build dependencies ... error
  ERROR: Command errored out with exit status 1:
   command: /gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/bin/python /gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-8e0sb81k/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- cython numpy scipy setuptools wheel
       cwd: None
  Complete output (351 lines):
  Collecting cython
    Using cached https://files.pythonhosted.org/packages/be/d2/7afd6361394d9bee2429e3e5bdc7fe363440b63f5fc8e08db3df9b8ef6b4/Cython-0.29.26-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl
  Collecting numpy
    Using cached https://files.pythonhosted.org/packages/50/e1/9b0c184f04b8cf5f3c941ffa56fbcbe936888bdac9aa7ba6bae405ac752b/numpy-1.22.0.zip
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
      Preparing wheel metadata: started
      Preparing wheel metadata: finished with status 'done'
  Collecting scipy
    Using cached https://files.pythonhosted.org/packages/61/67/1a654b96309c991762ee9bc39c363fc618076b155fe52d295211cf2536c7/scipy-1.7.3.tar.gz
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
      Preparing wheel metadata: started
      Preparing wheel metadata: finished with status 'error'
      ERROR: Command errored out with exit status 1:
       command: /gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/bin/python /gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpjii1wedc
           cwd: /tmp/pip-install-ydurjjq4/scipy
      Complete output (325 lines):
      Running scipy/linalg/_generate_pyx.py
      Running scipy/special/_generate_pyx.py
      Running scipy/stats/_generate_pyx.py
      Processing scipy/_lib/_ccallback_c.pyx
      Processing scipy/_lib/_test_deprecation_call.pyx
      Processing scipy/_lib/_test_deprecation_def.pyx
      Processing scipy/_lib/messagestream.pyx
      Processing scipy/cluster/_hierarchy.pyx
      Processing scipy/cluster/_optimal_leaf_ordering.pyx
      Processing scipy/cluster/_vq.pyx
      Processing scipy/fftpack/convolve.pyx
      Processing scipy/interpolate/_bspl.pyx
      Processing scipy/interpolate/_ppoly.pyx
      Processing scipy/interpolate/interpnd.pyx
      Processing scipy/io/matlab/mio5_utils.pyx
      Processing scipy/io/matlab/mio_utils.pyx
      Processing scipy/io/matlab/streams.pyx
      Processing scipy/linalg/_decomp_update.pyx.in
      Processing scipy/linalg/_matfuncs_sqrtm_triu.pyx
      Processing scipy/linalg/_solve_toeplitz.pyx
      Processing scipy/linalg/cython_blas.pyx
      Processing scipy/linalg/cython_lapack.pyx
      Processing scipy/ndimage/src/_cytest.pyx
      Processing scipy/ndimage/src/_ni_label.pyx
      Processing scipy/optimize/_bglu_dense.pyx
      Processing scipy/optimize/_group_columns.pyx
      Processing scipy/optimize/_highs/cython/src/_highs_constants.pyx
      Processing scipy/optimize/_highs/cython/src/_highs_wrapper.pyx
      Processing scipy/optimize/_lsq/givens_elimination.pyx
      Processing scipy/optimize/_trlib/_trlib.pyx
      Processing scipy/optimize/cython_optimize/_zeros.pyx.in
      Processing scipy/signal/_max_len_seq_inner.pyx
      Processing scipy/signal/_peak_finding_utils.pyx
      Processing scipy/signal/_sosfilt.pyx
      Processing scipy/signal/_spectral.pyx
      Processing scipy/signal/_upfirdn_apply.pyx
      Processing scipy/sparse/_csparsetools.pyx.in
      Processing scipy/sparse/csgraph/_flow.pyx
      Processing scipy/sparse/csgraph/_matching.pyx
      Processing scipy/sparse/csgraph/_min_spanning_tree.pyx
      Processing scipy/sparse/csgraph/_reordering.pyx
      Processing scipy/sparse/csgraph/_shortest_path.pyx
      Processing scipy/sparse/csgraph/_tools.pyx
      Processing scipy/sparse/csgraph/_traversal.pyx
      Processing scipy/spatial/_hausdorff.pyx
      Processing scipy/spatial/_voronoi.pyx
      Processing scipy/spatial/ckdtree.pyx
      Processing scipy/spatial/qhull.pyx
      Processing scipy/spatial/transform/rotation.pyx
      Processing scipy/special/_comb.pyx
      Processing scipy/special/_ellip_harm_2.pyx
      Processing scipy/special/_test_round.pyx
      Processing scipy/special/_ufuncs.pyx
      Processing scipy/special/_ufuncs_cxx.pyx
      Processing scipy/special/cython_special.pyx
      Processing scipy/stats/_qmc_cy.pyx
      Processing scipy/stats/_sobol.pyx
      Processing scipy/stats/_stats.pyx
      Processing scipy/stats/biasedurn.pyx
      Processing scipy/stats/_boost/src/beta_ufunc.pyx
      Processing scipy/stats/_boost/src/binom_ufunc.pyx
      Processing scipy/stats/_boost/src/nbinom_ufunc.pyx
      warning: _cython_special_custom.pxi:9:8: Unreachable code
      warning: _cython_special_custom.pxi:13:4: Unreachable code
      warning: _cython_special_custom.pxi:21:8: Unreachable code
      warning: _cython_special_custom.pxi:25:4: Unreachable code
      warning: _cython_special_custom.pxi:33:8: Unreachable code
      warning: _cython_special_custom.pxi:37:4: Unreachable code
      warning: _cython_special_custom.pxi:45:8: Unreachable code
      warning: _cython_special_custom.pxi:49:4: Unreachable code
      Cythonizing sources
      lapack_opt_info:
      lapack_mkl_info:
      customize UnixCCompiler
        libraries mkl_rt not found in ['/gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/']
        NOT AVAILABLE

      openblas_lapack_info:
      customize UnixCCompiler
      customize UnixCCompiler
        libraries openblas not found in ['/gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/']
        NOT AVAILABLE

      openblas_clapack_info:
      customize UnixCCompiler
      customize UnixCCompiler
        libraries openblas,lapack not found in ['/gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/']
        NOT AVAILABLE

      flame_info:
      customize UnixCCompiler
        libraries flame not found in ['/gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/']
        NOT AVAILABLE

      atlas_3_10_threads_info:
      Setting PTATLAS=ATLAS
      customize UnixCCompiler
        libraries lapack_atlas not found in /gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/lib
      customize UnixCCompiler
        libraries tatlas,tatlas not found in /gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/lib
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/local/lib64
      customize UnixCCompiler
        libraries tatlas,tatlas not found in /usr/local/lib64
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/local/lib
      customize UnixCCompiler
        libraries tatlas,tatlas not found in /usr/local/lib
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib64/atlas
      customize UnixCCompiler
        libraries tatlas,tatlas not found in /usr/lib64/atlas
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib64/sse2
      customize UnixCCompiler
        libraries tatlas,tatlas not found in /usr/lib64/sse2
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib64
      customize UnixCCompiler
        libraries tatlas,tatlas not found in /usr/lib64
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib/sse2
      customize UnixCCompiler
        libraries tatlas,tatlas not found in /usr/lib/sse2
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib
      customize UnixCCompiler
        libraries tatlas,tatlas not found in /usr/lib
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib/sse2
      customize UnixCCompiler
        libraries tatlas,tatlas not found in /usr/lib/sse2
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib/
      customize UnixCCompiler
        libraries tatlas,tatlas not found in /usr/lib/
      <class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
        NOT AVAILABLE

      atlas_3_10_info:
      customize UnixCCompiler
        libraries lapack_atlas not found in /gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/lib
      customize UnixCCompiler
        libraries satlas,satlas not found in /gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/lib
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/local/lib64
      customize UnixCCompiler
        libraries satlas,satlas not found in /usr/local/lib64
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/local/lib
      customize UnixCCompiler
        libraries satlas,satlas not found in /usr/local/lib
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib64/atlas
      customize UnixCCompiler
        libraries satlas,satlas not found in /usr/lib64/atlas
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib64/sse2
      customize UnixCCompiler
        libraries satlas,satlas not found in /usr/lib64/sse2
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib64
      customize UnixCCompiler
        libraries satlas,satlas not found in /usr/lib64
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib/sse2
      customize UnixCCompiler
        libraries satlas,satlas not found in /usr/lib/sse2
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib
      customize UnixCCompiler
        libraries satlas,satlas not found in /usr/lib
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib/sse2
      customize UnixCCompiler
        libraries satlas,satlas not found in /usr/lib/sse2
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib/
      customize UnixCCompiler
        libraries satlas,satlas not found in /usr/lib/
      <class 'numpy.distutils.system_info.atlas_3_10_info'>
        NOT AVAILABLE

      atlas_threads_info:
      Setting PTATLAS=ATLAS
      customize UnixCCompiler
        libraries lapack_atlas not found in /gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/lib
      customize UnixCCompiler
        libraries ptf77blas,ptcblas,atlas not found in /gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/lib
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/local/lib64
      customize UnixCCompiler
        libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib64
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/local/lib
      customize UnixCCompiler
        libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib64/atlas
      customize UnixCCompiler
        libraries ptf77blas,ptcblas,atlas not found in /usr/lib64/atlas
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib64/sse2
      customize UnixCCompiler
        libraries ptf77blas,ptcblas,atlas not found in /usr/lib64/sse2
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib64
      customize UnixCCompiler
        libraries ptf77blas,ptcblas,atlas not found in /usr/lib64
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib/sse2
      customize UnixCCompiler
        libraries ptf77blas,ptcblas,atlas not found in /usr/lib/sse2
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib
      customize UnixCCompiler
        libraries ptf77blas,ptcblas,atlas not found in /usr/lib
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib/sse2
      customize UnixCCompiler
        libraries ptf77blas,ptcblas,atlas not found in /usr/lib/sse2
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib/
      customize UnixCCompiler
        libraries ptf77blas,ptcblas,atlas not found in /usr/lib/
      <class 'numpy.distutils.system_info.atlas_threads_info'>
        NOT AVAILABLE

      atlas_info:
      customize UnixCCompiler
        libraries lapack_atlas not found in /gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/lib
      customize UnixCCompiler
        libraries f77blas,cblas,atlas not found in /gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/lib
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/local/lib64
      customize UnixCCompiler
        libraries f77blas,cblas,atlas not found in /usr/local/lib64
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/local/lib
      customize UnixCCompiler
        libraries f77blas,cblas,atlas not found in /usr/local/lib
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib64/atlas
      customize UnixCCompiler
        libraries f77blas,cblas,atlas not found in /usr/lib64/atlas
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib64/sse2
      customize UnixCCompiler
        libraries f77blas,cblas,atlas not found in /usr/lib64/sse2
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib64
      customize UnixCCompiler
        libraries f77blas,cblas,atlas not found in /usr/lib64
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib/sse2
      customize UnixCCompiler
        libraries f77blas,cblas,atlas not found in /usr/lib/sse2
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib
      customize UnixCCompiler
        libraries f77blas,cblas,atlas not found in /usr/lib
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib/sse2
      customize UnixCCompiler
        libraries f77blas,cblas,atlas not found in /usr/lib/sse2
      customize UnixCCompiler
        libraries lapack_atlas not found in /usr/lib/
      customize UnixCCompiler
        libraries f77blas,cblas,atlas not found in /usr/lib/
      <class 'numpy.distutils.system_info.atlas_info'>
        NOT AVAILABLE

      accelerate_info:
        NOT AVAILABLE

      lapack_info:
      customize UnixCCompiler
        libraries lapack not found in ['/gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/']
        NOT AVAILABLE

      lapack_src_info:
        NOT AVAILABLE

        NOT AVAILABLE

      setup.py:491: UserWarning: Unrecognized setuptools command ('dist_info --egg-base /tmp/pip-install-ydurjjq4/scipy/pip-wheel-metadata'), proceeding with generating Cython sources and expanding templates
        warnings.warn("Unrecognized setuptools command ('{}'), proceeding with "
      Running from SciPy source directory.
      /tmp/pip-build-env-4_0uorpp/overlay/lib/python3.8/site-packages/numpy/distutils/system_info.py:1712: UserWarning:
          Lapack (http://www.netlib.org/lapack/) libraries not found.
          Directories to search for the libraries can be specified in the
          numpy/distutils/site.cfg file (section [lapack]) or by setting
          the LAPACK environment variable.
        if getattr(self, '_calc_info_{}'.format(lapack))():
      /tmp/pip-build-env-4_0uorpp/overlay/lib/python3.8/site-packages/numpy/distutils/system_info.py:1712: UserWarning:
          Lapack (http://www.netlib.org/lapack/) sources not found.
          Directories to search for the sources can be specified in the
          numpy/distutils/site.cfg file (section [lapack_src]) or by setting
          the LAPACK_SRC environment variable.
        if getattr(self, '_calc_info_{}'.format(lapack))():
      Traceback (most recent call last):
        File "/gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 207, in <module>
          main()
        File "/gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 197, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 69, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)
        File "/tmp/pip-build-env-4_0uorpp/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 166, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/tmp/pip-build-env-4_0uorpp/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 258, in run_setup
          super(_BuildMetaLegacyBackend,
        File "/tmp/pip-build-env-4_0uorpp/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 150, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 631, in <module>
          setup_package()
        File "setup.py", line 627, in setup_package
          setup(**metadata)
        File "/tmp/pip-build-env-4_0uorpp/overlay/lib/python3.8/site-packages/numpy/distutils/core.py", line 137, in setup
          config = configuration()
        File "setup.py", line 529, in configuration
          raise NotFoundError(msg)
      numpy.distutils.system_info.NotFoundError: No BLAS/LAPACK libraries found.
      To build Scipy from sources, BLAS & LAPACK libraries need to be installed.
      See site.cfg.example in the Scipy source directory and
      https://docs.scipy.org/doc/scipy/reference/building/index.html for details.
      ----------------------------------------
  ERROR: Command errored out with exit status 1: /gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/bin/python /gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpjii1wedc Check the logs for full command output.
  WARNING: You are using pip version 19.2.3, however version 21.3.1 is available.
  You should consider upgrading via the 'pip install --upgrade pip' command.
  ----------------------------------------
ERROR: Command errored out with exit status 1: /gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/bin/python /gpfs0/home1/group/user001/Scratch/problems/user_problems/mvc002/venv-example/cnvkit/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-8e0sb81k/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- cython numpy scipy setuptools wheel Check the logs for full command output.
WARNING: You are using pip version 19.2.3, however version 21.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Amongst the many errors, I see

customize UnixCCompiler
libraries satlas,satlas not found in /usr/lib64/atlas

However I have those libraries installed via yum and they are in the /usr/lib64/atlas directory :

$ ls /usr/lib64/atlas
libsatlas.so.3  libsatlas.so.3.10  libtatlas.so.3  libtatlas.so.3.10

Also, I have my lapack libraries within LD_LIBRARY_PATH, i.e.

$ echo $LD_LIBRARY_PATH
/export/apps/opt/LAPACK/3.9.0-GCC-8.2.0/lib64:/gpfs0/export/apps/easybuild/software/binutils/2.31.1-GCCcore-8.2.0/lib:/gpfs0/export/apps/easybuild/software/zlib/1.2.11-GCCcore-8.2.0/lib:/gpfs0/export/apps/easybuild/software/GCCcore/8.2.0/lib64:/gpfs0/export/apps/easybuild/software/GCCcore/8.2.0/lib
$ ls /export/apps/opt/LAPACK/3.9.0-GCC-8.2.0/lib64
cmake       libblas.so.3      liblapack.so    liblapack.so.3.9.0
libblas.so  libblas.so.3.9.0  liblapack.so.3  pkgconfig

Questions:

  1. Why isn't the pip installation finding my lapack and atlas libraries? The atlas libraries are installed where it claims to have looked (/usr/lib64/atlas) and the lapack libraries are in my LD_LIBRARY_PATH
  2. Something seems very different with this pip installation. I've done several installs of scipy via virtual environments with pip and I did not have to install a lapack system library. Why must I install the system level libraries in this case when in previous instances I did not? Perhaps it is a difference between compiling from source vs. downloading precompiled executables?

Thank you for your time.

tetedange13 commented 2 years ago

I @astrophys ,

Not an author of CNVkit, but I think 2nd "atlas-related" error you got is not directly related to CNVkit itself => But rather your set up, or your PATH, or your Python installation or others => A bit hard to solve from here (as far as I am concerned, maybe someone could)

I advice you to get back to conda way of installing CNVkit (should be simpler and more predictable) => Error 1st you got looks like you tried to install CNVkit inside base env on your conda install ? Like conda install cnvkit ? => Generally speaking, this is something to be avoided with conda (can cause conflicts between versions = "were found to be incompatible with the existing python installation in your environment")

Better to create a dedicated env: conda create -n cnvkit Then install CNVkit package within this env: conda install -n cnvkit -c bioconda cnvkit => Tell me if this worked for you ?

Hope this helped. Have a nice day. Felix.

etal commented 2 years ago

Looks like CentOS 8 is past end of life, so dependency resolution will get trickier as time goes on.

Another option you can try besides conda is the prebuilt Docker image: https://hub.docker.com/r/etal/cnvkit/

etal commented 1 year ago

For HPC usage, you may also find conda or podman can avoid problems in the global environment.