search

kcleal / dysgu

Toolkit for calling structural variants using short or long reads
MIT License
98 stars 12 forks source link

Error - pip3 dysgu installation #31

Closed KristinaGagalova closed 2 years ago

KristinaGagalova commented 2 years ago

Hi, I am upgrading to the newest version of dysgu and I have the following error with pip3. I created a dedicated conda env especially for that.

Collecting dysgu
  Using cached dysgu-1.3.7-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (72.7 MB)
Collecting lightgbm
  Using cached lightgbm-3.3.2-py3-none-manylinux1_x86_64.whl (2.0 MB)
Requirement already satisfied: numpy>=1.16.5 in /home/kgagalova/.linuxbrew/lib/python3.9/site-packages (from dysgu) (1.19.5)
Collecting networkx>=2.4
  Using cached networkx-2.7.1-py3-none-any.whl (2.0 MB)
Requirement already satisfied: pandas in /home/kgagalova/.linuxbrew/lib/python3.9/site-packages (from dysgu) (1.3.3)
Requirement already satisfied: scipy in /home/kgagalova/.linuxbrew/lib/python3.9/site-packages (from dysgu) (1.7.1)
Collecting click>=8.0
  Using cached click-8.0.4-py3-none-any.whl (97 kB)
Collecting scikit-learn>=0.22
  Using cached scikit_learn-1.0.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (26.4 MB)
Collecting edlib
  Using cached edlib-1.3.9-cp39-cp39-manylinux2010_x86_64.whl (327 kB)
Requirement already satisfied: pysam in /home/kgagalova/.linuxbrew/lib/python3.9/site-packages (from dysgu) (0.17.0)
Collecting cython
  Using cached Cython-0.29.28-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (1.9 MB)
Collecting scikit-bio
  Using cached scikit-bio-0.5.6.tar.gz (8.4 MB)
  Preparing metadata (setup.py) ... done
Collecting sortedcontainers
  Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting joblib>=0.11
  Using cached joblib-1.1.0-py2.py3-none-any.whl (306 kB)
Collecting threadpoolctl>=2.0.0
  Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Requirement already satisfied: wheel in /home/kgagalova/.linuxbrew/lib/python3.9/site-packages (from lightgbm->dysgu) (0.37.1)
Requirement already satisfied: python-dateutil>=2.7.3 in /home/kgagalova/.linuxbrew/lib/python3.9/site-packages (from pandas->dysgu) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /home/kgagalova/.linuxbrew/lib/python3.9/site-packages (from pandas->dysgu) (2021.1)
Collecting lockfile>=0.10.2
  Using cached lockfile-0.12.2-py2.py3-none-any.whl (13 kB)
Collecting CacheControl>=0.11.5
  Using cached CacheControl-0.12.10-py2.py3-none-any.whl (20 kB)
Collecting decorator>=3.4.2
  Using cached decorator-5.1.1-py3-none-any.whl (9.1 kB)
Collecting IPython>=3.2.0
  Using cached ipython-8.1.1-py3-none-any.whl (750 kB)
Collecting matplotlib>=1.4.3
  Using cached matplotlib-3.5.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl (11.2 MB)
Collecting natsort>=4.0.3
  Using cached natsort-8.1.0-py3-none-any.whl (37 kB)
Collecting hdmedians>=0.13
  Using cached hdmedians-0.14.2.tar.gz (7.6 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: requests in /home/kgagalova/.linuxbrew/lib/python3.9/site-packages (from CacheControl>=0.11.5->scikit-bio->dysgu) (2.26.0)
Collecting msgpack>=0.5.2
  Using cached msgpack-1.0.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (322 kB)
Collecting prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0
  Using cached prompt_toolkit-3.0.28-py3-none-any.whl (380 kB)
Requirement already satisfied: setuptools>=18.5 in /home/kgagalova/.linuxbrew/lib/python3.9/site-packages (from IPython>=3.2.0->scikit-bio->dysgu) (60.5.0)
Collecting matplotlib-inline
  Using cached matplotlib_inline-0.1.3-py3-none-any.whl (8.2 kB)
Collecting pexpect>4.3
  Using cached pexpect-4.8.0-py2.py3-none-any.whl (59 kB)
Collecting stack-data
  Using cached stack_data-0.2.0-py3-none-any.whl (21 kB)
Collecting jedi>=0.16
  Using cached jedi-0.18.1-py2.py3-none-any.whl (1.6 MB)
Collecting traitlets>=5
  Using cached traitlets-5.1.1-py3-none-any.whl (102 kB)
Collecting backcall
  Using cached backcall-0.2.0-py2.py3-none-any.whl (11 kB)
Collecting pygments>=2.4.0
  Using cached Pygments-2.11.2-py3-none-any.whl (1.1 MB)
Collecting pickleshare
  Using cached pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB)
Collecting kiwisolver>=1.0.1
  Using cached kiwisolver-1.4.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB)
Collecting fonttools>=4.22.0
  Using cached fonttools-4.31.2-py3-none-any.whl (899 kB)
Collecting cycler>=0.10
  Using cached cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting pillow>=6.2.0
  Using cached Pillow-9.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.3 MB)
Collecting packaging>=20.0
  Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting pyparsing>=2.2.1
  Using cached pyparsing-3.0.7-py3-none-any.whl (98 kB)
Requirement already satisfied: six>=1.5 in /home/kgagalova/.linuxbrew/lib/python3.9/site-packages (from python-dateutil>=2.7.3->pandas->dysgu) (1.15.0)
Collecting parso<0.9.0,>=0.8.0
  Using cached parso-0.8.3-py2.py3-none-any.whl (100 kB)
Collecting ptyprocess>=0.5
  Using cached ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB)
Collecting wcwidth
  Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: idna<4,>=2.5 in /home/kgagalova/.linuxbrew/lib/python3.9/site-packages (from requests->CacheControl>=0.11.5->scikit-bio->dysgu) (3.2)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/kgagalova/.linuxbrew/lib/python3.9/site-packages (from requests->CacheControl>=0.11.5->scikit-bio->dysgu) (1.26.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in /home/kgagalova/.linuxbrew/lib/python3.9/site-packages (from requests->CacheControl>=0.11.5->scikit-bio->dysgu) (2.0.6)
Requirement already satisfied: certifi>=2017.4.17 in /home/kgagalova/.linuxbrew/lib/python3.9/site-packages (from requests->CacheControl>=0.11.5->scikit-bio->dysgu) (2021.5.30)
Collecting asttokens
  Using cached asttokens-2.0.5-py2.py3-none-any.whl (20 kB)
Collecting executing
  Using cached executing-0.8.3-py2.py3-none-any.whl (16 kB)
Collecting pure-eval
  Using cached pure_eval-0.2.2-py3-none-any.whl (11 kB)
Building wheels for collected packages: scikit-bio, hdmedians
  Building wheel for scikit-bio (setup.py) ... error
ERROR: Command errored out with exit status 1:
   command: /home/kgagalova/.linuxbrew/opt/python@3.9/bin/python3.9 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-22gwih1p/scikit-bio_fd5adfa3cbcc4059926676ebae73c81f/setup.py'"'"'; __file__='"'"'/tmp/pip-install-22gwih1p/scikit-bio_fd5adfa3cbcc4059926676ebae73c81f/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-c_4psgrc
       cwd: /tmp/pip-install-22gwih1p/scikit-bio_fd5adfa3cbcc4059926676ebae73c81f/
  Complete output (663 lines):
  running bdist_wheel
  running build
[....]
running build_ext
  creating build/temp.linux-x86_64-3.9
  creating build/temp.linux-x86_64-3.9/skbio
  creating build/temp.linux-x86_64-3.9/skbio/metadata
  gcc-5 -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/home/kgagalova/.linuxbrew/opt/python@3.9/lib/python3.9/site-packages/numpy/core/include -I/home/kgagalova/.linuxbrew/opt/python@3.9/include/python3.9 -c skbio/metadata/_intersection.c -o build/temp.linux-x86_64-3.9/skbio/metadata/_intersection.o
  error: command 'gcc-5' failed: No such file or directory
  ----------------------------------------
  ERROR: Failed building wheel for scikit-bio
  Running setup.py clean for scikit-bio
  Building wheel for hdmedians (pyproject.toml) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/kgagalova/.linuxbrew/opt/python@3.9/bin/python3.9 /home/kgagalova/.linuxbrew/opt/python@3.9/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmpke8nnicz
       cwd: /tmp/pip-install-22gwih1p/hdmedians_34f0c0b3631144b6aeec2bd7484540bd
  Complete output (9 lines):
  running bdist_wheel
  running build
  running build_py
  running build_ext
  cythoning hdmedians/geomedian.pyx to hdmedians/geomedian.c
  /tmp/pip-build-env-b8fpnjvp/overlay/lib/python3.9/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/pip-install-22gwih1p/hdmedians_34f0c0b3631144b6aeec2bd7484540bd/hdmedians/geomedian.pyx
    tree = Parsing.p_module(s, pxd, full_module_name)
  building 'hdmedians.geomedian' extension
  error: command 'gcc-5' failed: No such file or directory
  ----------------------------------------
  ERROR: Failed building wheel for hdmedians
Failed to build scikit-bio hdmedians
ERROR: Could not build wheels for hdmedians, which is required to install pyproject.toml-based projects

Looks like I have some issues installing gcc-5 on my system. 

ERROR conda.core.link:_execute(699): An error occurred while installing package 'psi4::gcc-5-5.2.0-1'.
Rolling back transaction: done

LinkError: post-link script failed for package psi4::gcc-5-5.2.0-1
location of failed script: /home/kgagalova/miniconda3/envs/py3.7/bin/.gcc-5-post-link.sh
==> script messages <==
<None>
==> script output <==
stdout: Couldn't locate crtXXX.o in default library search paths. You may not have it  at all. It is usually packaged in libc6-dev/glibc-devel packages. We will try  to locate crtXXX.o with system installed gcc...
Installation failed: gcc is not able to compile a simple 'Hello, World' program.

stderr: ln: failed to create symbolic link '/home/kgagalova/miniconda3/envs/py3.7/lib/gcc/x86_64-unknown-linux-gnu/5.2.0/crt1.o': File exists
ln: failed to create symbolic link '/home/kgagalova/miniconda3/envs/py3.7/lib/gcc/x86_64-unknown-linux-gnu/5.2.0/crti.o': File exists
ln: failed to create symbolic link '/home/kgagalova/miniconda3/envs/py3.7/lib/gcc/x86_64-unknown-linux-gnu/5.2.0/11.2.0': File exists
ln: failed to create symbolic link '/home/kgagalova/miniconda3/envs/py3.7/lib/gcc/x86_64-unknown-linux-gnu/5.2.0/crtn.o': File exists
ln: failed to create symbolic link '/home/kgagalova/miniconda3/envs/py3.7/lib/gcc/x86_64-unknown-linux-gnu/5.2.0/11.2.0': File exists
/home/kgagalova/miniconda3/envs/py3.7/bin/.gcc-5-post-link.sh: line 98: /home/kgagalova/miniconda3/envs/py3.7/lib/gcc/x86_64-conda-linux-gnu/11.2.0 /home/kgagalova/miniconda3/envs/py3.7/lib/gcc/x86_64-unknown-linux-gnu/5.2.0/specs: No such file or directory
sed: can't read /home/kgagalova/miniconda3/envs/py3.7/lib/gcc/x86_64-conda-linux-gnu/11.2.0 /home/kgagalova/miniconda3/envs/py3.7/lib/gcc/x86_64-unknown-linux-gnu/5.2.0/specs: No such file or directory
sed: can't read /home/kgagalova/miniconda3/envs/py3.7/lib/gcc/x86_64-conda-linux-gnu/11.2.0 /home/kgagalova/miniconda3/envs/py3.7/lib/gcc/x86_64-unknown-linux-gnu/5.2.0/specs: No such file or directory
sed: can't read /home/kgagalova/miniconda3/envs/py3.7/lib/gcc/x86_64-conda-linux-gnu/11.2.0 /home/kgagalova/miniconda3/envs/py3.7/lib/gcc/x86_64-unknown-linux-gnu/5.2.0/specs: No such file or directory
/home/kgagalova/miniconda3/envs/py3.7/gcc/libexec/gcc/x86_64-unknown-linux-gnu/5.2.0/cc1: error while loading shared libraries: libisl.so.10: cannot open shared object file: No such file or directory

return code: 1

()

Do you have any suggestion on how to install dysgu with pip3? Thank you in advance

kcleal commented 2 years ago

Hi @KristinaGagalova, As you say, it looks like the gcc-5 command is not working. Its possible your system has an older version of gcc that is functional, this may help https://askubuntu.com/questions/26498/how-to-choose-the-default-gcc-and-g-version If you have docker installed that might also work.

KristinaGagalova commented 2 years ago

Is there possible to provide an example of a Docker run? Thank you

kcleal commented 2 years ago

Yes of course. There is an example shown on dockerhub page: https://hub.docker.com/repository/docker/kcleal/dysgu/

An example workflow using docker:

# Make an 'input_folder', put your bam file and reference genome inside:

.
├── input_folder
│    ├──sample.bam
│    ├── reference.fasta
│    └── reference.fasta.fai
└── ...

# Make an 'output_folder':

mkdir output_folder

# Set up docker:

docker pull kcleal/dysgu
docker run -it \
     --mount src="${PWD}"/input_folder,target=/input_folder,type=bind \
     --mount src="${PWD}"/output_folder,target=/output_folder,type=bind \
     kcleal/dysgu
cd input_folder

# Run dysgu:

dysgu run reference.fasta wd sample.bam > ../output_folder/sample.vcf
KristinaGagalova commented 2 years ago

it worked fine, thank you.

kcleal commented 2 years ago

Good to hear