BinPro / CONCOCT

Clustering cONtigs with COverage and ComposiTion
Other
122 stars 48 forks source link

error with concoct (1.0.0) #245

Closed ucassee closed 5 years ago

ucassee commented 5 years ago

Hello developer, I install the the latest version of concoct(1.0.0), but there is an error to use it. Best regards Traceback (most recent call last): File "/data2017/share/tools/miniconda3/envs/concoct_env/bin/concoct", line 4, in import('pkg_resources').run_script('concoct==1.0.0', 'concoct') File "/data2017/share/tools/miniconda3/envs/concoct_env/lib/python3.5/site-packages/pkg_resources/init.py", line 661, in run_script self.require(requires)[0].run_script(script_name, ns) File "/data2017/share/tools/miniconda3/envs/concoct_env/lib/python3.5/site-packages/pkg_resources/init.py", line 1441, in run_script exec(code, namespace, namespace) File "/data2017/share/tools/miniconda3/envs/concoct_env/lib/python3.5/site-packages/concoct-1.0.0-py3.5-linux-x86_64.egg/EGG-INFO/scripts/concoct", line 6, in import vbgmm ImportError: libgsl.so.23: cannot open shared object file: No such file or directory

alneberg commented 5 years ago

That's weird. Did you install this using conda? It seems it is missing a gsl dependency. Can you specify the command you used and what packages that the conda environment include after the installation? Thanks!

ucassee commented 5 years ago

Hello @alneberg , I just use concoct -v to check the newly installed concoct. I install it using Anaconda follow the tutorial on your website. conda install cython numpy scipy biopython pandas pip scikit-learn

alneberg commented 5 years ago

Weird. Could you list the packages installed in your conda environment as listed by conda list? If that doesn't show anything strange I will attempt to reproduce this error.

ucassee commented 5 years ago

Hello @alneberg , The followings are packages installed in conda environment:

packages in environment at /data2017/share/tools/miniconda3:

#

Name Version Build Channel

asn1crypto 0.24.0 py36_0
ca-certificates 2018.03.07 0
certifi 2018.8.24 py36_1
cffi 1.11.5 py36h9745a5d_0
chardet 3.0.4 py36h0f667ec_1
conda 4.5.11 py36_0
conda-env 2.6.0 h36134e3_1
cryptography 2.2.2 py36h14c3975_0
h5py 2.8.0 idna 2.6 py36h82fb2a8_1
libedit 3.1.20170329 h6b74fdf_2
libffi 3.2.1 hd88cf55_4
libgcc-ng 7.2.0 hdf63c60_3
libstdcxx-ng 7.2.0 hdf63c60_3
metabat2 2.12.1 0 ursky ncurses 6.1 hf484d3e_0
numpy 1.15.4 ont-albacore 2.3.3 ont-fast5-api 1.0.1 openssl 1.0.2p h14c3975_0
pip 18.1 pip 10.0.1 py36_0
progressbar33 2.4 pycosat 0.6.3 py36h0a5515d_0
pycparser 2.18 py36hf9f622e_1
pyopenssl 18.0.0 py36_0
pysocks 1.6.8 py36_0
python 3.6.5 hc3d631a_2
python-dateutil 2.7.5 readline 7.0 ha6073c6_4
requests 2.18.4 py36he2e5f8d_1
ruamel_yaml 0.15.37 py36h14c3975_2
setuptools 39.2.0 py36_0
six 1.11.0 py36h372c433_1
sqlite 3.23.1 he433501_0
tk 8.6.7 hc745277_3
urllib3 1.22 py36hbe7ace6_0
wget 1.19.5 h1ad7b7a_0
wheel 0.31.1 py36_0
xz 5.2.4 h14c3975_4
yaml 0.1.7 had09818_2
zlib 1.2.11 ha838bed_2

alneberg commented 5 years ago

That doesn't look like the concoct environment. You will have to have the environment activated when running the command. Sorry for being unclear.

ucassee commented 5 years ago

Hi @alneberg

packages in environment at /data2017/share/tools/miniconda3/envs/concoct_env:

#

Name Version Build Channel

biopython 1.68 py35_0 bioconda blas 1.1 openblas conda-forge bzip2 1.0.6 h14c3975_1002 conda-forge ca-certificates 2018.11.29 ha4d7672_0 conda-forge certifi 2018.8.24 py35_1001 conda-forge concoct 1.0.0 cython 0.28.5 py35hfc679d8_0 conda-forge freetype 2.9.1 h94bbf69_1005 conda-forge jpeg 9c h14c3975_1001 conda-forge libffi 3.2.1 hf484d3e_1005 conda-forge libgcc-ng 7.3.0 hdf63c60_0 conda-forge libgfortran 3.0.0 1 conda-forge libpng 1.6.36 h84994c4_1000 conda-forge libstdcxx-ng 7.3.0 hdf63c60_0 conda-forge libtiff 4.0.10 h648cc4a_1001 conda-forge mmtf-python 1.0.2 py35_0 bioconda msgpack-python 0.5.6 py35h2d50403_3 conda-forge ncurses 6.1 hf484d3e_1002 conda-forge nose 1.3.7 numpy 1.15.2 py35_blas_openblashd3ea46f_0 [blas_openblas] conda-forge olefile 0.46 py_0 conda-forge openblas 0.2.20 8 conda-forge openssl 1.0.2p h14c3975_1002 conda-forge pandas 0.23.4 py35hf8a1672_0 conda-forge pillow 5.3.0 py35hc736899_0 conda-forge pip 18.0 py35_1001 conda-forge python 3.5.5 h5001a0f_2 conda-forge python-dateutil 2.8.0 py_0 conda-forge pytz 2018.9 py_0 conda-forge readline 7.0 hf8c457e_1001 conda-forge reportlab 3.5.8 py35h77bcf2c_0 conda-forge scikit-learn 0.20.0 py35_blas_openblasha84fab4_201 [blas_openblas] conda-forge scipy 1.1.0 py35_blas_openblash7943236_201 [blas_openblas] conda-forge setuptools 40.4.3 py35_0 conda-forge six 1.11.0 py35_1 conda-forge sqlite 3.26.0 h67949de_1000 conda-forge tk 8.6.9 h84994c4_1000 conda-forge wheel 0.32.0 py35_1000 conda-forge xz 5.2.4 h14c3975_1001 conda-forge zlib 1.2.11 h14c3975_1004 conda-forge

alneberg commented 5 years ago

Ah, sorry. I now realise that the installation documentation is really lacking. This specific issue was solved in the conda recipe to install concoct from conda:

conda create -n concoct_env python=3 concoct

But using the commands present in the documentation will result in this error as you have discovered. Could you try and reinstall concoct with the above command? I will make an effort to update the documentation. Thanks!

ucassee commented 5 years ago

I find an interesting thing. If I use the root account, I can use the concoct without error. But I will try to reinstall it, because it is not available to normal user.

ucassee commented 5 years ago

I reinstall it. There is still an error.

(concoct_env) [zhouyl@localhost ~]$ concoct -v Traceback (most recent call last): File "/data2017/share/tools/miniconda3/envs/concoct_env/bin/concoct", line 4, in import('pkg_resources').run_script('concoct==1.0.0', 'concoct') File "/data2017/share/tools/miniconda3/envs/concoct_env/lib/python3.5/site-packages/pkg_resources/init.py", line 661, in run_script self.require(requires)[0].run_script(script_name, ns) File "/data2017/share/tools/miniconda3/envs/concoct_env/lib/python3.5/site-packages/pkg_resources/init.py", line 1432, in run_script .format(**locals()), pkg_resources.ResolutionError: Script 'scripts/concoct' not found in metadata at '/data2017/share/tools/miniconda3/envs/concoct_env/lib/python3.5/site-packages/concoct-1.0.0-py3.5.egg-info'

the following is packages in new concoct_env: (concoct_env) [zhouyl@localhost ~]$ conda list

packages in environment at /data2017/share/tools/miniconda3/envs/concoct_env:

#

Name Version Build Channel

alabaster 0.7.12 py_0 conda-forge asn1crypto 0.24.0 py35_3 conda-forge babel 2.6.0 py_1 conda-forge biopython 1.68 py35_0 bioconda blas 1.1 openblas conda-forge bzip2 1.0.6 h14c3975_1002 conda-forge ca-certificates 2018.11.29 ha4d7672_0 conda-forge certifi 2018.8.24 py35_1001 conda-forge cffi 1.11.5 py35h5e8e0c9_1 conda-forge chardet 3.0.4 py35_3 conda-forge concoct 1.0.0 py35hd1ccf6d_1 bioconda concoct 1.0.0 cryptography 2.3.1 py35hdffb7b8_0 conda-forge cryptography-vectors 2.3.1 py35_0 conda-forge curl 7.64.0 h646f8bb_0 conda-forge cython 0.28.5 py35hfc679d8_0 conda-forge docutils 0.14 py35_1 conda-forge freetype 2.9.1 h94bbf69_1005 conda-forge gsl 2.2.1 blas_openblash47a8a8e_5 [blas_openblas] conda-forge idna 2.7 py35_2 conda-forge imagesize 1.1.0 py_0 conda-forge intel-openmp 2019.1 144
jinja2 2.10 py_1 conda-forge jpeg 9c h14c3975_1001 conda-forge krb5 1.16.3 hc83ff2d_1000 conda-forge libcurl 7.64.0 h01ee5af_0 conda-forge libdeflate 1.0 h14c3975_1 bioconda libedit 3.1.20170329 hf8c457e_1001 conda-forge libffi 3.2.1 hf484d3e_1005 conda-forge libgcc-ng 7.3.0 hdf63c60_0 conda-forge libgfortran 3.0.0 1 conda-forge libgfortran-ng 7.2.0 hdf63c60_3 conda-forge libopenblas 0.3.3 h5a2b251_3
libpng 1.6.36 h84994c4_1000 conda-forge libssh2 1.8.0 h1ad7b7a_1003 conda-forge libstdcxx-ng 7.3.0 hdf63c60_0 conda-forge libtiff 4.0.10 h648cc4a_1001 conda-forge llvm-meta 7.0.0 0 conda-forge markupsafe 1.0 py35h470a237_1 conda-forge mkl 2019.1 144
mkl_fft 1.0.9 py35_0 conda-forge mkl_random 1.0.1 py35_0 conda-forge mmtf-python 1.0.2 py35_0 bioconda msgpack-python 0.5.6 py35h2d50403_3 conda-forge ncurses 6.1 hf484d3e_1002 conda-forge nose 1.3.7 py35_2 conda-forge numpy 1.15.2 py35_blas_openblashd3ea46f_0 [blas_openblas] conda-forge numpy-base 1.15.2 py35h2f8d375_0
olefile 0.46 py_0 conda-forge openblas 0.2.20 8 conda-forge openmp 7.0.0 h2d50403_0 conda-forge openssl 1.0.2p h14c3975_1002 conda-forge packaging 19.0 py_0 conda-forge pandas 0.23.4 py35hf8a1672_0 conda-forge pillow 5.3.0 py35hc736899_0 conda-forge pip 18.0 py35_1001 conda-forge pycparser 2.19 py_0 conda-forge pygments 2.3.1 py_0 conda-forge pyopenssl 18.0.0 py35_0 conda-forge pyparsing 2.3.1 py_0 conda-forge pysocks 1.6.8 py35_2 conda-forge python 3.5.5 h5001a0f_2 conda-forge python-dateutil 2.8.0 py_0 conda-forge pytz 2018.9 py_0 conda-forge readline 7.0 hf8c457e_1001 conda-forge reportlab 3.5.8 py35h77bcf2c_0 conda-forge requests 2.19.1 py35_1 conda-forge samtools 1.9 h57cc563_6 bioconda scikit-learn 0.20.0 py35_blas_openblasha84fab4_201 [blas_openblas] conda-forge scipy 1.1.0 py35_blas_openblash7943236_201 [blas_openblas] conda-forge setuptools 40.4.3 py35_0 conda-forge six 1.11.0 py35_1 conda-forge snowballstemmer 1.2.1 py_1 conda-forge sphinx 1.8.1 py35_0 conda-forge sphinx_rtd_theme 0.4.3 py_0 conda-forge sphinxcontrib-websupport 1.1.0 py_1 conda-forge sqlite 3.26.0 h67949de_1000 conda-forge tk 8.6.9 h84994c4_1000 conda-forge urllib3 1.23 py35_1 conda-forge wheel 0.32.0 py35_1000 conda-forge xz 5.2.4 h14c3975_1001 conda-forge zlib 1.2.11 h14c3975_1004 conda-forge

alneberg commented 5 years ago

Well now it seems like you have two concoct installed in parallel. If you create a new environment and only run the conda command (and not any setup.py command) it should work.

ucassee commented 5 years ago

Hi @alneberg , It is work now. Thank you very much. Could you please help me to solve the input file problem you mentioned before. It also occurs in the new concoct version. Up and running. Check /dataNew/User/zhouyl/sediment/metaWRAP/B11/log.txt for progress /data2017/share/tools/miniconda3/envs/concoct_env/lib/python3.6/site-packages/concoct/input.py:82: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'. cov = p.read_table(cov_file, header=0, index_col=0) Traceback (most recent call last): File "/data2017/share/tools/miniconda3/envs/concoct_env/bin/concoct", line 88, in <module> results = main(args) File "/data2017/share/tools/miniconda3/envs/concoct_env/bin/concoct", line 40, in main args.seed File "/data2017/share/tools/miniconda3/envs/concoct_env/lib/python3.6/site-packages/concoct/transform.py", line 5, in perform_pca pca_object = PCA(n_components=nc, random_state=seed).fit(d) File "/data2017/share/tools/miniconda3/envs/concoct_env/lib/python3.6/site-packages/sklearn/decomposition/pca.py", line 340, in fit self._fit(X) File "/data2017/share/tools/miniconda3/envs/concoct_env/lib/python3.6/site-packages/sklearn/decomposition/pca.py", line 381, in _fit copy=self.copy) File "/data2017/share/tools/miniconda3/envs/concoct_env/lib/python3.6/site-packages/sklearn/utils/validation.py", line 582, in check_array context)) ValueError: Found array with 0 sample(s) (shape=(0, 139)) while a minimum of 1 is required.

alneberg commented 5 years ago

That's great!

Yes, I think it is caused by a mismatch of the lefthand column of the coverage table and the contig ids in the fasta file. Could you give me the first five contig names of the fasta file and see if these are also listed in the coverage file?

ucassee commented 5 years ago

the first five lines in coverage file are: 1 18.1576 2.85688 2 11.9123 7.08195 3 11.8578 7.6306 4 12.3037 7.30384 5 11.8619 7.26513

The first five contig names of the fasta file is

1 >2 >3 >4 >5

alneberg commented 5 years ago

Ah, it might be an old bug, where concoct doesn't allow contig ids to be only integers. Would that explain it?

ucassee commented 5 years ago

It is work after I replaced the contig ids. Thanks for your help~

alneberg commented 5 years ago

Ah, that's a relief! Good luck with your binning!