search

BinPro / CONCOCT

Clustering cONtigs with COverage and ComposiTion
Other
122 stars 48 forks source link

endless OpenBLAS Warning #232

Closed kodragonP closed 5 years ago

kodragonP commented 5 years ago

I installed concoct using conda, and when I run concoct main progress, endless "OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option. " would be printed. What happened?

alneberg commented 5 years ago

Hmm, that's not good. I've seen this once before but thought I was able to resolve it. I Was this in a fresh conda environment? What's your system? Linux, Mac?

kodragonP commented 5 years ago

Thank you for your reply! I install concoct in a fresh environment, and I run it on CentOS. I tried to rebuild the library with USE_OPENMP=1 option, but it not work. It seems that the software could run and get bins. I tried a very small dataset. However, the speed is still very slow. I have 856K contigs (8 samples), and the software had ran more than 1200 hours·threads. There was still no evidence when the progress would end.

alneberg commented 5 years ago

Yes I think this error could cause it to go slow and potentially the openmp is not working and thus it is not parallel at all. Maybe you should try to get this error to disappear befor making it run on a production data set?

Could you please give me the output from:

conda env export

From when you have activated the conda environment. Thank you!

kodragonP commented 5 years ago

name: concoct channels:

alneberg commented 5 years ago

I can confirm I get the same error on our cluster with CentOS. You should probably cancel this run since it is likely to be extremely slow due to this error. I will look into why this is happening and come back to you.

kodragonP commented 5 years ago

Waiting for your good news! ^_^

alneberg commented 5 years ago

Ok, so not so good news. It seems like OpenMP is disabled in the conda available versions of OpenBLAS. I think I got around it by downgrading the openblas to the version before it was disabled. But in order to get it working I think you need to recompile concoct:

git clone https://github.com/BinPro/CONCOCT.git cd CONCOCT conda uninstall concoct conda install -c conda-forge openblas=0.3.3 python setup.py install

Please let me know if you manage to get this to work. If I can get this confirmed I can fix the version of openblas in the conda recipe as well.

Thank you!

kodragonP commented 5 years ago

Hi, I think it's OK now. The standard output was just like this:

Generate input data 0,-2014653.435232,70441.097680 1,-1980568.436301,34084.998932

10 K contigs only need 45 min. Thank you!

alneberg commented 5 years ago

That's a relief. I'll update the bioconda recipe accordingly. Thank you!

ursky commented 5 years ago

I am unfortunately getting the same issue (repeating OpenBLAS Warning) in the metaWRAP conda environment:

name: metawrap-bare
channels:
  - ursky
  - bioconda
  - conda-forge
  - defaults
dependencies:
  - _r-mutex=1.0.0=anacondar_1
  - alabaster=0.7.12=py_0
  - aragorn=1.2.38=h470a237_2
  - asn1crypto=0.24.0=py27_1003
  - babel=2.6.0=py_1
  - backports=1.0=py_2
  - backports.functools_lru_cache=1.5=py_1
  - backports_abc=0.5=py_1
  - barrnap=0.9=2
  - bcftools=1.6=0
  - bedtools=2.27.1=he860b03_3
  - binutils_impl_linux-64=2.28.1=had2808c_3
  - binutils_linux-64=7.2.0=had2808c_27
  - biopython=1.68=py27_0
  - blas=1.1=openblas
  - blast=2.6.0=boost1.64_2
  - bmfilter=3.101=hfc679d8_2
  - bmtagger=3.101=h470a237_4
  - bmtool=3.101=hfc679d8_2
  - boost=1.64.0=py27_4
  - boost-cpp=1.64.0=1
  - bowtie2=2.3.4.3=py27h2d50403_0
  - brewer2mpl=1.4.1=py_3
  - bwa=0.7.15=1
  - bwidget=1.9.11=1
  - bz2file=0.98=py_0
  - bzip2=1.0.6=h14c3975_1002
  - ca-certificates=2018.11.29=ha4d7672_0
  - cairo=1.14.12=h80bd089_1005
  - certifi=2018.11.29=py27_1000
  - cffi=1.12.1=py27h9745a5d_0
  - chardet=3.0.4=py27_1003
  - checkm-genome=1.0.13=py27_0
  - concoct=1.0.0=py27h63c6309_2
  - cryptography=2.5=py27h1ba5d50_0
  - curl=7.62.0=hbc83047_0
  - cutadapt=1.18=py27h14c3975_1
  - cycler=0.10.0=py_1
  - cython=0.29.5=py27hf484d3e_0
  - dbus=1.13.2=h714fa37_1
  - dendropy=4.4.0=py_0
  - docutils=0.14=py27_1001
  - enum34=1.1.6=py27_1001
  - expat=2.2.5=hf484d3e_1002
  - extract_fullseq=3.101=3
  - fastqc=0.11.5=1
  - fontconfig=2.13.1=h2176d3f_1000
  - fraggenescan=1.31=h470a237_0
  - freetype=2.9.1=h94bbf69_1005
  - functools32=3.2.3.2=py_3
  - futures=3.2.0=py27_1000
  - gcc_impl_linux-64=7.2.0=habb00fd_3
  - gcc_linux-64=7.2.0=h550dcbe_27
  - gfortran_impl_linux-64=7.2.0=hdf63c60_3
  - gfortran_linux-64=7.2.0=h550dcbe_27
  - glib=2.56.2=hd408876_0
  - gmp=6.1.2=hf484d3e_1000
  - gnutls=3.5.19=h2a4e5f8_1
  - graphite2=1.3.13=hf484d3e_1000
  - gsl=2.2.1=blas_openblashddceaf2_6
  - gst-plugins-base=1.14.0=hbbd80ab_1
  - gstreamer=1.14.0=hb453b48_1
  - gxx_impl_linux-64=7.2.0=hdf63c60_3
  - gxx_linux-64=7.2.0=h550dcbe_27
  - harfbuzz=1.9.0=he243708_1001
  - hmmer=3.1b2=3
  - htslib=1.6=0
  - icu=58.2=hf484d3e_1000
  - idba=1.1.3=1
  - idna=2.8=py27_1000
  - imagesize=1.1.0=py_0
  - infernal=1.1.2=h14c3975_2
  - ipaddress=1.0.22=py_1
  - java-jdk=8.0.92=1
  - jellyfish=1.1.12=h2d50403_0
  - jemalloc=4.5.0=0
  - jinja2=2.10=py_1
  - jpeg=9c=h14c3975_1001
  - kiwisolver=1.0.1=py27h6bb024c_1002
  - kraken=1.1=h470a237_2
  - krb5=1.14.6=0
  - krona=2.7=pl526_2
  - libcurl=7.62.0=h20c2e04_0
  - libffi=3.2.1=hf484d3e_1005
  - libgcc=7.2.0=h69d50b8_2
  - libgcc-ng=7.3.0=hdf63c60_0
  - libgfortran=3.0.0=1
  - libgfortran-ng=7.2.0=hdf63c60_3
  - libiconv=1.14=4
  - libidn11=1.33=0
  - libopenblas=0.2.20=h9ac9557_7
  - libpng=1.6.36=h84994c4_1000
  - libssh2=1.8.0=1
  - libstdcxx-ng=7.3.0=hdf63c60_0
  - libtiff=4.0.10=h648cc4a_1001
  - libuuid=2.32.1=h14c3975_1000
  - libxcb=1.13=h14c3975_1002
  - libxml2=2.9.9=he19cac6_0
  - llvm-meta=7.0.0=0
  - markupsafe=1.1.0=py27h14c3975_1000
  - matplotlib=2.2.3=py27h8a2030e_1
  - matplotlib-base=2.2.3=py27h60b886d_1
  - maxbin2=2.2.5=0
  - megahit=1.1.3=py27_0
  - metabat2=2.12.1=0
  - minced=0.3.2=0
  - mkl=11.3.3=0
  - mmtf-python=1.0.2=py27_0
  - msgpack-python=0.6.1=py27h6bb024c_0
  - ncurses=6.1=hf484d3e_1002
  - nettle=3.3=0
  - nose=1.3.7=py27_1002
  - numpy=1.16.1=py27_blas_openblash1522bff_0
  - olefile=0.46=py_0
  - openblas=0.3.3=ha44fe06_1
  - openjdk=11.0.1=h14c3975_1014
  - openmp=7.0.0=h2d50403_0
  - openssl=1.1.1a=h14c3975_1000
  - packaging=19.0=py_0
  - pandas=0.23.4=py27h637b7d7_1000
  - pango=1.40.14=hf0c64fd_1003
  - parallel=20160622=1
  - patsy=0.5.1=py_0
  - pcre=8.42=h439df22_0
  - perl=5.26.2=h14c3975_1002
  - perl-app-cpanminus=1.7044=pl526_1
  - perl-bioperl=1.6.924=4
  - perl-carp=1.38=pl526_1
  - perl-constant=1.33=pl526_1
  - perl-encode=2.88=pl526_1
  - perl-encode-locale=1.05=pl526_6
  - perl-exporter=5.72=pl526_1
  - perl-extutils-makemaker=7.34=pl526_3
  - perl-file-path=2.15=pl526_0
  - perl-file-temp=0.2304=pl526_2
  - perl-lwp-simple=6.15=pl526h470a237_4
  - perl-parent=0.236=pl526_1
  - perl-threaded=5.22.0=13
  - perl-xml-namespacesupport=1.12=pl526_0
  - perl-xml-parser=2.44=pl526h3a4f0e9_6
  - perl-xml-sax=1.00=pl526_0
  - perl-xml-sax-base=1.09=pl526_0
  - perl-xml-sax-expat=0.51=pl526_2
  - perl-xml-simple=2.25=pl526_0
  - perl-yaml=1.27=pl526_0
  - pigz=2.3.4=0
  - pillow=5.4.1=py27h00a061d_1000
  - pip=19.0.3=py27_0
  - pixman=0.34.0=h14c3975_1003
  - pplacer=1.1.alpha17=0
  - prodigal=2.6.3=1
  - prokka=1.13=0
  - pthread-stubs=0.4=h14c3975_1001
  - pycairo=1.16.3=py27_0
  - pycparser=2.19=py_0
  - pygments=2.3.1=py_0
  - pyopenssl=19.0.0=py27_0
  - pyparsing=2.3.1=py_0
  - pyqt=5.6.0=py27h13b7fb3_1008
  - pysam=0.13.0=py27_htslib1.6_0
  - pysocks=1.6.8=py27_1002
  - python=2.7.15=h9bab390_6
  - python-dateutil=2.8.0=py_0
  - pytz=2018.9=py_0
  - qt=5.6.3=h8bf5577_3
  - quast=4.1=py27_0
  - r-assertthat=0.2.0=r343h889e2dd_0
  - r-base=3.4.3=h9bb98a2_5
  - r-boot=1.3_20=r343h889e2dd_0
  - r-class=7.3_14=r343h086d26f_4
  - r-cli=1.0.0=r343h6115d3f_1
  - r-cluster=2.0.6=r343h4829c52_0
  - r-codetools=0.2_15=r343h889e2dd_0
  - r-colorspace=1.3_2=r343h086d26f_0
  - r-crayon=1.3.4=r343h889e2dd_0
  - r-dichromat=2.0_0=r343h889e2dd_4
  - r-digest=0.6.13=r343h086d26f_0
  - r-foreign=0.8_69=r343h086d26f_0
  - r-ggplot2=2.2.1=r343h889e2dd_0
  - r-glue=1.2.0=r343h086d26f_0
  - r-gtable=0.2.0=r343h889e2dd_0
  - r-kernsmooth=2.23_15=r343h4829c52_4
  - r-labeling=0.3=r343h889e2dd_4
  - r-lattice=0.20_35=r343h086d26f_0
  - r-lazyeval=0.2.1=r343h086d26f_0
  - r-magrittr=1.5=r343h889e2dd_4
  - r-mass=7.3_48=r343h086d26f_0
  - r-matrix=1.2_12=r343h086d26f_0
  - r-mgcv=1.8_22=r343h086d26f_0
  - r-munsell=0.4.3=r343h889e2dd_0
  - r-nlme=3.1_131=r343h4829c52_0
  - r-nnet=7.3_12=r343h086d26f_0
  - r-pillar=1.0.1=r343h889e2dd_0
  - r-plyr=1.8.4=r343h599a50d_0
  - r-r6=2.2.2=r343h889e2dd_0
  - r-rcolorbrewer=1.1_2=r343h889e2dd_0
  - r-rcpp=0.12.14=r343h599a50d_0
  - r-recommended=3.4.3=r343_0
  - r-reshape2=1.4.3=r343h599a50d_0
  - r-rlang=0.1.6=r343h086d26f_0
  - r-rpart=4.1_11=r343h086d26f_0
  - r-scales=0.5.0=r343h599a50d_0
  - r-spatial=7.3_11=r343h086d26f_4
  - r-stringi=1.1.6=r343h599a50d_0
  - r-stringr=1.2.0=r343h889e2dd_0
  - r-survival=2.41_3=r343h086d26f_0
  - r-tibble=1.4.1=r343h086d26f_0
  - r-utf8=1.1.2=r343h086d26f_0
  - r-viridislite=0.2.0=r343h889e2dd_0
  - readline=7.0=hf8c457e_1001
  - reportlab=3.4.0=py27_0
  - requests=2.21.0=py27_1000
  - salmon=0.10.1=1
  - samtools=1.6=h02bfda8_2
  - scikit-learn=0.20.2=py27_blas_openblashebff5e3_1400
  - scipy=1.2.1=py27_blas_openblash1522bff_0
  - seaborn=0.8.1=py_1
  - setuptools=40.8.0=py27_0
  - singledispatch=3.4.0.3=py27_1000
  - sip=4.18=py27_1
  - six=1.12.0=py27_1000
  - snowballstemmer=1.2.1=py_1
  - spades=3.13.0=0
  - sphinx=1.8.4=py27_0
  - sphinx_rtd_theme=0.4.3=py_0
  - sphinxcontrib-websupport=1.1.0=py_1
  - sqlite=3.26.0=h67949de_1000
  - srprism=2.4.24=2
  - statsmodels=0.9.0=py27h3010b51_1000
  - subprocess32=3.2.7=py27_0
  - taxator-tk=1.3.3e=0
  - tbb=2019.3=h6bb024c_1000
  - tbl2asn=25.6=3
  - tk=8.6.9=h84994c4_1000
  - tktable=2.10=h14c3975_0
  - tornado=5.1.1=py27h14c3975_1000
  - trim-galore=0.4.5=2
  - typing=3.5.2.2=py27_0
  - urllib3=1.24.1=py27_1000
  - wheel=0.33.1=py27_0
  - xopen=0.5.0=py_0
  - xorg-kbproto=1.0.7=h14c3975_1002
  - xorg-libice=1.0.9=h14c3975_1004
  - xorg-libsm=1.2.3=h4937e3b_1000
  - xorg-libx11=1.6.7=h14c3975_1000
  - xorg-libxau=1.0.9=h14c3975_0
  - xorg-libxdmcp=1.1.2=h14c3975_1007
  - xorg-libxext=1.3.3=h14c3975_1004
  - xorg-libxrender=0.9.10=h14c3975_1002
  - xorg-renderproto=0.11.1=h14c3975_1002
  - xorg-xextproto=7.3.0=h14c3975_1002
  - xorg-xproto=7.0.31=h14c3975_1007
  - xz=5.2.4=h14c3975_1001
  - zlib=1.2.11=h14c3975_1004
prefix: /home/guritsk1/miniconda2/envs/metawrap-bare

Please advise.

ursky commented 5 years ago

I should also mention that I see this behavior ONLY when I use more than one thread.

alneberg commented 5 years ago

I'm not exactly sure what package that is triggering this error. You seem to have the correct version of openblas as specified in the conda recipe. I will check if this error still persists in the latest version of openblas. Does this occur if you install concoct in an isolated environment as well?

ursky commented 5 years ago

Good question. I just installed it in a clear conda environment and still got the same issue, which is good because now there are less distracting factors:

name: concoct-env
channels:
  - ursky
  - bioconda
  - conda-forge
  - defaults
dependencies:
  - biopython=1.68=py27_0
  - blas=1.0=mkl
  - ca-certificates=2018.11.29=ha4d7672_0
  - certifi=2018.11.29=py27_1000
  - concoct=1.0.0=py27h63c6309_2
  - cython=0.29.5=py27hf484d3e_0
  - freetype=2.9.1=h94bbf69_1005
  - gsl=2.2.1=h0c605f7_3
  - intel-openmp=2019.1=144
  - jpeg=9c=h14c3975_1001
  - libffi=3.2.1=hf484d3e_1005
  - libgcc-ng=7.3.0=hdf63c60_0
  - libgfortran=3.0.0=1
  - libgfortran-ng=7.2.0=hdf63c60_3
  - libpng=1.6.36=h84994c4_1000
  - libstdcxx-ng=7.3.0=hdf63c60_0
  - libtiff=4.0.10=h648cc4a_1001
  - llvm-meta=7.0.0=0
  - mkl=2019.1=144
  - mkl_fft=1.0.10=py27h14c3975_1
  - mkl_random=1.0.2=py27h637b7d7_2
  - mmtf-python=1.0.2=py27_0
  - msgpack-python=0.6.1=py27h6bb024c_0
  - ncurses=6.1=hf484d3e_1002
  - nose=1.3.7=py27_1002
  - numpy=1.15.4=py27h7e9f1db_0
  - numpy-base=1.15.4=py27hde5b4d6_0
  - olefile=0.46=py_0
  - openblas=0.3.3=ha44fe06_1
  - openmp=7.0.0=h2d50403_0
  - openssl=1.1.1a=h14c3975_1000
  - pandas=0.24.1=py27hf484d3e_0
  - pillow=5.4.1=py27h00a061d_1000
  - pip=19.0.3=py27_0
  - python=2.7.15=h9bab390_6
  - python-dateutil=2.8.0=py_0
  - pytz=2018.9=py_0
  - readline=7.0=hf8c457e_1001
  - reportlab=3.5.13=py27hbd3ef63_1000
  - samtools=1.3.1=0
  - scikit-learn=0.20.2=py27hd81dba3_0
  - scipy=1.2.1=py27h7c811a0_0
  - setuptools=40.8.0=py27_0
  - six=1.12.0=py27_1000
  - sqlite=3.26.0=h67949de_1000
  - tk=8.6.9=h84994c4_1000
  - wheel=0.33.1=py27_0
  - xz=5.2.4=h14c3975_1001
  - zlib=1.2.11=h14c3975_1004
prefix: /home/guritsk1/miniconda2/envs/concoct-env
alneberg commented 5 years ago

Ok, I agree, less distracting factors. I have to have a look at this right away.

alneberg commented 5 years ago

I am indeed able to reproduce this problem. I was however able to solve it locally by upgrading openblas to 0.3.5 instead of 0.3.3 and reinstall concoct with python setup.py install. I will make this the default behaviour of the conda package and hope that this solves the problem.

kodragonP commented 5 years ago

I think the problem is not the version of openblas. If you reinstall openblas after all packages installed, the problem will be solved.

alneberg commented 5 years ago

Hi @kodragonP, sorry I'm not following completely. Are you saying that if you reinstall the same version of openblas and still using conda, the problem is solved? You then have to uninstall both concoct and openblas first right?

If you have the commands to reproduce this I would be very interested.

alneberg commented 5 years ago

I think I managed to fix the current bioconda recipe. @ursky, please could you verify that the latest build of concoct works for you? I've tested it on a fresh ubuntu container with python 2.7 and 3.7, and the '-t' parameter is now functional.

ursky commented 5 years ago

That actually broke it, i think:

(metawrap-test) guritsk1@comp2:~$ concoct
Traceback (most recent call last):
  File "/home/guritsk1/miniconda2/envs/metawrap-test/bin/concoct", line 6, in <module>
    import vbgmm
ImportError: libgslcblas.so.0: cannot open shared object file: No such file or directory
ursky commented 5 years ago

The issue is that the libgslblas.so.0 symlink points to something does not exist. I can manually configure the symlink to point to libgslcblas.so like below. While this works, ideally this should not be necessary for the end user to do themselves.

rm ${conda_path}/lib/libgslcblas.so.0
ln ${conda_path}/lib/libgslcblas.so ${conda_path}/lib/libgslcblas.so.0
slambrechts commented 5 years ago

I'm having the same problem. Should I manually configure the symlink as ursky suggested (and if so, can I literally copy paste those commands?) Or should I wait for concoct to finish and ignore the warnings? Or cancel the run and try again when the problem has been fixed?

Ash1One commented 5 years ago

I'm having the same problem. Should I manually configure the symlink as ursky suggested (and if so, can I literally copy paste those commands?) Or should I wait for concoct to finish and ignore the warnings? Or cancel the run and try again when the problem has been fixed?

I think you should cancel it and try to configure the symlink. I had this problem last week, too. Then I updated metawrap to 1.18, and it worked.😂

Ash1One commented 5 years ago

I'm having the same problem. Should I manually configure the symlink as ursky suggested (and if so, can I literally copy paste those commands?) Or should I wait for concoct to finish and ignore the warnings? Or cancel the run and try again when the problem has been fixed?

I think you should cancel it and try to configure the symlink. I had this problem last week, too. Then I updated metawrap to 1.18, and it worked.😂

But I met another problem that getting only one and big unbinning file when I ran CONCOCT by 'metawrap binning --CONCOCT'. Then I ran CONCOCT solely, and it was normal. I still don't understand.

ursky commented 5 years ago

Can you give some details on the issue? Also, if concoct is already running and not exiting, you should be ok without restarting.

slambrechts commented 5 years ago

I am running CONCOCT in the metaWRAP 1.1.8 conda environment, and as far as I can tell concoct is still very active and not exiting (lots of activity on htop, using all the specified threads or more, and 4.6G Mem). The concoct_out/log.txt file lists the following output:

2019-04-18 08:19:01,380:INFO:root:Results created at /home/sjalambr/Shotgun_metagenomics_pilot/INITIAL_BINNING/work_files/concoct_out
2019-04-18 08:29:30,581:INFO:root:Successfully loaded composition data.
2019-04-18 08:29:33,270:INFO:root:Successfully loaded coverage data.
2019-04-18 08:29:52,811:INFO:root:Performed PCA, resulted in 42 dimensions
2019-04-18 08:32:59,472:INFO:root:Wrote original filtered data file.
2019-04-18 08:33:58,090:INFO:root:Wrote PCA transformed file.
2019-04-18 08:33:58,096:INFO:root:Wrote PCA components file.
2019-04-18 08:33:58,096:INFO:root:PCA transformed data.
2019-04-18 08:33:58,097:INFO:root:Will call vbgmm with parameters: INITIAL_BINNING/work_files/concoct_out/, 400, 1000, 14

That last entry: 2019-04-18 08:33:58,097:INFO:root:Will call vbgmm with parameters: INITIAL_BINNING/work_files/concoct_out/, 400, 1000, 14 is however from more then 6 hours ago, should I be worried that it's not really running properly anymore? Or is it completely normal I don't see anything happening in the log file for a while? My assembly file is 1.13 gigabytes, so relatively big I guess

ursky commented 5 years ago

Thats quite a large assembly, expect it to take a long time.

ursky commented 5 years ago

Does anyone have any insight about what actually causes into the OpenBLAS Warning : Detect OpenMP Loop endless warning issue? Even with the updated concoct and openblas=0.3.5 i sometimes get it in some environments, but not others. Very frustrating.

An example of an environment that produces the error:

_r-mutex                  1.0.0               anacondar_1
aragorn                   1.2.38               h470a237_2    bioconda
backports                 1.0                        py_2    conda-forge
backports.functools_lru_cache 1.5                        py_1    conda-forge
backports_abc             0.5                        py_1    conda-forge
barrnap                   0.9                           2    bioconda
bcftools                  1.9                  ha228f0b_3    bioconda
bedtools                  2.28.0               hdf88d34_0    bioconda
binutils_impl_linux-64    2.31.1               h6176602_1
binutils_linux-64         2.31.1               h6176602_3
biopython                 1.68                     py27_0    bioconda
blas                      2.5                    openblas    conda-forge
blast                     2.6.0               boost1.64_2    bioconda
bmfilter                  3.101                hfc679d8_2    bioconda
bmtagger                  3.101                h470a237_4    bioconda
bmtool                    3.101                hfc679d8_2    bioconda
boost                     1.64.0                   py27_4    conda-forge
boost-cpp                 1.64.0                        1    conda-forge
bowtie2                   2.3.5            py27he860b03_0    bioconda
bwa                       0.7.17               h84994c4_5    bioconda
bwidget                   1.9.11                        1
bz2file                   0.98                       py_0    conda-forge
bzip2                     1.0.6             h14c3975_1002    conda-forge
ca-certificates           2019.3.9             hecc5488_0    conda-forge
cairo                     1.16.0            ha4e643d_1000    conda-forge
certifi                   2019.3.9                 py27_0    conda-forge
checkm-genome             1.0.12                   py27_0    bioconda
circos                    0.69.6                        4    bioconda
concoct                   1.0.0            py27h7724fef_4    bioconda
curl                      7.64.1               hf8cf82a_0    conda-forge
cutadapt                  1.18             py27h14c3975_1    bioconda
cycler                    0.10.0                     py_1    conda-forge
cython                    0.29.7           py27he1b5a44_0    conda-forge
dbus                      1.13.6               he372182_0    conda-forge
dendropy                  4.4.0                      py_1    bioconda
expat                     2.2.5             hf484d3e_1002    conda-forge
extract_fullseq           3.101                         3    bioconda
fastqc                    0.11.8                        1    bioconda
font-ttf-dejavu-sans-mono 2.37                 h6964260_0
fontconfig                2.13.1            he4413a7_1000    conda-forge
fraggenescan              1.31                 h470a237_0    bioconda
freetype                  2.10.0               he983fc9_0    conda-forge
functools32               3.2.3.2                    py_3    conda-forge
futures                   3.2.0                 py27_1000    conda-forge
gcc_impl_linux-64         7.3.0                habb00fd_1    conda-forge
gcc_linux-64              7.3.0                h553295d_3    conda-forge
gettext                   0.19.8.1          hc5be6a0_1002    conda-forge
gfortran_impl_linux-64    7.3.0                hdf63c60_1
gfortran_linux-64         7.3.0                h553295d_3
giflib                    5.1.9                h516909a_0    conda-forge
glib                      2.58.3            hf63aee3_1001    conda-forge
glimmerhmm                3.0.4                h2d50403_2    bioconda
graphite2                 1.3.13            hf484d3e_1000    conda-forge
gsl                       2.4               h294904e_1006    conda-forge
gst-plugins-base          1.14.4            hdf3bae2_1001    conda-forge
gstreamer                 1.14.4            h66beb1c_1001    conda-forge
gxx_impl_linux-64         7.3.0                hdf63c60_1    conda-forge
gxx_linux-64              7.3.0                h553295d_3    conda-forge
harfbuzz                  2.4.0                h37c48d4_0    conda-forge
hmmer                     3.2.1                hf484d3e_1    bioconda
htslib                    1.9                  ha228f0b_7    bioconda
icu                       58.2              hf484d3e_1000    conda-forge
idba                      1.1.3                         1    bioconda
infernal                  1.1.2                h14c3975_2    bioconda
jellyfish                 1.1.12               h2d50403_0    bioconda
jemalloc                  5.2.0                he1b5a44_0    conda-forge
joblib                    0.13.2                     py_0    conda-forge
jpeg                      9c                h14c3975_1001    conda-forge
kiwisolver                1.0.1           py27h6bb024c_1002    conda-forge
kraken                    1.1                  h470a237_2    bioconda
krb5                      1.16.3            h05b26f9_1001    conda-forge
krona                     2.7.1                   pl526_0    bioconda
libblas                   3.8.0                5_openblas    conda-forge
libboost                  1.67.0               h46d08c1_4
libcblas                  3.8.0                5_openblas    conda-forge
libcurl                   7.64.1               hda55be3_0    conda-forge
libdeflate                1.0                  h14c3975_1    bioconda
libedit                   3.1.20170329      hf8c457e_1001    conda-forge
libffi                    3.2.1             he1b5a44_1006    conda-forge
libgcc                    7.2.0                h69d50b8_2    conda-forge
libgcc-ng                 8.2.0                hdf63c60_1
libgd                     2.2.5             h0d07dcb_1005    conda-forge
libgfortran-ng            7.3.0                hdf63c60_0
libiconv                  1.15              h516909a_1005    conda-forge
libidn11                  1.34                 h1cef754_0    conda-forge
liblapack                 3.8.0                5_openblas    conda-forge
liblapacke                3.8.0                5_openblas    conda-forge
libpng                    1.6.37               hed695b0_0    conda-forge
libssh2                   1.8.2                h22169c7_2    conda-forge
libstdcxx-ng              8.2.0                hdf63c60_1
libtiff                   4.0.10            h648cc4a_1001    conda-forge
libuuid                   2.32.1            h14c3975_1000    conda-forge
libwebp                   1.0.2                h576950b_1    conda-forge
libxcb                    1.13              h14c3975_1002    conda-forge
libxml2                   2.9.9                h13577e0_0    conda-forge
llvm-openmp               8.0.0                hc9558a2_0    conda-forge
make                      4.2.1             h14c3975_2004    conda-forge
matplotlib                2.2.3            py27hb69df0a_0
maxbin2                   2.2.6                h14c3975_0    bioconda
megahit                   1.1.3                    py27_0    bioconda
metabat2                  2.12.1                        0    ursky
metawrap                  1.2                           0    bioconda
minced                    0.3.2                         0    bioconda
mmtf-python               1.0.2                    py27_0    bioconda
msgpack-python            0.6.1            py27h6bb024c_0    conda-forge
ncurses                   6.1               hf484d3e_1002    conda-forge
nose                      1.3.7                 py27_1002    conda-forge
numpy                     1.16.3           py27he5ce36f_0    conda-forge
olefile                   0.46                       py_0    conda-forge
openblas                  0.3.5             h9ac9557_1001    conda-forge
openjdk                   11.0.1            h516909a_1015    conda-forge
openmp                    8.0.0                         0    conda-forge
openssl                   1.1.1b               h14c3975_1    conda-forge
pandas                    0.24.2           py27hf484d3e_0    conda-forge
pango                     1.40.14           h4ea9474_1004    conda-forge
parallel                  20160622                      1    bioconda
patsy                     0.5.1                      py_0    conda-forge
pcre                      8.41              hf484d3e_1003    conda-forge
perl                      5.26.2            h516909a_1006    conda-forge
perl-app-cpanminus        1.7044                  pl526_1    bioconda
perl-autoloader           5.74                    pl526_2    bioconda
perl-bioperl              1.6.924                       4    bioconda
perl-carp                 1.38                    pl526_2    bioconda
perl-clone                0.41            pl526h470a237_0    bioconda
perl-config-general       2.63                    pl526_0    bioconda
perl-constant             1.33                    pl526_1    bioconda
perl-digest-perl-md5      1.9                     pl526_1    bioconda
perl-dynaloader           1.25                    pl526_1    bioconda
perl-encode               2.88                    pl526_1    bioconda
perl-encode-locale        1.05                    pl526_6    bioconda
perl-exporter             5.72                    pl526_1    bioconda
perl-exporter-tiny        1.002001                pl526_0    bioconda
perl-extutils-makemaker   7.34                    pl526_3    bioconda
perl-file-path            2.16                    pl526_0    bioconda
perl-file-temp            0.2304                  pl526_2    bioconda
perl-font-ttf             1.06                    pl526_0    bioconda
perl-gd                   2.70            pl526he941832_0    bioconda
perl-io-string            1.08                    pl526_3    bioconda
perl-list-moreutils       0.428                   pl526_1    bioconda
perl-list-moreutils-xs    0.428                   pl526_0    bioconda
perl-lwp-simple           6.15            pl526h470a237_4    bioconda
perl-math-bezier          0.01                    pl526_1    bioconda
perl-math-round           0.07                    pl526_1    bioconda
perl-math-vecstat         0.08                    pl526_1    bioconda
perl-module-implementation 0.09                    pl526_2    bioconda
perl-module-runtime       0.016                   pl526_0    bioconda
perl-number-format        1.75                    pl526_3    bioconda
perl-params-validate      1.29            pl526h470a237_0    bioconda
perl-parent               0.236                   pl526_1    bioconda
perl-pathtools            3.75            pl526h14c3975_0    bioconda
perl-readonly             2.05                    pl526_0    bioconda
perl-regexp-common        2017060201              pl526_0    bioconda
perl-scalar-list-utils    1.45            pl526h470a237_3    bioconda
perl-set-intspan          1.19                    pl526_1    bioconda
perl-statistics-basic     1.6611                  pl526_2    bioconda
perl-svg                  2.84                    pl526_0    bioconda
perl-text-format          0.59                    pl526_2    bioconda
perl-threaded             5.26.0                        0    bioconda
perl-time-hires           1.9758          pl526h14c3975_1    bioconda
perl-try-tiny             0.30                    pl526_1    bioconda
perl-xml-namespacesupport 1.12                    pl526_0    bioconda
perl-xml-parser           2.44            pl526h3a4f0e9_6    bioconda
perl-xml-sax              1.00                    pl526_0    bioconda
perl-xml-sax-base         1.09                    pl526_0    bioconda
perl-xml-sax-expat        0.51                    pl526_3    bioconda
perl-xml-simple           2.25                    pl526_1    bioconda
perl-xsloader             0.24                    pl526_0    bioconda
perl-yaml                 1.27                    pl526_0    bioconda
pigz                      2.3.4                         0    conda-forge
pillow                    6.0.0            py27he7afcd5_0    conda-forge
pip                       19.0.3                   py27_0    conda-forge
pixman                    0.34.0            h14c3975_1003    conda-forge
pplacer                   1.1.alpha19                   1    bioconda
prodigal                  2.6.3                         1    bioconda
prokka                    1.13                          0    bioconda
pthread-stubs             0.4               h14c3975_1001    conda-forge
pyparsing                 2.4.0                      py_0    conda-forge
pyqt                      5.9.2            py27h05f1152_2
pysam                     0.15.2           py27h4b7d16d_3    bioconda
python                    2.7.15            h721da81_1008    conda-forge
python-dateutil           2.8.0                      py_0    conda-forge
pytz                      2019.1                     py_0    conda-forge
qt                        5.9.7                h52cfd70_1    conda-forge
quast                     5.0.2           py27pl526ha92aebf_0    bioconda
r-assertthat              0.2.1            r351h6115d3f_0    conda-forge
r-base                    3.5.1             h271c98b_1006    conda-forge
r-bitops                  1.0_6           r351h96ca727_1002    conda-forge
r-boot                    1.3_20                r351_1000    conda-forge
r-catools                 1.17.1.2         r351h29659fb_0    conda-forge
r-class                   7.3_15          r351h96ca727_1000    conda-forge
r-cli                     1.1.0            r351h6115d3f_0    conda-forge
r-cluster                 2.0.8            r351h9bbef5b_0    conda-forge
r-codetools               0.2_16          r351h6115d3f_1000    conda-forge
r-colorspace              1.4_1            r351hcdcec82_0    conda-forge
r-crayon                  1.3.4           r351h6115d3f_1001    conda-forge
r-digest                  0.6.18          r351h96ca727_1000    conda-forge
r-fansi                   0.4.0           r351h96ca727_1000    conda-forge
r-foreign                 0.8_71          r351h96ca727_1002    conda-forge
r-gdata                   2.18.0          r351h6115d3f_1001    conda-forge
r-ggplot2                 3.1.0           r351h6115d3f_1000    conda-forge
r-glue                    1.3.1            r351hcdcec82_0    conda-forge
r-gplots                  3.0.1.1          r351h6115d3f_0    conda-forge
r-gtable                  0.3.0            r351h6115d3f_0    conda-forge
r-gtools                  3.8.1           r351h96ca727_1002    conda-forge
r-kernsmooth              2.23_15         r351ha65eedd_1002    conda-forge
r-labeling                0.3             r351h6115d3f_1001    conda-forge
r-lattice                 0.20_38         r351h96ca727_1000    conda-forge
r-lazyeval                0.2.2            r351hcdcec82_0    conda-forge
r-magrittr                1.5             r351h6115d3f_1001    conda-forge
r-mass                    7.3_51.3         r351hcdcec82_0    conda-forge
r-matrix                  1.2_17           r351hcdcec82_0    conda-forge
r-mgcv                    1.8_28           r351hcdcec82_0    conda-forge
r-munsell                 0.5.0           r351h6115d3f_1001    conda-forge
r-nlme                    3.1_139          r351h9bbef5b_0    conda-forge
r-nnet                    7.3_12          r351h96ca727_1002    conda-forge
r-pillar                  1.3.1           r351h6115d3f_1000    conda-forge
r-pkgconfig               2.0.2           r351h6115d3f_1001    conda-forge
r-plyr                    1.8.4           r351h29659fb_1002    conda-forge
r-r6                      2.4.0            r351h6115d3f_0    conda-forge
r-rcolorbrewer            1.1_2           r351h6115d3f_1001    conda-forge
r-rcpp                    1.0.1            r351h0357c0b_0    conda-forge
r-recommended             3.5.1                 r351_1001    conda-forge
r-reshape2                1.4.3           r351h29659fb_1003    conda-forge
r-rlang                   0.3.4            r351hcdcec82_0    conda-forge
r-rpart                   4.1_15           r351hcdcec82_0    conda-forge
r-scales                  1.0.0           r351h29659fb_1001    conda-forge
r-spatial                 7.3_11          r351h96ca727_1002    conda-forge
r-stringi                 1.4.3            r351h0357c0b_0    conda-forge
r-stringr                 1.4.0            r351h6115d3f_0    conda-forge
r-survival                2.44_1.1         r351hcdcec82_0    conda-forge
r-tibble                  2.1.1            r351hcdcec82_0    conda-forge
r-utf8                    1.1.4           r351h96ca727_1000    conda-forge
r-viridislite             0.3.0           r351h6115d3f_1001    conda-forge
r-withr                   2.1.2           r351h6115d3f_1000    conda-forge
readline                  7.0               hf8c457e_1001    conda-forge
reportlab                 3.5.19           py27h7d98c4e_0    conda-forge
salmon                    0.13.1               h86b0361_0    bioconda
samtools                  1.9                 h8571acd_11    bioconda
scikit-learn              0.20.3           py27ha8026db_1    conda-forge
scipy                     1.2.1            py27h09a28d5_1    conda-forge
seaborn                   0.9.0                      py_0    conda-forge
setuptools                41.0.1                   py27_0    conda-forge
simplejson                3.8.1                    py27_0    bioconda
singledispatch            3.4.0.3               py27_1000    conda-forge
sip                       4.19.8          py27hf484d3e_1000    conda-forge
six                       1.12.0                py27_1000    conda-forge
spades                    3.13.0                        0    bioconda
sqlite                    3.26.0            h67949de_1001    conda-forge
srprism                   2.4.24               h96824bc_3    bioconda
statsmodels               0.9.0           py27h3010b51_1000    conda-forge
subprocess32              3.5.3            py27h14c3975_0    conda-forge
tbb                       2019.5               hc9558a2_0    conda-forge
tbl2asn                   25.6                          3    bioconda
tk                        8.6.9             h84994c4_1001    conda-forge
tktable                   2.10                 h14c3975_0
tornado                   5.1.1           py27h14c3975_1000    conda-forge
trim-galore               0.5.0                         0    bioconda
wheel                     0.33.1                   py27_0    conda-forge
xopen                     0.5.0                      py_0    bioconda
xorg-kbproto              1.0.7             h14c3975_1002    conda-forge
xorg-libice               1.0.9             h516909a_1004    conda-forge
xorg-libsm                1.2.3             h84519dc_1000    conda-forge
xorg-libx11               1.6.7             h14c3975_1000    conda-forge
xorg-libxau               1.0.9                h14c3975_0    conda-forge
xorg-libxdmcp             1.1.3                h516909a_0    conda-forge
xorg-libxext              1.3.4                h516909a_0    conda-forge
xorg-libxrender           0.9.10            h516909a_1002    conda-forge
xorg-renderproto          0.11.1            h14c3975_1002    conda-forge
xorg-xextproto            7.3.0             h14c3975_1002    conda-forge
xorg-xproto               7.0.31            h14c3975_1007    conda-forge
xz                        5.2.4             h14c3975_1001    conda-forge
zlib                      1.2.11            h14c3975_1004    conda-forge
ursky commented 5 years ago

Something very strange is going on with CONCOCT. Previously, I was able to get it to work in a stand-alone conda environement, but now I get the Openblas issue every time no matter what I try. See environment below. This is after clearing the downloaded packages in miniconda2/pkgs, so this is truly a "fresh" install - nothing to do with metawrap.

I am beginning suspect this might have something to do with a specific build version of some package (rather than then software version), but I can no longer produce a functional concoct environment to compare. If someone can produce a working stand-alone concoct conda environment, can you post the results of conda list here? Maybe we can stop some differences in the build versions.

# packages in environment at /home/guritsk1/miniconda2/envs/concoct-env-re:
#
# Name                    Version                   Build  Channel
biopython                 1.68                     py27_0    bioconda
bzip2                     1.0.6             h14c3975_1002    conda-forge
ca-certificates           2019.3.9             hecc5488_0    conda-forge
certifi                   2019.3.9                 py27_0    conda-forge
concoct                   1.0.0            py27h7724fef_4    bioconda
curl                      7.64.1               hf8cf82a_0    conda-forge
cython                    0.29.7           py27he1b5a44_0    conda-forge
freetype                  2.10.0               he983fc9_0    conda-forge
gsl                       2.4               h294904e_1006    conda-forge
jpeg                      9c                h14c3975_1001    conda-forge
krb5                      1.16.3            h05b26f9_1001    conda-forge
libblas                   3.8.0                7_openblas    conda-forge
libcblas                  3.8.0                7_openblas    conda-forge
libcurl                   7.64.1               hda55be3_0    conda-forge
libdeflate                1.0                  h14c3975_1    bioconda
libedit                   3.1.20170329      hf8c457e_1001    conda-forge
libffi                    3.2.1             he1b5a44_1006    conda-forge
libgcc-ng                 8.2.0                hdf63c60_1
libgfortran-ng            7.3.0                hdf63c60_0
liblapack                 3.8.0                7_openblas    conda-forge
libpng                    1.6.37               hed695b0_0    conda-forge
libssh2                   1.8.2                h22169c7_2    conda-forge
libstdcxx-ng              8.2.0                hdf63c60_1
libtiff                   4.0.10            h648cc4a_1001    conda-forge
llvm-openmp               8.0.0                hc9558a2_0    conda-forge
mmtf-python               1.0.2                    py27_0    bioconda
msgpack-python            0.6.1            py27h6bb024c_0    conda-forge
ncurses                   6.1               hf484d3e_1002    conda-forge
nose                      1.3.7                 py27_1002    conda-forge
numpy                     1.16.3           py27he5ce36f_0    conda-forge
olefile                   0.46                       py_0    conda-forge
openblas                  0.3.5             h9ac9557_1001    conda-forge
openmp                    8.0.0                         0    conda-forge
openssl                   1.1.1b               h14c3975_1    conda-forge
pandas                    0.24.2           py27hf484d3e_0    conda-forge
pillow                    6.0.0            py27he7afcd5_0    conda-forge
pip                       19.1                     py27_0    conda-forge
python                    2.7.15            h721da81_1008    conda-forge
python-dateutil           2.8.0                      py_0    conda-forge
pytz                      2019.1                     py_0    conda-forge
readline                  7.0               hf8c457e_1001    conda-forge
reportlab                 3.5.21           py27h7d98c4e_0    conda-forge
samtools                  1.9                 h8571acd_11    bioconda
scikit-learn              0.20.3           py27ha8026db_1    conda-forge
scipy                     1.2.1            py27h09a28d5_1    conda-forge
setuptools                41.0.1                   py27_0    conda-forge
six                       1.12.0                py27_1000    conda-forge
sqlite                    3.26.0            h67949de_1001    conda-forge
tk                        8.6.9             h84994c4_1001    conda-forge
wheel                     0.33.1                   py27_0    conda-forge
xz                        5.2.4             h14c3975_1001    conda-forge
zlib                      1.2.11            h14c3975_1004    conda-forge
kodragonP commented 5 years ago

Hi, My concoct is still work now. dependencies:

slambrechts commented 5 years ago

Not sure if this information is useful to you, but for me the problem still exists when installing with conda, but on the HPC of our university (where they install from source using the EasyBuild installation tool) the endless OpenBLAS warnings do not occur. The MetaWRAP support they are adding to EasyBuild is here: https://github.com/easybuilders/easybuild-easyconfigs/pull/7896 . The HPC administrator also told me they build OpenBLAS with OpenMP support to prevent these kind of issues (again not sure if this is relevant here).

ursky commented 5 years ago

Thanks, @kodragonP, but this doesn't appear to be a fresh install, and concoct is not even installed through conda (not in the list). If you don't mind, could you try to create a fresh environment (conda create -y -n test-env python=2.7), activate it (source activate test-env), install conda there (conda install concoct=1.0), and see if that works? Then post the environment where the only thing you installed was concoct itself and its prerequisites.

Ash1One commented 5 years ago

Hello, @ursky . Here is a fresh concoct install by conda install -c local concoct-1.0.0-py27h7724fef_4.tar.bz2 and conda update concoct. I hope it helps you.

# Name                    Version                   Build  Channel
biopython                 1.73             py27h7b6447c_0    defaults
blas                      1.0                         mkl    defaults
bzip2                     1.0.6                h14c3975_5    defaults
ca-certificates           2019.1.23                     0    defaults
certifi                   2019.3.9                 py27_0    defaults
concoct                   1.0.0            py27h7724fef_4    <unknown>
curl                      7.64.1               hbc83047_0    defaults
cython                    0.29.7           py27he6710b0_0    defaults
gsl                       2.4                  h14c3975_4    defaults
intel-openmp              2019.3                      199    defaults
krb5                      1.16.1               h173b8e3_7    defaults
libcurl                   7.64.1               h20c2e04_0    defaults
libdeflate                1.0                  h14c3975_1    bioconda
libedit                   3.1.20181209         hc058e9b_0    defaults
libffi                    3.2.1                hd88cf55_4    defaults
libgcc-ng                 8.2.0                hdf63c60_1    defaults
libgfortran-ng            7.3.0                hdf63c60_0    defaults
libssh2                   1.8.2                h1ba5d50_0    defaults
libstdcxx-ng              8.2.0                hdf63c60_1    defaults
llvm-openmp               8.0.0                hc9558a2_0    conda-forge
mkl                       2019.3                      199    defaults
mkl_fft                   1.0.12           py27ha843d7b_0    defaults
mkl_random                1.0.2            py27hd81dba3_0    defaults
ncurses                   6.1                  he6710b0_1    defaults
nose                      1.3.7                    py27_2    defaults
numpy                     1.16.3           py27h7e9f1db_0    defaults
numpy-base                1.16.3           py27hde5b4d6_0    defaults
openblas                  0.3.5             h9ac9557_1001    conda-forge
openmp                    8.0.0                         0    conda-forge
openssl                   1.1.1b               h7b6447c_1    defaults
pandas                    0.24.2           py27he6710b0_0    defaults
pip                       19.1                     py27_0    defaults
python                    2.7.16               h9bab390_0    defaults
python-dateutil           2.8.0                    py27_0    defaults
pytz                      2019.1                     py_0    defaults
readline                  7.0                  h7b6447c_5    defaults
samtools                  1.9                 h8571acd_11    bioconda
scikit-learn              0.20.3           py27hd81dba3_0    defaults
scipy                     1.2.1            py27h7c811a0_0    defaults
setuptools                41.0.1                   py27_0    defaults
six                       1.12.0                   py27_0    defaults
sqlite                    3.28.0               h7b6447c_0    defaults
tk                        8.6.8                hbc83047_0    defaults
wheel                     0.33.1                   py27_0    defaults
xz                        5.2.4                h14c3975_4    defaults
zlib                      1.2.11               h7b6447c_3    defaults
ursky commented 5 years ago

Thanks @Ash1One! With this info I FINALLY got to the bottom of this. I noticed that you have the defaults conda channel in priority over conda-forge (which is what I usually use, and recommend for metawrap in the install instructions). After changing the priority with defaults ahead of conda-forge, everything started working again. Most likely one of the programs builds int he other channel was different. Unfortunately it was neither openblas or openmp - that would be too easy (they always come from conda-forge)... I know that setting defaults in high priority often breaks some other packages (fancier configurations of samtools and some python packager start conflicting, iirc), so I started installing programs one at a time to find the exact program that caused this.

THE PROBLEM:

The issue comes from the blas package build version. To run concoct, you need the mkl version of this package, which also installs the appropriate mkl-supported versions of the blas libraries. Interestingly the mkl versions of blas are still available from conda-forge, so there is no need to mess with channel ordering. I think the reason that @Ash1One (and more non-metawrap users) got it right the first time is because the default channel ONLY has the mkl version.

THE FIX:

You need to install a mkl supported build of blas, which can be any version (I believe). In the latest version of metawrap, this fix works without messing with other programs. To do this, run:

conda install blas=2.5=mkl

Let me know if this fix works for people. I'll point out that in some fancier custom environment, this breaks some other libraries if they were installed int he wrong order, but at least the OPEN blas warnings are fixed.

alneberg commented 5 years ago

Wow, great work @ursky! Let me know what should also go into the requirements.txt or the conda installation recipe. Thanks!

ursky commented 5 years ago

Thanks @Ash1One! This information finally helped me identify the issue. The problem does not come from any software version, but rather its build. In particular, the blas software NEEDS to have the mkl build version, (which also updates the appropriate mkl-supported libraries).

To fix this, just install a mkl build of blas. As far as I can tell, any version of blas is supported. In metawrap=1.2 environment, this fix works without interfering with other packages. To fix the OPEN blas warning messages, run:

conda install blas=2.5=mkl
ursky commented 5 years ago

As for fixing the CONCOCT and metaWRAP conda recipes, the only change that needs to happen is to enforce the blas mkl build. For metawral, I will be enforcing - blas 2.5 mkl, but I am not sure how to handle the CONCOCT meta.yaml recipe, since the blas package can be of any version, and every version has a mkl and non-mkl build. I don't know how to do "OR" statements in yaml. As I mentioned, setting defaults conda channel higher than the conda-forge channel also seems to fix the issue, so you could recommend that (although this isn't a true fix).

Ash1One commented 5 years ago

@ursky ，😄. It has been a pleasure to make a little contribution for fixing the problem.

qiyunzhu commented 5 years ago

@ursky The fix didn't work for me. I followed the "better" installation instruction. It said:

(metawrap) -bash-4.1$ conda install blas=2.5=mkl
Fetching package metadata .................
Solving package specifications: 

PackageNotFoundError: Packages missing in current channels:

  - blas 2.5 mkl -> libblas 3.8.0 5_mkl -> mkl >=2019.0,<2020.0a0
  - blas 2.5 mkl -> libcblas 3.8.0 5_mkl
  - blas 2.5 mkl -> libgcc-ng >=7.3.0
  - blas 2.5 mkl -> libgfortran-ng >=7,<8.0a0
  - blas 2.5 mkl -> liblapack 3.8.0 5_mkl
  - blas 2.5 mkl -> liblapacke 3.8.0 5_mkl

We have searched for the packages in the following channels:

  - https://conda.anaconda.org/ursky/linux-64
  - https://conda.anaconda.org/ursky/noarch
  - https://conda.anaconda.org/bioconda/linux-64
  - https://conda.anaconda.org/bioconda/noarch
  - https://conda.anaconda.org/conda-forge/linux-64
  - https://conda.anaconda.org/conda-forge/noarch
  - https://repo.continuum.io/pkgs/free/linux-64
  - https://repo.continuum.io/pkgs/free/noarch
  - https://repo.continuum.io/pkgs/r/linux-64
  - https://repo.continuum.io/pkgs/r/noarch
  - https://repo.continuum.io/pkgs/pro/linux-64
  - https://repo.continuum.io/pkgs/pro/noarch
  - https://conda.anaconda.org/biocore/linux-64
  - https://conda.anaconda.org/biocore/noarch
ucassee commented 5 years ago

Hi @ursky and @alneberg , I use Best (manual) installation to install metawrap(v=1.2.2). But OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option. this error still occur when I run concoct with metawrap in a cluster.

ursky commented 5 years ago

Did you try the fix?

ursky commented 5 years ago

And if the package is not found try searching for any mkl package. See all of the available ones with conda search blas, and it should show you which one you have installed and which are available.

ucassee commented 5 years ago

@ursky Thanks for your reply.

(base) [zhouyingli@ln01 ~]$ conda search blas Loading channels: done

Name Version Build Channel

blas 1.0 mkl anaconda/cloud/conda-forge blas 1.0 mkl pkgs/free
blas 1.0 mkl pkgs/main
blas 1.0 noblas anaconda/cloud/conda-forge blas 1.0 openblas anaconda/cloud/conda-forge blas 1.0 openblas pkgs/free
blas 1.0 openblas pkgs/main
blas 1.1 openblas anaconda/cloud/conda-forge blas 2.4 blis anaconda/cloud/conda-forge blas 2.4 mkl anaconda/cloud/conda-forge blas 2.4 netlib anaconda/cloud/conda-forge blas 2.4 openblas anaconda/cloud/conda-forge blas 2.5 blis anaconda/cloud/conda-forge blas 2.5 mkl anaconda/cloud/conda-forge blas 2.5 netlib anaconda/cloud/conda-forge blas 2.5 openblas anaconda/cloud/conda-forge blas 2.6 blis anaconda/cloud/conda-forge blas 2.6 mkl anaconda/cloud/conda-forge blas 2.6 openblas anaconda/cloud/conda-forge blas 2.7 blis anaconda/cloud/conda-forge blas 2.7 mkl anaconda/cloud/conda-forge blas 2.7 openblas anaconda/cloud/conda-forge blas 2.8 blis anaconda/cloud/conda-forge blas 2.8 mkl anaconda/cloud/conda-forge blas 2.8 openblas anaconda/cloud/conda-forge blas 2.9 blis anaconda/cloud/conda-forge blas 2.9 mkl anaconda/cloud/conda-forge blas 2.9 openblas anaconda/cloud/conda-forge blas 2.10 blis anaconda/cloud/conda-forge blas 2.10 mkl anaconda/cloud/conda-forge blas 2.10 openblas anaconda/cloud/conda-forge

What should I do next？ Concoct works normal on the server in my lab. But it went error on the cluster in my institute.

ursky commented 5 years ago

And if you run conda list | grep blas? And conda install blas=2.5=mkl? Did you add the recommended channels to your env?

ucassee commented 5 years ago

Hi @ursky

(base) [zhouyingli@ln01 ~]$ conda list | grep blas blas 2.7 openblas https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge blast 2.6.0 boost1.64_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/bioconda libblas 3.8.0 7_openblas https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libcblas 3.8.0 7_openblas https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge liblapack 3.8.0 7_openblas https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge liblapacke 3.8.0 7_openblas https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge openblas 0.3.5 h9ac9557_1001 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge

Can I run conda install blas=2.5=mkl directly to fix the bug?

ursky commented 5 years ago

Looks like you have blas=2.7. So running conda install blas=2.7=mkl should fix it.

ucassee commented 5 years ago

@ursky Thank you~ it works

VGalata commented 5 years ago

I have the same problem: I use snakemake and it installs CONCOCT into a new fresh conda environment. I also specified an mkl-version of blas. CONCOCT does start multiple threads (though more than specified), but I also get a lot of error messages: OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option. :(

The used YAML file:

channels:
    - conda-forge
    - defaults
    - bioconda
dependencies:
    - blas==2.10=mkl
    - concoct==1.0.0
    - maxbin2==2.2.6
    - metabat2==2.13
    - das_tool==1.1.1
ucassee commented 5 years ago

Hi @ursky @alneberg . I reinstall the metawrap in my server. But the endless openblas warning occured again when run concoct.

The following is the blas version. What should I do to fix it?

(metawrap-env) [liwl@ln01 ~]$ conda list | grep blas blas 2.5 mkl conda-forge blast 2.6.0 boost1.64_2 bioconda libblas 3.8.0 5_mkl conda-forge libcblas 3.8.0 5_mkl conda-forge openblas 0.3.3 ha44fe06_1 conda-forge

jolespin commented 5 years ago

I'm no expert on this but I THINK that I got mine to work. After reading through @ursky 's detective work, I did the following. It's running right now, I'll let you know if I get the error:

# Move defaults to the top 
conda config --add channels defaults

# Force the reinstall
conda install blas=2.5=mkl --force-reinstall
jolespin commented 5 years ago

Update, the above did'nt work. I got it to to run until completion on a small dataset but not for a larger. The following error message repeats:

OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option.

Do we want openBLAS and BLAS installed?