jessieren / DeepVirFinder

Identifying viruses from metagenomic data by deep learning
Other
116 stars 32 forks source link

HPC and DeepVirFinder #9

Open soilmicrobiome opened 4 years ago

soilmicrobiome commented 4 years ago

Hi there,

Is there a way to have DeepVirFinder available through singularity or docker? I am working on an HPC and conda is not allowed as it causes conflicts.

I tried to run get the program working as follows:

$ module load python $USER/deepvirfinder_env $USER/deepvirfinder_env/bin/activate $ pip install numpy theano keras scikit-learn $ pip install biopython

But when i run the program i get the following issues:

Using Theano backend. Traceback (most recent call last): File "/cvmfs/soft.HPC/easybuild/software/2017/Core/python/3.6.3/lib/python3.6/configparser.py", line 1138, in _unify_values sectiondict = self._sections[section] KeyError: 'blas'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/USER/deepvirfinder_env/lib/python3.6/site-packages/theano/configparser.py", line 168, in fetch_val_for_key return theano_cfg.get(section, option) File "/cvmfs/soft.HPC/easybuild/software/2017/Core/python/3.6.3/lib/python3.6/configparser.py", line 781, in get d = self._unify_values(section, vars) File "/cvmfs/soft.HPC/easybuild/software/2017/Core/python/3.6.3/lib/python3.6/configparser.py", line 1141, in _unify_values raise NoSectionError(section) configparser.NoSectionError: No section: 'blas'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/USER/deepvirfinder_env/lib/python3.6/site-packages/theano/configparser.py", line 328, in get delete_key=delete_key) File "/home/USER/deepvirfinder_env/lib/python3.6/site-packages/theano/configparser.py", line 172, in fetch_val_for_key raise KeyError(key) KeyError: 'blas.ldflags'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/USER/deepvirfinder_env/lib/python3.6/site-packages/theano/configdefaults.py", line 1250, in check_mkl_openmp import mkl ModuleNotFoundError: No module named 'mkl'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/USER/deepvirfinder_env/bin/DeepVirFinder/dvf.py", line 53, in import keras File "/home/USER/deepvirfinder_env/lib/python3.6/site-packages/keras/init.py", line 3, in from . import utils File "/home/USER/deepvirfinder_env/lib/python3.6/site-packages/keras/utils/init.py", line 6, in from . import conv_utils File "/home/USER/deepvirfinder_env/lib/python3.6/site-packages/keras/utils/conv_utils.py", line 9, in from .. import backend as K File "/home/USER/deepvirfinder_env/lib/python3.6/site-packages/keras/backend/init.py", line 1, in from .load_backend import epsilon File "/home/USER/deepvirfinder_env/lib/python3.6/site-packages/keras/backend/load_backend.py", line 87, in from .theano_backend import * File "/home/USER/deepvirfinder_env/lib/python3.6/site-packages/keras/backend/theano_backend.py", line 7, in import theano File "/home/USER/deepvirfinder_env/lib/python3.6/site-packages/theano/init.py", line 124, in from theano.scan_module import (scan, map, reduce, foldl, foldr, clone, File "/home/USER/deepvirfinder_env/lib/python3.6/site-packages/theano/scan_module/init.py", line 41, in from theano.scan_module import scan_opt File "/home/USER/deepvirfinder_env/lib/python3.6/site-packages/theano/scan_module/scan_opt.py", line 60, in from theano import tensor, scalar File "/home/USER/deepvirfinder_env/lib/python3.6/site-packages/theano/tensor/init.py", line 17, in from theano.tensor import blas File "/home/USER/deepvirfinder_env/lib/python3.6/site-packages/theano/tensor/blas.py", line 155, in from theano.tensor.blas_headers import blas_header_text File "/home/USER/deepvirfinder_env/lib/python3.6/site-packages/theano/tensor/blas_headers.py", line 987, in if not config.blas.ldflags: File "/home/USER/deepvirfinder_env/lib/python3.6/site-packages/theano/configparser.py", line 332, in get val_str = self.default() File "/home/USER/deepvirfinder_env/lib/python3.6/site-packages/theano/configdefaults.py", line 1430, in default_blas_ldflags check_mkl_openmp() File "/home/USER/deepvirfinder_env/lib/python3.6/site-packages/theano/configdefaults.py", line 1262, in check_mkl_openmp """) RuntimeError: Could not import 'mkl'. Either install mkl-service with conda or set MKL_THREADING_LAYER=GNU in your environment for MKL 2018.

If you have MKL 2017 install and are not in a conda environment you can set the Theano flag blas.check_openmp to False. Be warned that if you set this flag and don't set the appropriate environment or make sure you have the right version you will get wrong results.

Any thoughts?

Thanks