search

cmks / DAS_Tool

DAS Tool
Other
139 stars 17 forks source link

v1.1 error: mv: cannot stat `*.faa.scg': No such file or directory #13

Closed snayfach closed 7 years ago

snayfach commented 7 years ago

I'm running into an error running v1.1 on the example dataset. Here's the error I get with usearch:

./DAS_Tool -i sample_data/sample.human.gut_concoct_scaffolds2bin.tsv,sample_data/sample.human.gut_maxbin2_scaffolds2bin.tsv,sample_data/sample.human.gut_metabat_scaffolds2bin.tsv,sample_data/sample.human.gut_tetraESOM_scaffolds2bin.tsv -l concoct,maxbin,metabat,tetraESOM -c sample_data/sample.human.gut_contigs.fa -o sample_output/DASToolRun -t 30
predicting genes using prodigal
identifying single copy genes using usearch
mv: cannot stat `sample_output/DASToolRun_proteins.faa.scg': No such file or directory
running DAS Tool with 30 threads
Execution halted

And the same error with blast:

./DAS_Tool -i sample_data/sample.human.gut_concoct_scaffolds2bin.tsv,sample_data/sample.human.gut_maxbin2_scaffolds2bin.tsv,sample_data/sample.human.gut_metabat_scaffolds2bin.tsv,sample_data/sample.human.gut_tetraESOM_scaffolds2bin.tsv -l concoct,maxbin,metabat,tetraESOM -c sample_data/sample.human.gut_contigs.fa -o sample_output/DASToolRun -t 30 --search_engine blast
predicting genes using prodigal
identifying single copy genes using blast
mv: cannot stat `sample_output/DASToolRun_proteins.faa.scg': No such file or directory
running DAS Tool with 30 threads
Execution halted

I do not get this error with v1.0. v1.0 completes this step successfully, but runs into a different error at a later step.

snayfach commented 7 years ago

Turns out the issue caused by the wrong Prodigal (2.5 instead of 2.6)

jvollme commented 6 years ago

I have the same problem. Using the correct version of prodigal (v.2.6.3). But the "scg" files should not be created by prodigal anyway. That should be part of the subsequent "ruby" step, right? So How exactly did you solve this?

rotoscan commented 6 years ago

I am also interested and how you did to solve this problem as I am tackling the same issue.

jvollme commented 6 years ago

In my case, the problem was that the shebang of the ruby scripts called by the DAStool pipeline at this point were not suitable for my system/setup. They pointed to the global ruby installation, installed under /usr/bin (which was an old version, and which I had no control over, since I work on a public server and have to install all my personal dependencies under a local conda environment). Changing the shebang to "'#!/usr/bin/env ruby" fixed the problem for me. See also this thread for details: #15

nick-youngblut commented 5 years ago

I'm running das_tool with prodigal 2.6.3, and from looking at the das_tool code, it appears that the shabang lines for the ruby scripts have been changed to #!/usr/bin/env ruby. Still, I'm seemingly getting the error as reported above:

diamond v0.9.25.126 | by Benjamin Buchfink <buchfink@gmail.com>
Licensed under the GNU GPL <https://www.gnu.org/licenses/gpl.txt>
Check http://github.com/bbuchfink/diamond for updates.

/ebio/abt3_projects/bin/llmga/.snakemake/conda/5d764adc/bin/DAS_Tool: line 241: usearch: command not found
Running DAS Tool using 12 threads.
predicting genes using Prodigal V2.6.3: February, 2016
diamond v0.9.25.126 | by Benjamin Buchfink <buchfink@gmail.com>
Licensed under the GNU GPL <https://www.gnu.org/licenses/gpl.txt>
Check http://github.com/bbuchfink/diamond for updates.

identifying single copy genes using diamond version 0.9.25
mv: cannot stat '/ebio/abt3_projects/DAS_Tool/bins_proteins.faa.scg': No such file or directory
single copy gene prediction using diamond failed. Aborting

This only seems to happen with some das_tool jobs (certain datasets) and not others.

My conda env:

_libgcc_mutex             0.1                        main
_r-mutex                  1.0.1               anacondar_1    conda-forge
binutils_impl_linux-64    2.31.1               h6176602_1
binutils_linux-64         2.31.1               h6176602_8
blast                     2.9.0           pl526h979a64d_3    bioconda
boost                     1.70.0           py37h9de70de_1    conda-forge
boost-cpp                 1.70.0               ha2d47e9_1    conda-forge
bwidget                   1.9.11                        0    conda-forge
bzip2                     1.0.8                h516909a_0    conda-forge
ca-certificates           2019.6.16            hecc5488_0    conda-forge
cairo                     1.16.0            h18b612c_1001    conda-forge
certifi                   2019.6.16                py37_1    conda-forge
curl                      7.65.3               hf8cf82a_0    conda-forge
das_tool                  1.1.1                    r351_2    bioconda
diamond                   0.9.25               hfb76ee0_0    bioconda
entrez-direct             11.0                    pl526_0    bioconda
expat                     2.2.5             he1b5a44_1003    conda-forge
fontconfig                2.13.1            he4413a7_1000    conda-forge
freetype                  2.10.0               he983fc9_0    conda-forge
gawk                      5.0.1                h516909a_0    conda-forge
gcc_impl_linux-64         7.3.0                habb00fd_1    conda-forge
gcc_linux-64              7.3.0                h553295d_8    conda-forge
gettext                   0.19.8.1          hc5be6a0_1002    conda-forge
gfortran_impl_linux-64    7.3.0                hdf63c60_1
gfortran_linux-64         7.3.0                h553295d_8
glib                      2.58.3            h6f030ca_1002    conda-forge
gmp                       6.1.2             hf484d3e_1000    conda-forge
graphite2                 1.3.13            hf484d3e_1000    conda-forge
gsl                       2.5                  h294904e_0    conda-forge
gxx_impl_linux-64         7.3.0                hdf63c60_1    conda-forge
gxx_linux-64              7.3.0                h553295d_8    conda-forge
harfbuzz                  2.4.0                h37c48d4_1    conda-forge
icu                       58.2              hf484d3e_1000    conda-forge
jpeg                      9c                h14c3975_1001    conda-forge
krb5                      1.16.3            h05b26f9_1001    conda-forge
libblas                   3.8.0               11_openblas    conda-forge
libcblas                  3.8.0               11_openblas    conda-forge
libcurl                   7.65.3               hda55be3_0    conda-forge
libedit                   3.1.20170329      hf8c457e_1001    conda-forge
libffi                    3.2.1             he1b5a44_1006    conda-forge
libgcc-ng                 9.1.0                hdf63c60_0
libgfortran-ng            7.3.0                hdf63c60_0
libiconv                  1.15              h516909a_1005    conda-forge
liblapack                 3.8.0               11_openblas    conda-forge
libopenblas               0.3.6                h6e990d7_6    conda-forge
libpng                    1.6.37               hed695b0_0    conda-forge
libssh2                   1.8.2                h22169c7_2    conda-forge
libstdcxx-ng              9.1.0                hdf63c60_0
libtiff                   4.0.10            h57b8799_1003    conda-forge
libuuid                   2.32.1            h14c3975_1000    conda-forge
libxcb                    1.13              h14c3975_1002    conda-forge
libxml2                   2.9.9                h13577e0_2    conda-forge
lz4-c                     1.8.3             he1b5a44_1001    conda-forge
make                      4.2.1             h14c3975_2004    conda-forge
ncurses                   6.1               hf484d3e_1002    conda-forge
numpy                     1.17.0           py37h95a1406_0    conda-forge
openssl                   1.1.1c               h516909a_0    conda-forge
pango                     1.40.14           he7ab937_1005    conda-forge
pcre                      8.41              hf484d3e_1003    conda-forge
perl                      5.26.2            h516909a_1006    conda-forge
perl-app-cpanminus        1.7044                  pl526_1    bioconda
perl-archive-tar          2.32                    pl526_0    bioconda
perl-base                 2.23                    pl526_1    bioconda
perl-business-isbn        3.004                   pl526_0    bioconda
perl-business-isbn-data   20140910.003            pl526_0    bioconda
perl-carp                 1.38                    pl526_3    bioconda
perl-common-sense         3.74                    pl526_2    bioconda
perl-compress-raw-bzip2   2.086           pl526hf484d3e_0    bioconda
perl-compress-raw-zlib    2.086           pl526h6bb024c_1    bioconda
perl-constant             1.33                    pl526_1    bioconda
perl-data-dumper          2.173                   pl526_0    bioconda
perl-digest-hmac          1.03                    pl526_3    bioconda
perl-digest-md5           2.55                    pl526_0    bioconda
perl-encode               2.88                    pl526_1    bioconda
perl-encode-locale        1.05                    pl526_6    bioconda
perl-exporter             5.72                    pl526_1    bioconda
perl-exporter-tiny        1.002001                pl526_0    bioconda
perl-extutils-makemaker   7.36                    pl526_1    bioconda
perl-file-listing         6.04                    pl526_1    bioconda
perl-file-path            2.16                    pl526_0    bioconda
perl-file-temp            0.2304                  pl526_2    bioconda
perl-html-parser          3.72            pl526h6bb024c_5    bioconda
perl-html-tagset          3.20                    pl526_3    bioconda
perl-html-tree            5.07                    pl526_1    bioconda
perl-http-cookies         6.04                    pl526_0    bioconda
perl-http-daemon          6.01                    pl526_1    bioconda
perl-http-date            6.02                    pl526_3    bioconda
perl-http-message         6.18                    pl526_0    bioconda
perl-http-negotiate       6.01                    pl526_3    bioconda
perl-io-compress          2.086           pl526hf484d3e_0    bioconda
perl-io-html              1.001                   pl526_2    bioconda
perl-io-socket-ssl        2.066                   pl526_0    bioconda
perl-io-zlib              1.10                    pl526_2    bioconda
perl-json                 4.02                    pl526_0    bioconda
perl-json-xs              2.34            pl526h6bb024c_3    bioconda
perl-libwww-perl          6.39                    pl526_0    bioconda
perl-list-moreutils       0.428                   pl526_1    bioconda
perl-list-moreutils-xs    0.428                   pl526_0    bioconda
perl-lwp-mediatypes       6.04                    pl526_0    bioconda
perl-lwp-protocol-https   6.07                    pl526_4    bioconda
perl-mime-base64          3.15                    pl526_1    bioconda
perl-mozilla-ca           20180117                pl526_1    bioconda
perl-net-http             6.19                    pl526_0    bioconda
perl-net-ssleay           1.88            pl526h90d6eec_0    bioconda
perl-ntlm                 1.09                    pl526_4    bioconda
perl-parent               0.236                   pl526_1    bioconda
perl-pathtools            3.75            pl526h14c3975_1    bioconda
perl-scalar-list-utils    1.50            pl526h14c3975_0    bioconda
perl-socket               2.027                   pl526_1    bioconda
perl-storable             3.15            pl526h14c3975_0    bioconda
perl-test-requiresinternet 0.05                    pl526_0    bioconda
perl-time-local           1.28                    pl526_1    bioconda
perl-try-tiny             0.30                    pl526_1    bioconda
perl-types-serialiser     1.0                     pl526_2    bioconda
perl-uri                  1.76                    pl526_0    bioconda
perl-www-robotrules       6.02                    pl526_3    bioconda
perl-xml-namespacesupport 1.12                    pl526_0    bioconda
perl-xml-parser           2.44_01         pl526ha1d75be_1002    conda-forge
perl-xml-sax              1.02                    pl526_0    bioconda
perl-xml-sax-base         1.09                    pl526_0    bioconda
perl-xml-sax-expat        0.51                    pl526_3    bioconda
perl-xml-simple           2.25                    pl526_1    bioconda
perl-xsloader             0.24                    pl526_0    bioconda
pigz                      2.3.4                         0    conda-forge
pip                       19.2.1                   py37_0    conda-forge
pixman                    0.38.0            h516909a_1003    conda-forge
prodigal                  2.6.3                         1    bioconda
pthread-stubs             0.4               h14c3975_1001    conda-forge
pullseq                   1.0.2                h84994c4_4    bioconda
python                    3.7.3                h5b0a415_0    conda-forge
r-assertthat              0.2.1             r35h6115d3f_1    conda-forge
r-backports               1.1.4             r35hcdcec82_1    conda-forge
r-base                    3.5.1             hea2b9be_1007    conda-forge
r-cli                     1.1.0             r35h6115d3f_1    conda-forge
r-codetools               0.2_16          r35h6115d3f_1001    conda-forge
r-colorspace              1.4_1             r35hcdcec82_1    conda-forge
r-crayon                  1.3.4           r35h6115d3f_1002    conda-forge
r-data.table              1.12.2            r35hcdcec82_1    conda-forge
r-digest                  0.6.20            r35h0357c0b_1    conda-forge
r-domc                    1.3.6                     r35_0    conda-forge
r-ellipsis                0.2.0.1           r35hcdcec82_1    conda-forge
r-fansi                   0.4.0           r35hcdcec82_1001    conda-forge
r-foreach                 1.4.7             r35h6115d3f_0    conda-forge
r-ggplot2                 3.2.0             r35h6115d3f_1    conda-forge
r-glue                    1.3.1             r35hcdcec82_1    conda-forge
r-gtable                  0.3.0             r35h6115d3f_2    conda-forge
r-iterators               1.0.12            r35h6115d3f_0    conda-forge
r-labeling                0.3             r35h6115d3f_1002    conda-forge
r-lattice                 0.20_38         r35hcdcec82_1002    conda-forge
r-lazyeval                0.2.2             r35hcdcec82_1    conda-forge
r-magrittr                1.5             r35h6115d3f_1002    conda-forge
r-mass                    7.3_51.4          r35hcdcec82_1    conda-forge
r-matrix                  1.2_17            r35hcdcec82_1    conda-forge
r-mgcv                    1.8_28            r35hcdcec82_1    conda-forge
r-munsell                 0.5.0           r35h6115d3f_1002    conda-forge
r-nlme                    3.1_140           r35h9bbef5b_1    conda-forge
r-pillar                  1.4.2             r35h6115d3f_2    conda-forge
r-pkgconfig               2.0.2           r35h6115d3f_1002    conda-forge
r-plyr                    1.8.4           r35h0357c0b_1003    conda-forge
r-r6                      2.4.0             r35h6115d3f_2    conda-forge
r-rcolorbrewer            1.1_2           r35h6115d3f_1002    conda-forge
r-rcpp                    1.0.2             r35h0357c0b_0    conda-forge
r-reshape2                1.4.3           r35h0357c0b_1004    conda-forge
r-rlang                   0.4.0             r35hcdcec82_1    conda-forge
r-scales                  1.0.0           r35h0357c0b_1002    conda-forge
r-stringi                 1.4.3             r35h0357c0b_2    conda-forge
r-stringr                 1.4.0             r35h6115d3f_1    conda-forge
r-tibble                  2.1.3             r35hcdcec82_1    conda-forge
r-utf8                    1.1.4           r35hcdcec82_1001    conda-forge
r-vctrs                   0.2.0             r35hcdcec82_1    conda-forge
r-viridislite             0.3.0           r35h6115d3f_1002    conda-forge
r-withr                   2.1.2           r35h6115d3f_1001    conda-forge
r-zeallot                 0.1.0           r35h6115d3f_1001    conda-forge
readline                  7.0               hf8c457e_1001    conda-forge
ruby                      2.4.5             haa072f8_1001    conda-forge
setuptools                41.0.1                   py37_0    conda-forge
sqlite                    3.28.0               h8b20d00_0    conda-forge
tk                        8.6.9             hed695b0_1002    conda-forge
tktable                   2.10                 h555a92e_1    conda-forge
unzip                     6.0                  h516909a_0    conda-forge
wheel                     0.33.4                   py37_0    conda-forge
xorg-kbproto              1.0.7             h14c3975_1002    conda-forge
xorg-libice               1.0.10               h516909a_0    conda-forge
xorg-libsm                1.2.3             h84519dc_1000    conda-forge
xorg-libx11               1.6.8                h516909a_0    conda-forge
xorg-libxau               1.0.9                h14c3975_0    conda-forge
xorg-libxdmcp             1.1.3                h516909a_0    conda-forge
xorg-libxext              1.3.4                h516909a_0    conda-forge
xorg-libxrender           0.9.10            h516909a_1002    conda-forge
xorg-renderproto          0.11.1            h14c3975_1002    conda-forge
xorg-xextproto            7.3.0             h14c3975_1002    conda-forge
xorg-xproto               7.0.31            h14c3975_1007    conda-forge
xz                        5.2.4             h14c3975_1001    conda-forge
yaml                      0.1.7             h14c3975_1001    conda-forge
zlib                      1.2.11            h516909a_1005    conda-forge
zstd                      1.4.0                h3b9ef0a_0    conda-forge
yanhui09 commented 3 years ago

@jvollme mention #https://github.com/cmks/DAS_Tool/issues/13#issuecomment-359875950 is corrct. It's probably due to the "ruby" conflicts between the base and conda environment. I use a public server, which has ruby installed already. Although in a conda environment ruby points to the correct bin path, it collaps when I call it. I know little about ruby, so I can't give more explanation about this.

As I have limited control of the public sever, my solution is to solely remove the ruby in the conda environment, and use the default one with the server. (It may have some issues with nodes without ruby, which is decided by how the cluster is set up) Something like:

conda remove ruby --force