search

cmks / DAS_Tool

DAS Tool
Other
138 stars 17 forks source link

Bad file descriptor @ rb_file_flock #74

Open nick-youngblut opened 2 years ago

nick-youngblut commented 2 years ago

I'm using bioconda::das_tool=1.1.3 with the following command:

DAS_Tool --search_engine diamond             -i $SCAF2BIN             -l $SCAF2BINIDs             -c /ebio/abt3_scratch/nyoungblut/LLMGA_122597181487/all_contigs/ERS235591/contigs_final.fasta             -o /ebio/abt3_projects/databases_no-backup/DeepMAsED/version2/real_data_eval/UHGG/LLMGQC_r100/LLMGA/bin_refine/ERS235591/DAS_Tool/bins             --write_bins 1             --threads 8             --debug

$SCAF2BIN and $SCAF2BINIDs are variables listing the input directories (n=3; comma-delim) and binner IDs, respectively.

The output:

User environment details:
Rscript path:  /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb228
41d8103fa9/bin/Rscript
Rscript version:  R scripting front-end version 4.0.3 (2020-10-10)
pullseq path:  /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb228
41d8103fa9/bin/pullseq
pullseq version:  Version: 1.0.2 Name lookup method: UTHASH
prodigal path:  /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22
841d8103fa9/bin/prodigal
prodigal version:  Prodigal V2.6.3: February, 2016
ruby path:  /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d
8103fa9/bin/ruby
ruby version:  ruby 3.1.0p0 (2021-12-25 revision fb4df44d16) [x86_64-linux]
diamond path:  /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb228
41d8103fa9/bin/diamond
diamond version:  diamond version 0.9.36

Running DAS Tool using 8 threads.
predicting genes using Prodigal V2.6.3: February, 2016
identifying single copy genes using diamond version 2.0.14
/tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems.rb:793:in `flock': Bad file descriptor @ rb_file_flock - /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/gems/3.1.0/specifications/default/abbrev-0.1.0.gemspec (Errno::EBADF)
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems.rb:793:in `block in open_with_flock'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems.rb:790:in `open'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems.rb:790:in `open_with_flock'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems/specification.rb:1119:in `load'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems/specification.rb:800:in `block in each_spec'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems/specification.rb:772:in `block (2 levels) in each_gemspec'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems/specification.rb:771:in `each'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems/specification.rb:771:in `block in each_gemspec'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems/specification.rb:770:in `each'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems/specification.rb:770:in `each_gemspec'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems/specification.rb:799:in `each_spec'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems/specification.rb:870:in `load_defaults'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems.rb:1353:in `<top (required)>'
    from <internal:gem_prelude>:2:in `require'
    from <internal:gem_prelude>:2:in `<internal:gem_prelude>'
mv: cannot stat '/ebio/abt3_projects/databases_no-backup/DeepMAsED/version2/real_data_eval/UHGG/LLMGQC_r100/LLMGA/bin_refine/ERS235591/DAS_Tool/bins_proteins.faa.scg': No such file or directory
/tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems.rb:793:in `flock': Bad file descriptor @ rb_file_flock - /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/gems/3.1.0/specifications/default/abbrev-0.1.0.gemspec (Errno::EBADF)
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems.rb:793:in `block in open_with_flock'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems.rb:790:in `open'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems.rb:790:in `open_with_flock'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems/specification.rb:1119:in `load'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems/specification.rb:800:in `block in each_spec'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems/specification.rb:772:in `block (2 levels) in each_gemspec'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems/specification.rb:771:in `each'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems/specification.rb:771:in `block in each_gemspec'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems/specification.rb:770:in `each'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems/specification.rb:770:in `each_gemspec'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems/specification.rb:799:in `each_spec'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems/specification.rb:870:in `load_defaults'
    from /tmp/global2/nyoungblut/code/dev/ll_pipelines/llmga/.snakemake/conda/1e05d32c9539bd805bb22841d8103fa9/lib/ruby/3.1.0/rubygems.rb:1353:in `<top (required)>'
    from <internal:gem_prelude>:2:in `require'
    from <internal:gem_prelude>:2:in `<internal:gem_prelude>'
mv: cannot stat '/ebio/abt3_projects/databases_no-backup/DeepMAsED/version2/real_data_eval/UHGG/LLMGQC_r100/LLMGA/bin_refine/ERS235591/DAS_Tool/bins_proteins.faa.scg': No such file or directory
rm: cannot remove '/ebio/abt3_projects/databases_no-backup/DeepMAsED/version2/real_data_eval/UHGG/LLMGQC_r100/LLMGA/bin_refine/ERS235591/DAS_Tool/bins_proteins.faa.findSCG.b6': No such file or directory
rm: cannot remove '/ebio/abt3_projects/databases_no-backup/DeepMAsED/version2/real_data_eval/UHGG/LLMGQC_r100/LLMGA/bin_refine/ERS235591/DAS_Tool/bins_proteins.faa.scg.candidates.faa': No such file or directory
rm: cannot remove '/ebio/abt3_projects/databases_no-backup/DeepMAsED/version2/real_data_eval/UHGG/LLMGQC_r100/LLMGA/bin_refine/ERS235591/DAS_Tool/bins_proteins.faa.all.b6': No such file or directory
single copy gene prediction using diamond failed. Aborting

My conda env:

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
_r-mutex                  1.0.1               anacondar_1    conda-forge
binutils_impl_linux-64    2.36.1               h193b22a_2    conda-forge
binutils_linux-64         2.36                 hf3e587d_4    conda-forge
blast                     2.12.0               h3289130_3    bioconda
boost-cpp                 1.74.0               h6cacc03_7    conda-forge
bwidget                   1.9.14               ha770c72_1    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.18.1               h7f98852_0    conda-forge
ca-certificates           2021.10.8            ha878542_0    conda-forge
cairo                     1.16.0            ha00ac49_1009    conda-forge
curl                      7.81.0               h494985f_0    conda-forge
das_tool                  1.1.3             r41hdfd78af_0    bioconda
diamond                   2.0.14               hdcc8f71_0    bioconda
entrez-direct             16.2                 he881be0_0    bioconda
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.13.96              ha180cfb_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
freetype                  2.10.4               h0708190_1    conda-forge
fribidi                   1.0.10               h36c2ea0_0    conda-forge
gawk                      5.1.0                h7f98852_0    conda-forge
gcc_impl_linux-64         9.4.0               h03d3576_12    conda-forge
gcc_linux-64              9.4.0                h391b98a_4    conda-forge
gdbm                      1.18                 h0a1914f_2    conda-forge
gettext                   0.19.8.1          h73d1719_1008    conda-forge
gfortran_impl_linux-64    9.4.0               h0003116_12    conda-forge
gfortran_linux-64         9.4.0                hf0ab688_4    conda-forge
gmp                       6.2.1                h58526e2_0    conda-forge
graphite2                 1.3.13            h58526e2_1001    conda-forge
gsl                       2.7                  he838d99_0    conda-forge
gxx_impl_linux-64         9.4.0               h03d3576_12    conda-forge
gxx_linux-64              9.4.0                h0316aca_4    conda-forge
harfbuzz                  3.3.1                hb4a5f5f_0    conda-forge
icu                       69.1                 h9c3ff4c_0    conda-forge
jbig                      2.1               h7f98852_2003    conda-forge
jpeg                      9e                   h7f98852_0    conda-forge
kernel-headers_linux-64   2.6.32              he073ed8_15    conda-forge
krb5                      1.19.2               h48eae69_3    conda-forge
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
lerc                      3.0                  h9c3ff4c_0    conda-forge
libblas                   3.9.0           13_linux64_openblas    conda-forge
libcblas                  3.9.0           13_linux64_openblas    conda-forge
libcurl                   7.81.0               h494985f_0    conda-forge
libdeflate                1.8                  h7f98852_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-devel_linux-64     9.4.0               hd854feb_12    conda-forge
libgcc-ng                 11.2.0              h1d223b6_12    conda-forge
libgfortran-ng            11.2.0              h69a702a_12    conda-forge
libgfortran5              11.2.0              h5c6108e_12    conda-forge
libglib                   2.70.2               h174f98d_2    conda-forge
libgomp                   11.2.0              h1d223b6_12    conda-forge
libiconv                  1.16                 h516909a_0    conda-forge
liblapack                 3.9.0           13_linux64_openblas    conda-forge
libnghttp2                1.46.0               ha19adfc_0    conda-forge
libopenblas               0.3.18          pthreads_h8fe5266_0    conda-forge
libpng                    1.6.37               h21135ba_2    conda-forge
libsanitizer              9.4.0               h79bfe98_12    conda-forge
libssh2                   1.10.0               ha35d2d1_2    conda-forge
libstdcxx-devel_linux-64  9.4.0               hd854feb_12    conda-forge
libstdcxx-ng              11.2.0              he4da1e4_12    conda-forge
libtiff                   4.3.0                h6f004c6_2    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libwebp-base              1.2.2                h7f98852_1    conda-forge
libxcb                    1.13              h7f98852_1004    conda-forge
libxml2                   2.9.12               h885dcf4_1    conda-forge
libzlib                   1.2.11            h36c2ea0_1013    conda-forge
lz4-c                     1.9.3                h9c3ff4c_1    conda-forge
make                      4.3                  hd18ef5c_1    conda-forge
ncurses                   6.2                  h58526e2_4    conda-forge
openssl                   3.0.0                h7f98852_2    conda-forge
pango                     1.50.3               h9967ed3_0    conda-forge
pcre                      8.45                 h9c3ff4c_0    conda-forge
pcre2                     10.37                h032f7d1_0    conda-forge
perl                      5.32.1          1_h7f98852_perl5    conda-forge
perl-archive-tar          2.40            pl5321hdfd78af_0    bioconda
perl-carp                 1.50            pl5321hd8ed1ab_0    conda-forge
perl-common-sense         3.75            pl5321hdfd78af_0    bioconda
perl-compress-raw-bzip2   2.101           pl5321h1b792b2_0    bioconda
perl-compress-raw-zlib    2.101           pl5321h1b792b2_1    bioconda
perl-exporter             5.74            pl5321hd8ed1ab_0    conda-forge
perl-exporter-tiny        1.002002        pl5321hdfd78af_0    bioconda
perl-extutils-makemaker   7.64            pl5321hd8ed1ab_0    conda-forge
perl-io-compress          2.087           pl5321h1b792b2_1    bioconda
perl-io-zlib              1.10                          1    bioconda
perl-json                 4.05            pl5321hdfd78af_0    bioconda
perl-json-xs              2.34            pl5321h7d875b9_4    bioconda
perl-list-moreutils       0.428           pl5321hdfd78af_2    bioconda
perl-list-moreutils-xs    0.430           pl5321h779adbc_0    bioconda
perl-pathtools            3.75            pl5321h779adbc_2    bioconda
perl-scalar-list-utils    1.60            pl5321h779adbc_0    bioconda
perl-threaded             5.32.1               hdfd78af_1    bioconda
perl-types-serialiser     1.01            pl5321hdfd78af_0    bioconda
pigz                      2.6                  h27826a3_0    conda-forge
pixman                    0.40.0               h36c2ea0_0    conda-forge
prodigal                  2.6.3                h779adbc_3    bioconda
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
pullseq                   1.0.2                h38613fd_6    bioconda
r-assertthat              0.2.1             r41hc72bb7e_2    conda-forge
r-backports               1.4.1             r41hcfec24a_0    conda-forge
r-base                    4.1.2                h2553ce4_1    conda-forge
r-brio                    1.1.3             r41hcfec24a_0    conda-forge
r-callr                   3.7.0             r41hc72bb7e_0    conda-forge
r-cli                     3.1.1             r41h03ef668_0    conda-forge
r-codetools               0.2_18            r41hc72bb7e_0    conda-forge
r-colorspace              2.0_2             r41hcfec24a_0    conda-forge
r-crayon                  1.4.2             r41hc72bb7e_0    conda-forge
r-data.table              1.14.2            r41hcfec24a_0    conda-forge
r-desc                    1.4.0             r41hc72bb7e_0    conda-forge
r-diffobj                 0.3.5             r41hcfec24a_0    conda-forge
r-digest                  0.6.29            r41h03ef668_0    conda-forge
r-domc                    1.3.8             r41ha770c72_0    conda-forge
r-ellipsis                0.3.2             r41hcfec24a_0    conda-forge
r-evaluate                0.14              r41hc72bb7e_2    conda-forge
r-fansi                   1.0.2             r41hcfec24a_0    conda-forge
r-farver                  2.1.0             r41h03ef668_0    conda-forge
r-foreach                 1.5.2             r41hc72bb7e_0    conda-forge
r-ggplot2                 3.3.5             r41hc72bb7e_0    conda-forge
r-glue                    1.6.1             r41hcfec24a_0    conda-forge
r-gtable                  0.3.0             r41hc72bb7e_3    conda-forge
r-isoband                 0.2.5             r41h03ef668_0    conda-forge
r-iterators               1.0.14            r41hc72bb7e_0    conda-forge
r-jsonlite                1.7.3             r41hcfec24a_0    conda-forge
r-labeling                0.4.2             r41hc72bb7e_1    conda-forge
r-lattice                 0.20_45           r41hcfec24a_0    conda-forge
r-lifecycle               1.0.1             r41hc72bb7e_0    conda-forge
r-magrittr                2.0.2             r41hcfec24a_0    conda-forge
r-mass                    7.3_55            r41hcfec24a_0    conda-forge
r-matrix                  1.4_0             r41he454529_0    conda-forge
r-mgcv                    1.8_38            r41he454529_0    conda-forge
r-munsell                 0.5.0           r41hc72bb7e_1004    conda-forge
r-nlme                    3.1_155           r41h859d828_0    conda-forge
r-pillar                  1.7.0             r41hc72bb7e_0    conda-forge
r-pkgconfig               2.0.3             r41hc72bb7e_1    conda-forge
r-pkgload                 1.2.4             r41h03ef668_0    conda-forge
r-praise                  1.0.0           r41hc72bb7e_1005    conda-forge
r-processx                3.5.2             r41hcfec24a_0    conda-forge
r-ps                      1.6.0             r41hcfec24a_0    conda-forge
r-r6                      2.5.1             r41hc72bb7e_0    conda-forge
r-rcolorbrewer            1.1_2           r41h785f33e_1003    conda-forge
r-rcpp                    1.0.8             r41h03ef668_0    conda-forge
r-rematch2                2.1.2             r41hc72bb7e_1    conda-forge
r-rlang                   0.4.12            r41hcfec24a_0    conda-forge
r-rprojroot               2.0.2             r41hc72bb7e_0    conda-forge
r-rstudioapi              0.13              r41hc72bb7e_0    conda-forge
r-scales                  1.1.1             r41hc72bb7e_0    conda-forge
r-testthat                3.1.2             r41h03ef668_0    conda-forge
r-tibble                  3.1.6             r41hcfec24a_0    conda-forge
r-utf8                    1.2.2             r41hcfec24a_0    conda-forge
r-vctrs                   0.3.8             r41hcfec24a_1    conda-forge
r-viridislite             0.4.0             r41hc72bb7e_0    conda-forge
r-waldo                   0.3.1             r41hc72bb7e_0    conda-forge
r-withr                   2.4.3             r41hc72bb7e_0    conda-forge
readline                  8.1                  h46c0cb4_0    conda-forge
ruby                      3.1.0                h134ee0a_1    conda-forge
sed                       4.8                  he412f7d_0    conda-forge
sysroot_linux-64          2.12                he073ed8_15    conda-forge
tk                        8.6.11               h27826a3_1    conda-forge
tktable                   2.10                 hb7b940f_3    conda-forge
unzip                     6.0                  h7f98852_3    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.0.10               h7f98852_0    conda-forge
xorg-libsm                1.2.3             hd9c2040_1000    conda-forge
xorg-libx11               1.7.2                h7f98852_0    conda-forge
xorg-libxau               1.0.9                h7f98852_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h7f98852_1    conda-forge
xorg-libxrender           0.9.10            h7f98852_1003    conda-forge
xorg-libxt                1.2.1                h7f98852_2    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-xextproto            7.3.0             h7f98852_1002    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
yaml                      0.2.5                h7f98852_2    conda-forge
zlib                      1.2.11            h36c2ea0_1013    conda-forge
zstd                      1.5.2                ha95c52a_0    conda-forge
nick-youngblut commented 2 years ago

The same error occurs if I downgrade to bioconda::diamond=0.9.36, which downgrades das-tool to 1.1.2

nick-youngblut commented 2 years ago

The same error occurs if using --search_engine blast

nick-youngblut commented 2 years ago

It turns out that the issue is due to the NFS that das-tool was installed on. When I installed das-tool on a different NFS, it ran without errors.

These seems to be a (somewhat) known issue with flock: https://github.com/Linuxbrew/legacy-linuxbrew/issues/1017

cmks commented 2 years ago

That's interesting, thanks sharing the issue and your workaround.