search

phac-nml / mob-suite

MOB-suite: Software tools for clustering, reconstruction and typing of plasmids from draft assemblies
Apache License 2.0
118 stars 31 forks source link

No such file or directory: mge.blast.results.txt #115

Closed nick-youngblut closed 1 year ago

nick-youngblut commented 2 years ago

It appears that mob_recon expects blast results for the mge report generation, but such blast results are not necessarily generated: 

Traceback (most recent call last):
  File "/tmp/global2/nyoungblut/code/dev/ll_pipelines/llg/.snakemake/conda/db9f22b4bb1b99b6c9c05ebc0a31965c/bin/mob_recon", line 10, in <module>
    sys.exit(main())
  File "/tmp/global2/nyoungblut/code/dev/ll_pipelines/llg/.snakemake/conda/db9f22b4bb1b99b6c9c05ebc0a31965c/lib/python3.8/site-packages/mob_suite/mob_recon.py", line 1433, in main
    mge_results = blast_mge(fixed_fasta, repetitive_mask_file, tmp_dir, min_length,
  File "/tmp/global2/nyoungblut/code/dev/ll_pipelines/llg/.snakemake/conda/db9f22b4bb1b99b6c9c05ebc0a31965c/lib/python3.8/site-packages/mob_suite/utils.py", line 1308, in blast_mge
    if os.path.getsize(blast_results) == 0:
  File "/tmp/global2/nyoungblut/code/dev/ll_pipelines/llg/.snakemake/conda/db9f22b4bb1b99b6c9c05ebc0a31965c/lib/python3.8/genericpath.py", line 50, in getsize
    return os.stat(filename).st_size
FileNotFoundError: [Errno 2] No such file or directory: '/ebio/abt3_scratch/nyoungblut/LLG_152808410656/mob_suite/recon/Bacteroides_pectinophilus/__tmp/mge.blast.results.txt'

My conda env:

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
alsa-lib                  1.2.6.1              h7f98852_0    conda-forge
attr                      2.5.1                h166bdaf_1    conda-forge
biopython                 1.79             py38h0a891b7_2    conda-forge
blast                     2.13.0               hf3cf87c_0    bioconda
blosc                     1.21.1               h83bc5f7_3    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.18.1               h7f98852_0    conda-forge
ca-certificates           2022.6.15            ha878542_0    conda-forge
capnproto                 0.9.1                h780b84a_5    conda-forge
curl                      7.83.1               h7bff187_0    conda-forge
dbus                      1.13.6               h5008d03_3    conda-forge
entrez-direct             16.2                 he881be0_1    bioconda
ete3                      3.1.2              pyh9f0ad1d_0    conda-forge
expat                     2.4.8                h27087fc_0    conda-forge
fftw                      3.3.10          nompi_hf0379b8_105    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.0               h8e229c2_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
freetype                  2.12.1               hca18f0e_0    conda-forge
gettext                   0.19.8.1          h73d1719_1008    conda-forge
glib                      2.72.1               h6239696_0    conda-forge
glib-tools                2.72.1               h6239696_0    conda-forge
gsl                       2.7                  he838d99_0    conda-forge
gst-plugins-base          1.20.3               hf6a322e_0    conda-forge
gstreamer                 1.20.3               hd4edc92_1    conda-forge
hdf5                      1.12.2          nompi_h2386368_100    conda-forge
icu                       70.1                 h27087fc_0    conda-forge
jack                      1.9.18            h8c3723f_1002    conda-forge
jpeg                      9e                   h166bdaf_2    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
krb5                      1.19.3               h3790be6_0    conda-forge
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
libblas                   3.9.0           16_linux64_openblas    conda-forge
libcap                    2.64                 ha37c62d_0    conda-forge
libcblas                  3.9.0           16_linux64_openblas    conda-forge
libclang                  14.0.6          default_h2e3cab8_0    conda-forge
libclang13                14.0.6          default_h3a83d3e_0    conda-forge
libcups                   2.3.3                h3e49a29_2    conda-forge
libcurl                   7.83.1               h7bff187_0    conda-forge
libdb                     6.2.32               h9c3ff4c_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libevent                  2.1.10               h9b69904_4    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libflac                   1.3.4                h27087fc_0    conda-forge
libgcc-ng                 12.1.0              h8d9b700_16    conda-forge
libgfortran-ng            12.1.0              h69a702a_16    conda-forge
libgfortran5              12.1.0              hdcd56e2_16    conda-forge
libglib                   2.72.1               h2d90d5f_0    conda-forge
libgomp                   12.1.0              h8d9b700_16    conda-forge
libiconv                  1.16                 h516909a_0    conda-forge
libidn2                   2.3.3                h166bdaf_0    conda-forge
liblapack                 3.9.0           16_linux64_openblas    conda-forge
libllvm14                 14.0.6               he0ac6c6_0    conda-forge
libnghttp2                1.47.0               hdcd2b5c_1    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libogg                    1.3.4                h7f98852_1    conda-forge
libopenblas               0.3.21          pthreads_h78a6416_3    conda-forge
libopus                   1.3.1                h7f98852_1    conda-forge
libpng                    1.6.37               h753d276_4    conda-forge
libpq                     14.5                 hd77ab85_0    conda-forge
libsndfile                1.0.31               h9c3ff4c_1    conda-forge
libsqlite                 3.39.2               h753d276_1    conda-forge
libssh2                   1.10.0               haa6b8db_3    conda-forge
libstdcxx-ng              12.1.0              ha89aaad_16    conda-forge
libtool                   2.4.6             h9c3ff4c_1008    conda-forge
libudev1                  249                  h166bdaf_4    conda-forge
libunistring              0.9.10               h7f98852_0    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libvorbis                 1.3.7                h9c3ff4c_0    conda-forge
libxcb                    1.13              h7f98852_1004    conda-forge
libxkbcommon              1.0.3                he3ba5ed_0    conda-forge
libxml2                   2.9.14               h22db469_4    conda-forge
libxslt                   1.1.35               h8affb1d_0    conda-forge
libzlib                   1.2.12               h166bdaf_2    conda-forge
lxml                      4.9.1            py38h0a891b7_0    conda-forge
lz4-c                     1.9.3                h9c3ff4c_1    conda-forge
lzo                       2.10              h516909a_1000    conda-forge
mash                      2.3                  ha9a2dd8_3    bioconda
mob_suite                 3.1.0              pyhdfd78af_0    bioconda
mysql-common              8.0.30               haf5c9bc_1    conda-forge
mysql-libs                8.0.30               h28c427c_1    conda-forge
ncurses                   6.3                  h27087fc_1    conda-forge
nomkl                     1.0                  h5ca1d4c_0    conda-forge
nspr                      4.32                 h9c3ff4c_1    conda-forge
nss                       3.78                 h2350873_0    conda-forge
numexpr                   2.8.3           py38h97b1c41_100    conda-forge
numpy                     1.23.2           py38h3a7f9d9_0    conda-forge
openssl                   1.1.1q               h166bdaf_0    conda-forge
packaging                 21.3               pyhd8ed1ab_0    conda-forge
pandas                    1.0.5            py38hcb8c335_0    conda-forge
pcre                      8.45                 h9c3ff4c_0    conda-forge
perl                      5.32.1          2_h7f98852_perl5    conda-forge
perl-archive-tar          2.40            pl5321hdfd78af_0    bioconda
perl-carp                 1.50            pl5321hd8ed1ab_0    conda-forge
perl-common-sense         3.75            pl5321hd8ed1ab_0    conda-forge
perl-compress-raw-bzip2   2.201           pl5321h166bdaf_0    conda-forge
perl-compress-raw-zlib    2.202           pl5321h166bdaf_0    conda-forge
perl-encode               3.19            pl5321hec16e2b_1    bioconda
perl-exporter             5.74            pl5321hd8ed1ab_0    conda-forge
perl-exporter-tiny        1.002002        pl5321hd8ed1ab_0    conda-forge
perl-extutils-makemaker   7.64            pl5321hd8ed1ab_0    conda-forge
perl-io-compress          2.106           pl5321h87f3376_1    bioconda
perl-io-zlib              1.11            pl5321hdfd78af_0    bioconda
perl-json                 4.09            pl5321hdfd78af_0    bioconda
perl-json-xs              2.34            pl5321h9f5acd7_5    bioconda
perl-list-moreutils       0.430           pl5321hdfd78af_0    bioconda
perl-list-moreutils-xs    0.430           pl5321hec16e2b_1    bioconda
perl-parent               0.238           pl5321hd8ed1ab_0    conda-forge
perl-pathtools            3.75            pl5321h166bdaf_0    conda-forge
perl-scalar-list-utils    1.63            pl5321h166bdaf_0    conda-forge
perl-types-serialiser     1.01            pl5321hdfd78af_0    bioconda
pigz                      2.6                  h27826a3_0    conda-forge
pip                       22.2.2             pyhd8ed1ab_0    conda-forge
ply                       3.11                       py_1    conda-forge
portaudio                 19.6.0               h57a0ea0_5    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
pulseaudio                14.0                 h7f54b18_8    conda-forge
pycurl                    7.45.1           py38h61f0cdf_2    conda-forge
pyparsing                 3.0.9              pyhd8ed1ab_0    conda-forge
pyqt                      5.15.7           py38h7492b6b_0    conda-forge
pyqt5-sip                 12.11.0          py38hfa26641_0    conda-forge
pytables                  3.7.0            py38hf632491_2    conda-forge
python                    3.8.13          h582c2e5_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python_abi                3.8                      2_cp38    conda-forge
pytz                      2022.2.1           pyhd8ed1ab_0    conda-forge
qt-main                   5.15.4               ha5833f6_2    conda-forge
readline                  8.1.2                h0f457ee_0    conda-forge
scipy                     1.9.1            py38hea3f02b_0    conda-forge
setuptools                59.8.0           py38h578d9bd_1    conda-forge
sip                       6.6.2            py38hfa26641_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
snappy                    1.1.9                hbd366e4_1    conda-forge
sqlite                    3.39.2               h4ff8645_1    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
wget                      1.20.3               ha56f1ee_1    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
xcb-util                  0.4.0                h516909a_0    conda-forge
xcb-util-image            0.4.0                h166bdaf_0    conda-forge
xcb-util-keysyms          0.4.0                h516909a_0    conda-forge
xcb-util-renderutil       0.3.9                h166bdaf_0    conda-forge
xcb-util-wm               0.4.1                h516909a_0    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.0.10               h7f98852_0    conda-forge
xorg-libsm                1.2.3             hd9c2040_1000    conda-forge
xorg-libx11               1.7.2                h7f98852_0    conda-forge
xorg-libxau               1.0.9                h7f98852_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h7f98852_1    conda-forge
xorg-libxrender           0.9.10            h7f98852_1003    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-xextproto            7.3.0             h7f98852_1002    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
zlib                      1.2.12               h166bdaf_2    conda-forge
zstd                      1.5.2                h6239696_4    conda-forge
nick-youngblut commented 2 years ago

The bug seems to be:

    #return empty dataframe if no blast results generated
    if os.path.getsize(blast_results) == 0:
        return {}

os.path.getsize() assumes that the file exists. Changing the code to the following fixes the issue:

    #return empty dataframe if no blast results generated
    if not os.path.isfile(blast_results):
        open(blast_results, 'w').close()
     if os.path.getsize(blast_results) == 0:
        return {}
kbessonov1984 commented 2 years ago

I agree, this is a bug to correct for all BLAST results queries. Could you provide us with the input file that you are using or accession number of the assembly so we could include it in our unit tests?

nick-youngblut commented 2 years ago

The "problem" genome is GCA_000434155.1 (Bacteroides_pectinophilus)

jrober84 commented 1 year ago

Thank you for pointing this out, I will make an update for this issue