search

phac-nml / mob-suite

MOB-suite: Software tools for clustering, reconstruction and typing of plasmids from draft assemblies
Apache License 2.0
124 stars 33 forks source link

mob_recon's error: "TypeError: to_csv() got an unexpected keyword argument 'line_terminator'" #155

Closed wanyuac closed 11 months ago

wanyuac commented 11 months ago

mob_recon v3.1.8

Command:

mob_init -d /mnt/data/databases/mobsuite/db_20231221
mob_recon --outdir "mob_recon/sample1" --infile sample1.fasta --num_threads 8 --sample_id sample1 --database_directory /mnt/data/databases/mobsuite/db_20231221

Error:

2023-12-21 22:30:13,093 mob_suite.mob_recon INFO: MOB-recon version 3.1.8  [in /usr/local/bin/miniconda/condabin/env/mobsuite3.1.8/lib/python3.9/site-packages/mob_suite/mob_recon.py:981]
2023-12-21 22:30:13,094 mob_suite.mob_recon INFO: SUCCESS: Found program blastn at /usr/local/bin/miniconda/condabin/env/mobsuite3.1.8/bin/blastn [in /usr/local/bin/miniconda/condabin/env/mobsuite3.
1.8/lib/python3.9/site-packages/mob_suite/utils.py:597]
2023-12-21 22:30:13,094 mob_suite.mob_recon INFO: SUCCESS: Found program makeblastdb at /usr/local/bin/miniconda/condabin/env/mobsuite3.1.8/bin/makeblastdb [in /usr/local/bin/miniconda/condabin/env/
mobsuite3.1.8/lib/python3.9/site-packages/mob_suite/utils.py:597]
2023-12-21 22:30:13,094 mob_suite.mob_recon INFO: SUCCESS: Found program tblastn at /usr/local/bin/miniconda/condabin/env/mobsuite3.1.8/bin/tblastn [in /usr/local/bin/miniconda/condabin/env/mobsuite3.1.8/lib/python3.9/site-packages/mob_suite/utils.py:597]
2023-12-21 22:30:13,094 mob_suite.mob_recon INFO: Processing fasta file sample1.fasta [in /usr/local/bin/miniconda/condabin/env/mobsuite3.1.8/lib/python3.9/site-packages/mob_suite/mob_recon.py:1008]
2023-12-21 22:30:13,094 mob_suite.mob_recon INFO: Analysis directory mob_recon/sample1 [in /usr/local/bin/miniconda/condabin/env/mobsuite3.1.8/lib/python3.9/site-packages/mob_suite/mob_recon.py:1009]
2023-12-21 22:30:15,538 mob_suite.mob_recon INFO: Writing cleaned header input fasta file from sample1.fasta to mob_recon/sample1/__tmp/fixed.input.fasta [in /usr/local/bin/miniconda/condabin/env/mobsuite3.1.8/lib/python3.9/site-packages/mob_suite/mob_recon.py:1104]
2023-12-21 22:30:15,776 root INFO: Blasting replicon sequences /mnt/data/databases/mobsuite/db_20231221/rep.dna.fas against mob_recon/sample1/__tmp/fixed.input.fasta [in /usr/local/bin/miniconda/condabin/env/mobsuite3.1.8/lib/python3.9/site-packages/mob_suite/utils.py:1156]
Traceback (most recent call last):
  File "/usr/local/bin/miniconda/condabin/env/mobsuite3.1.8/bin/mob_recon", line 10, in <module>
    sys.exit(main())
  File "/usr/local/bin/miniconda/condabin/env/mobsuite3.1.8/lib/python3.9/site-packages/mob_suite/mob_recon.py", line 1148, in main
    identify_biomarkers(contig_info, fixed_fasta, tmp_dir, min_length, logging, \
  File "/usr/local/bin/miniconda/condabin/env/mobsuite3.1.8/lib/python3.9/site-packages/mob_suite/utils.py", line 1157, in identify_biomarkers
    blastn(input_fasta=replicon_ref, blastdb=fixed_fasta, min_ident=min_rep_ident, min_cov=min_rep_cov,
  File "/usr/local/bin/miniconda/condabin/env/mobsuite3.1.8/lib/python3.9/site-packages/mob_suite/utils.py", line 338, in blastn
    blast_df.to_csv(blast_results_file, sep='\t', header=True, line_terminator='\n', index=False)
TypeError: to_csv() got an unexpected keyword argument 'line_terminator'

Result: only the temporary directory __tmp appears in the output directory.

kbessonov1984 commented 11 months ago

What is your pandas version? If it is conda environment, list all your packages.

We recommend pandas 1.0.5. This error is due to incompatible pandas version which does not have this line_terminator parameter in its to_csv() function. Unfortunately pandas versions are not always back compatible. MOBSuite does not support pandas>=2.0 which have this parameter written as lineterminator (without underscore).

From pandas documentation 2.1.4: lineterminator str, optional Changed in version 1.5.0: Previously was line_terminator, changed for consistency with read_csv and the standard library ‘csv’ module.

wanyuac commented 11 months ago

Thank you for your prompt reply. It is pandas v1.5.3 in my conda environment.

$ conda list
# packages in environment at /usr/local/bin/miniconda/condabin/env/mobsuite3.1.8:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
alsa-lib                  1.2.10               hd590300_0    conda-forge
archspec                  0.2.2              pyhd8ed1ab_0    conda-forge
attr                      2.5.1                h166bdaf_1    conda-forge
biopython                 1.81             py39hd1e30aa_1    conda-forge
blast                     2.14.1          pl5321h6f7f691_0    bioconda
blosc                     1.21.5               h0f2a231_0    conda-forge
boltons                   23.0.0             pyhd8ed1ab_0    conda-forge
brotli-python             1.1.0            py39h3d6467e_1    conda-forge
bzip2                     1.0.8                hd590300_5    conda-forge
c-ares                    1.24.0               hd590300_0    conda-forge
c-blosc2                  2.11.3               hb4ffafa_0    conda-forge
ca-certificates           2023.11.17           hbcca054_0    conda-forge
cairo                     1.18.0               h3faef2a_0    conda-forge
capnproto                 0.9.1                ha19adfc_4    conda-forge
certifi                   2023.11.17         pyhd8ed1ab_0    conda-forge
cffi                      1.16.0           py39h7a31438_0    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
conda                     23.11.0          py39hf3d152e_1    conda-forge
conda-libmamba-solver     23.12.0            pyhd8ed1ab_0    conda-forge
conda-package-handling    2.2.0              pyh38be061_0    conda-forge
conda-package-streaming   0.9.0              pyhd8ed1ab_0    conda-forge
curl                      8.5.0                hca28451_0    conda-forge
dbus                      1.13.6               h5008d03_3    conda-forge
distro                    1.8.0              pyhd8ed1ab_0    conda-forge
entrez-direct             16.2                 he881be0_1    bioconda
ete3                      3.1.3              pyhd8ed1ab_0    conda-forge
expat                     2.5.0                hcb278e6_1    conda-forge
fmt                       10.1.1               h00ab1b0_1    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 h77eed37_1    conda-forge
fontconfig                2.14.2               h14ed4e7_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
gettext                   0.21.1               h27087fc_0    conda-forge
glib                      2.78.3               hfc55251_0    conda-forge
glib-tools                2.78.3               hfc55251_0    conda-forge
graphite2                 1.3.13            h58526e2_1001    conda-forge
gsl                       2.7                  he838d99_0    conda-forge
gst-plugins-base          1.22.8               h8e1006c_0    conda-forge
gstreamer                 1.22.8               h98fc4e7_0    conda-forge
harfbuzz                  8.3.0                h3d44ed6_0    conda-forge
hdf5                      1.14.3          nompi_h4f84152_100    conda-forge
icu                       73.2                 h59595ed_0    conda-forge
idna                      3.6                pyhd8ed1ab_0    conda-forge
jsonpatch                 1.33               pyhd8ed1ab_0    conda-forge
jsonpointer               2.4              py39hf3d152e_3    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
krb5                      1.21.2               h659d440_0    conda-forge
lame                      3.100             h166bdaf_1003    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
libaec                    1.1.2                h59595ed_1    conda-forge
libarchive                3.7.2                h2aa1ff5_1    conda-forge
libblas                   3.9.0           20_linux64_openblas    conda-forge
libcap                    2.69                 h0f662aa_0    conda-forge
libcblas                  3.9.0           20_linux64_openblas    conda-forge
libclang                  15.0.7          default_hb11cfb5_4    conda-forge
libclang13                15.0.7          default_ha2b6cf4_4    conda-forge
libcups                   2.3.3                h4637d8d_4    conda-forge
libcurl                   8.5.0                hca28451_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 hd590300_2    conda-forge
libevent                  2.1.12               hf998b51_1    conda-forge
libexpat                  2.5.0                hcb278e6_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libflac                   1.4.3                h59595ed_0    conda-forge
libgcc-ng                 13.2.0               h807b86a_3    conda-forge
libgcrypt                 1.10.3               hd590300_0    conda-forge
libgfortran-ng            13.2.0               h69a702a_3    conda-forge
libgfortran5              13.2.0               ha4646dd_3    conda-forge
libglib                   2.78.3               h783c2da_0    conda-forge
libgomp                   13.2.0               h807b86a_3    conda-forge
libgpg-error              1.47                 h71f35ed_0    conda-forge
libiconv                  1.17                 hd590300_2    conda-forge
libidn2                   2.3.4                h166bdaf_0    conda-forge
libjpeg-turbo             3.0.0                hd590300_1    conda-forge
liblapack                 3.9.0           20_linux64_openblas    conda-forge
libllvm15                 15.0.7               hb3ce162_4    conda-forge
libmamba                  1.5.5                had39da4_0    conda-forge
libmambapy                1.5.5            py39h10defb6_0    conda-forge
libnghttp2                1.58.0               h47da74e_1    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libogg                    1.3.4                h7f98852_1    conda-forge
libopenblas               0.3.25          pthreads_h413a1c8_0    conda-forge
libopus                   1.3.1                h7f98852_1    conda-forge
libpng                    1.6.39               h753d276_0    conda-forge
libpq                     16.1                 h33b98f1_7    conda-forge
libsndfile                1.2.2                hc60ed4a_1    conda-forge
libsolv                   0.7.27               hfc55251_0    conda-forge
libsqlite                 3.44.2               h2797004_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx-ng              13.2.0               h7e041cc_3    conda-forge
libsystemd0               255                  h3516f8a_0    conda-forge
libunistring              0.9.10               h7f98852_0    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libvorbis                 1.3.7                h9c3ff4c_0    conda-forge
libxcb                    1.15                 h0b41bf4_0    conda-forge
libxkbcommon              1.6.0                hd429924_1    conda-forge
libxml2                   2.12.3               h232c23b_0    conda-forge
libxslt                   1.1.39               h76b75d6_0    conda-forge
libzlib                   1.2.13               hd590300_5    conda-forge
lxml                      4.9.1            py39hb9d737c_0    conda-forge
lz4-c                     1.9.4                hcb278e6_0    conda-forge
lzo                       2.10              h516909a_1000    conda-forge
mamba                     1.5.5            py39hc5d2bb1_0    conda-forge
mash                      2.3                  ha9a2dd8_3    bioconda
menuinst                  2.0.1            py39hf3d152e_0    conda-forge
mob_suite                 3.1.8              pyhdfd78af_0    bioconda
mpg123                    1.32.3               h59595ed_0    conda-forge
mysql-common              8.0.33               hf1915f5_6    conda-forge
mysql-libs                8.0.33               hca2cd23_6    conda-forge
ncbi-vdb                  3.0.0           pl5321h87f3376_0    bioconda
ncurses                   6.4                  h59595ed_2    conda-forge
nomkl                     1.0                  h5ca1d4c_0    conda-forge
nspr                      4.35                 h27087fc_0    conda-forge
nss                       3.96                 h1d7d5a4_0    conda-forge
numexpr                   2.8.7           py39hb68c0d3_104    conda-forge
numpy                     1.26.2           py39h474f0d3_0    conda-forge
openssl                   3.2.0                hd590300_1    conda-forge
packaging                 23.2               pyhd8ed1ab_0    conda-forge
pandas                    1.5.3            py39h2ad29b5_1    conda-forge
pcre                      8.45                 h9c3ff4c_0    conda-forge
pcre2                     10.42                hcad00b1_0    conda-forge
perl                      5.32.1          4_hd590300_perl5    conda-forge
perl-archive-tar          2.40            pl5321hdfd78af_0    bioconda
perl-carp                 1.50            pl5321hd8ed1ab_0    conda-forge
perl-common-sense         3.75            pl5321hd8ed1ab_0    conda-forge
perl-compress-raw-bzip2   2.201           pl5321h166bdaf_0    conda-forge
perl-compress-raw-zlib    2.202           pl5321h166bdaf_0    conda-forge
perl-encode               3.19            pl5321h166bdaf_0    conda-forge
perl-exporter             5.74            pl5321hd8ed1ab_0    conda-forge
perl-exporter-tiny        1.002002        pl5321hd8ed1ab_0    conda-forge
perl-extutils-makemaker   7.70            pl5321hd8ed1ab_0    conda-forge
perl-io-compress          2.201           pl5321hdbdd923_2    bioconda
perl-io-zlib              1.14            pl5321hdfd78af_0    bioconda
perl-json                 4.10            pl5321hdfd78af_0    bioconda
perl-json-xs              2.34            pl5321h4ac6f70_6    bioconda
perl-list-moreutils       0.430           pl5321hdfd78af_0    bioconda
perl-list-moreutils-xs    0.430           pl5321h031d066_2    bioconda
perl-parent               0.241           pl5321hd8ed1ab_0    conda-forge
perl-pathtools            3.75            pl5321h166bdaf_0    conda-forge
perl-scalar-list-utils    1.63            pl5321h166bdaf_0    conda-forge
perl-storable             3.15            pl5321h166bdaf_0    conda-forge
perl-types-serialiser     1.01            pl5321hdfd78af_0    bioconda
pip                       23.3.2             pyhd8ed1ab_0    conda-forge
pixman                    0.42.2               h59595ed_0    conda-forge
platformdirs              4.1.0              pyhd8ed1ab_0    conda-forge
pluggy                    1.3.0              pyhd8ed1ab_0    conda-forge
ply                       3.11                       py_1    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
pulseaudio-client         16.1                 hb77b528_5    conda-forge
py-cpuinfo                9.0.0              pyhd8ed1ab_0    conda-forge
pybind11-abi              4                    hd8ed1ab_3    conda-forge
pycosat                   0.6.6            py39hd1e30aa_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pycurl                    7.45.1           py39h9297c8b_3    conda-forge
pyqt                      5.15.9           py39h52134e7_5    conda-forge
pyqt5-sip                 12.12.2          py39h3d6467e_5    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
pytables                  3.9.2            py39hfbd31a7_1    conda-forge
python                    3.9.18          h0755675_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python_abi                3.9                      4_cp39    conda-forge
pytz                      2023.3.post1       pyhd8ed1ab_0    conda-forge
qt-main                   5.15.8              h450f30e_18    conda-forge
readline                  8.2                  h8228510_1    conda-forge
reproc                    14.2.4.post0         hd590300_1    conda-forge
reproc-cpp                14.2.4.post0         h59595ed_1    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
ruamel.yaml               0.18.5           py39hd1e30aa_0    conda-forge
ruamel.yaml.clib          0.2.7            py39hd1e30aa_2    conda-forge
scipy                     1.11.4           py39h474f0d3_0    conda-forge
setuptools                68.2.2             pyhd8ed1ab_0    conda-forge
sip                       6.7.12           py39h3d6467e_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
snappy                    1.1.10               h9fff704_0    conda-forge
tk                        8.6.13          noxft_h4845f30_101    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
tqdm                      4.66.1             pyhd8ed1ab_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
urllib3                   2.1.0              pyhd8ed1ab_0    conda-forge
wget                      1.20.3               ha35d2d1_1    conda-forge
wheel                     0.42.0             pyhd8ed1ab_0    conda-forge
xcb-util                  0.4.0                hd590300_1    conda-forge
xcb-util-image            0.4.0                h8ee46fc_1    conda-forge
xcb-util-keysyms          0.4.0                h8ee46fc_1    conda-forge
xcb-util-renderutil       0.3.9                hd590300_1    conda-forge
xcb-util-wm               0.4.1                h8ee46fc_1    conda-forge
xkeyboard-config          2.40                 hd590300_0    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.1.1                hd590300_0    conda-forge
xorg-libsm                1.2.4                h7391055_0    conda-forge
xorg-libx11               1.8.7                h8ee46fc_0    conda-forge
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h0b41bf4_2    conda-forge
xorg-libxrender           0.9.11               hd590300_0    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-xextproto            7.3.0             h0b41bf4_1003    conda-forge
xorg-xf86vidmodeproto     2.3.1             h7f98852_1002    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
yaml-cpp                  0.8.0                h59595ed_0    conda-forge
zlib                      1.2.13               hd590300_5    conda-forge
zlib-ng                   2.0.7                h0b41bf4_0    conda-forge
zstandard                 0.22.0           py39h6e5214e_0    conda-forge
zstd                      1.5.5                hfc55251_0    conda-forge
kbessonov1984 commented 11 months ago

Downgrade to 1.5.0 or lower for pandas and issue will be solved. Will make a more restrictive conda recipe build 1 for this 3.1.8 version which will restrict pandas to 1.5.0. Thank you for reporting

wanyuac commented 11 months ago

Yes, as you suggested, this issue has been solved by installing pandas v1.5.0 first and then mob-suite v3.1.8. Thank you very much for helping me solve this problem so quickly.

kbessonov1984 commented 11 months ago

Updated mob_suite 3.1.8 conda recipe build #1 to limit pandas version to 1.5.3 upper bound and tested to make sure pandas 1.5.3 and lower versions work via a pytest incorporated into the conda build and test stages. The pandas 1.5.3 should of given deprecation waring instead of error FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead. as per this code decorator https://github.com/pandas-dev/pandas/blob/2e218d10984e9919f0296931d92ea851c6a6faf5/pandas/core/generic.py#L3541