search

biobakery / phylophlan

Precise phylogenetic analysis of microbial isolates and genomes from metagenomes
https://huttenhower.sph.harvard.edu/phylophlan
MIT License
128 stars 33 forks source link

Error when running for Enterococcus faecium #82

Closed llk578496 closed 2 years ago

llk578496 commented 2 years ago

Hi We got an error while running the analysis for Enterococcus faecium. We have this running normally about 2 months ago, but now we got an error using the exact same command and version (v3.0.2). May I know how could we solve this problem?

Thank you very much!

(phylophlan) mir@PYN-ANPT:~/VREfm_wd$ phylophlan -i 12_phylophlan/12.4_input_isolates -o 12_phylophlan/12.5_output_isolates -d 12_phylophlan/12.2_db/s__Enterococcus_faecium --trim greedy --not_variant_threshold 0.99 --remove_fragmentary_entries --fragmentary_threshold 0.67 --min_num_entries 11 -t a -f 12_phylophlan/12.3_configs/isolates_config.cfg --diversity low --force_nucleotides --nproc 20 --verbose 2>&1 | tee 12_phylophlan/12.1_logs/phylophlan_output_isolates.log PhyloPhlAn version 3.0.60 (27 November 2020)

Command line: /home/mir/anaconda3/envs/phylophlan/bin/phylophlan -i 12_phylophlan/12.4_input_isolates -o 12_phylophlan/12.5_output_isolates -d 12_phylophlan/12.2_db/s__Enterococcus_faecium --trim greedy --not_variant_threshold 0.99 --remove_fragmentary_entries --fragmentary_threshold 0.67 --min_num_entries 11 -t a -f 12_phylophlan/12.3_configs/isolates_config.cfg --diversity low --force_nucleotides --nproc 20 --verbose

Automatically setting "database=sEnterococcus_faecium" and "databases_folder=/home/mir/VREfm_wd/12_phylophlan/12.2_db" Automatically setting "input=12.4_input_isolates" and "input_folder=/home/mir/VREfm_wd/12_phylophlan" [e] "/home/mir/anaconda3/envs/phylophlan/lib/python3.10/site-packages/phylophlan/phylophlan_configs/" folder does not exists Creating folder "12_phylophlan/12.5_output_isolates" Creating folder "12_phylophlan/12.5_output_isolates/tmp" "low-accurate" preset Arguments: {'input': '12.4_input_isolates', 'clean': None, 'output': '12_phylophlan/12.5_output_isolates', 'database': 'sEnterococcus_faecium', 'db_type': 'a', 'config_file': '12_phylophlan/12.3_configs/isolates_config.cfg', 'diversity': 'low', 'accurate': True, 'fast': False, 'clean_all': False, 'database_list': False, 'submat': 'pfasum60', 'submat_list': False, 'submod_list': False, 'nproc': 20, 'min_num_proteins': 1, 'min_len_protein': 50, 'min_num_markers': 1, 'trim': 'greedy', 'gap_perc_threshold': 0.67, 'not_variant_threshold': 0.99, 'subsample': None, 'unknown_fraction': 0.3, 'scoring_function': None, 'sort': False, 'remove_fragmentary_entries': True, 'fragmentary_threshold': 0.67, 'min_num_entries': 11, 'maas': None, 'remove_only_gaps_entries': False, 'mutation_rates': False, 'force_nucleotides': True, 'input_folder': '/home/mir/VREfm_wd/12_phylophlan/12.4_input_isolates', 'data_folder': '12_phylophlan/12.5_output_isolates/tmp', 'databases_folder': '/home/mir/VREfm_wd/12_phylophlan/12.2_db', 'submat_folder': '/home/mir/anaconda3/envs/phylophlan/lib/python3.10/site-packages/phylophlan/phylophlan_substitution_matrices/', 'submod_folder': '/home/mir/anaconda3/envs/phylophlan/lib/python3.10/site-packages/phylophlan/phylophlan_substitution_models/', 'configs_folder': None, 'output_folder': '', 'genome_extension': '.fna', 'proteome_extension': '.faa', 'update': False, 'verbose': True} Loading configuration file "12_phylophlan/12.3_configs/isolates_config.cfg" Checking configuration file Checking "/home/mir/anaconda3/envs/phylophlan/bin/diamond" Checking "/home/mir/anaconda3/envs/phylophlan/bin/mafft" Checking "/home/mir/anaconda3/envs/phylophlan/bin/trimal" Checking "/home/mir/anaconda3/envs/phylophlan/bin/FastTreeMP" Checking "/home/mir/anaconda3/envs/phylophlan/bin/raxmlHPC-PTHREADS-SSE3" File "/home/mir/VREfm_wd/12_phylophlan/12.2_db/sEnterococcus_faecium/sEnterococcus_faecium.faa" present Generating "db_aa" indexed database "sEnterococcus_faecium" "db_aa" ("s__Enterococcus_faecium") generated Loading files from "/home/mir/VREfm_wd/12_phylophlan/12.4_input_isolates" Creating folder "12_phylophlan/12.5_output_isolates/tmp/clean_dna" Cleaning 12 inputs Cleaning "/home/mir/VREfm_wd/12_phylophlan/12.4_input_isolates/D10Case00007-21MB786389.consensus.masked.subs.chr.fna" Cleaning "/home/mir/VREfm_wd/12_phylophlan/12.4_input_isolates/D10Case00010-21MB787262.consensus.masked.subs.chr.fna" Cleaning "/home/mir/VREfm_wd/12_phylophlan/12.4_input_isolates/D10Case00006-21MB786210.consensus.masked.subs.chr.fna" Cleaning "/home/mir/VREfm_wd/12_phylophlan/12.4_input_isolates/D10Case00011-21MB802711.consensus.masked.subs.chr.fna" Cleaning "/home/mir/VREfm_wd/12_phylophlan/12.4_input_isolates/D10Case00001-21MB890613.consensus.masked.subs.chr.fna" Cleaning "/home/mir/VREfm_wd/12_phylophlan/12.4_input_isolates/D10Case00005-21MB785702.consensus.masked.subs.chr.fna" Cleaning "/home/mir/VREfm_wd/12_phylophlan/12.4_input_isolates/D10Case00008-21MB787476.consensus.masked.subs.chr.fna" Cleaning "/home/mir/VREfm_wd/12_phylophlan/12.4_input_isolates/D10Case00002-21MB890617.consensus.masked.subs.chr.fna" Cleaning "/home/mir/VREfm_wd/12_phylophlan/12.4_input_isolates/D10Case00004-21MB785422.consensus.masked.subs.chr.fna" Cleaning "/home/mir/VREfm_wd/12_phylophlan/12.4_input_isolates/D10Case00003-21MB785426.consensus.masked.subs.chr.fna" Cleaning "/home/mir/VREfm_wd/12_phylophlan/12.4_input_isolates/GCF_012933345.2_ASM1293334v2_genomic.chr.fna" Cleaning "/home/mir/VREfm_wd/12_phylophlan/12.4_input_isolates/D10Case00009-21MB787257.consensus.masked.subs.chr.fna" "12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00008-21MB787476.consensus.masked.subs.chr.fna" generated in 0s "12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00004-21MB785422.consensus.masked.subs.chr.fna" generated in 0s "12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00009-21MB787257.consensus.masked.subs.chr.fna" generated in 0s "12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00007-21MB786389.consensus.masked.subs.chr.fna" generated in 0s "12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00006-21MB786210.consensus.masked.subs.chr.fna" generated in 0s "12_phylophlan/12.5_output_isolates/tmp/clean_dna/GCF_012933345.2_ASM1293334v2_genomic.chr.fna" generated in 0s "12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00010-21MB787262.consensus.masked.subs.chr.fna" generated in 0s "12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00003-21MB785426.consensus.masked.subs.chr.fna" generated in 0s "12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00001-21MB890613.consensus.masked.subs.chr.fna" generated in 0s "12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00011-21MB802711.consensus.masked.subs.chr.fna" generated in 0s "12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00002-21MB890617.consensus.masked.subs.chr.fna" generated in 0s "12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00005-21MB785702.consensus.masked.subs.chr.fna" generated in 0s Loading files from "12_phylophlan/12.5_output_isolates/tmp/clean_dna" Creating folder "12_phylophlan/12.5_output_isolates/tmp/map_dna" Mapping "sEnterococcus_faecium" on 12 inputs (key: "map_dna") Mapping "12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00007-21MB786389.consensus.masked.subs.chr.fna" Mapping "12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00010-21MB787262.consensus.masked.subs.chr.fna" Mapping "12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00006-21MB786210.consensus.masked.subs.chr.fna" Mapping "12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00011-21MB802711.consensus.masked.subs.chr.fna" Mapping "12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00001-21MB890613.consensus.masked.subs.chr.fna" Mapping "12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00005-21MB785702.consensus.masked.subs.chr.fna" Mapping "12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00008-21MB787476.consensus.masked.subs.chr.fna" Mapping "12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00003-21MB785426.consensus.masked.subs.chr.fna" Mapping "12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00002-21MB890617.consensus.masked.subs.chr.fna" Mapping "12_phylophlan/12.5_output_isolates/tmp/clean_dna/GCF_012933345.2_ASM1293334v2_genomic.chr.fna" Mapping "12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00009-21MB787257.consensus.masked.subs.chr.fna" Mapping "12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00004-21MB785422.consensus.masked.subs.chr.fna" Removing "12_phylophlan/12.5_output_isolates/tmp/map_dna/D10Case00010-21MB787262.consensus.masked.subs.chr.b6o.bkp"

[e] Command '['/home/mir/anaconda3/envs/phylophlan/bin/diamond', 'blastx', '--quiet', '--threads', '1', '--outfmt', '6', '--more-sensitive', '--id', '50', '--max-hsps', '35', '-k', '0', '--query', '12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00010-21MB787262.consensus.masked.subs.chr.fna', '--db', '/home/mir/VREfm_wd/12_phylophlan/12.2_db/s__Enterococcus_faecium/s__Enterococcus_faecium.dmnd', '--out', '12_phylophlan/12.5_output_isolates/tmp/map_dna/D10Case00010-21MB787262.consensus.masked.subs.chr.b6o.bkp']' died with <Signals.SIGKILL: 9>.

[e] cannot execute command command_line: /home/mir/anaconda3/envs/phylophlan/bin/diamond blastx --quiet --threads 1 --outfmt 6 --more-sensitive --id 50 --max-hsps 35 -k 0 --query 12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00010-21MB787262.consensus.masked.subs.chr.fna --db /home/mir/VREfm_wd/12_phylophlan/12.2_db/s__Enterococcus_faecium/s__Enterococcus_faecium.dmnd --out 12_phylophlan/12.5_output_isolates/tmp/map_dna/D10Case00010-21MB787262.consensus.masked.subs.chr.b6o.bkp stdin: None stdout: None env: {'LESSOPEN': '| /usr/bin/lesspipe %s', 'CONDA_PROMPT_MODIFIER': '(phylophlan) ', 'LANGUAGE': 'en_HK:en', 'USER': 'mir', 'SSH_AGENT_PID': '1831', 'XDG_SESSION_TYPE': 'x11', 'SHLVL': '0', 'HOME': '/home/mir', 'CONDA_SHLVL': '2', 'DESKTOP_SESSION': 'ubuntu', 'NVM_BIN': '/home/mir/.nvm/versions/node/v16.2.0/bin', 'NVM_INC': '/home/mir/.nvm/versions/node/v16.2.0/include/node', 'GNOME_SHELL_SESSION_MODE': 'ubuntu', 'GTK_MODULES': 'gail:atk-bridge', 'MANAGERPID': '1640', 'DBUS_STARTER_BUS_TYPE': 'session', 'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/1000/bus,guid=bc9d950cb4f901d071e9a4f761e15d32', 'COLORTERM': 'truecolor', '_CE_M': '', 'NVM_DIR': '/home/mir/.nvm', 'MANDATORY_PATH': '/usr/share/gconf/ubuntu.mandatory.path', 'IM_CONFIG_PHASE': '1', 'LOGNAME': 'mir', 'JOURNALSTREAM': '8:43658', '': '/home/mir/anaconda3/envs/phylophlan/bin/phylophlan', 'XDG_SESSION_CLASS': 'user', 'DEFAULTS_PATH': '/usr/share/gconf/ubuntu.default.path', 'USERNAME': 'mir', 'TERM': 'xterm-256color', 'GNOME_DESKTOP_SESSION_ID': 'this-is-deprecated', '_CE_CONDA': '', 'WINDOWPATH': '2', 'PATH': '/home/mir/.nvm/versions/node/v16.2.0/bin:/home/mir/anaconda3/envs/phylophlan/bin:/home/mir/anaconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/mir/gatk-4.2.0.0:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/mir/picard/build/libs', 'SESSION_MANAGER': 'local/PYN-ANPT:@/tmp/.ICE-unix/1874,unix/PYN-ANPT:/tmp/.ICE-unix/1874', 'INVOCATION_ID': 'ed0f6c33326640308265a3f5f7618995', 'XDG_MENU_PREFIX': 'gnome-', 'GNOME_TERMINAL_SCREEN': '/org/gnome/Terminal/screen/778ac088_eb09_4d6d_8d77_558d6faedfd6', 'XDG_RUNTIME_DIR': '/run/user/1000', 'DISPLAY': ':0', 'LANG': 'en_HK.UTF-8', 'XDG_CURRENT_DESKTOP': 'ubuntu:GNOME', 'CONDA_PREFIX_1': '/home/mir/anaconda3', 'XMODIFIERS': '@im=ibus', 'XDG_SESSION_DESKTOP': 'ubuntu', 'XAUTHORITY': '/run/user/1000/gdm/Xauthority', 'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:.tar=01;31:.tgz=01;31:.arc=01;31:.arj=01;31:.taz=01;31:.lha=01;31:.lz4=01;31:.lzh=01;31:.lzma=01;31:.tlz=01;31:.txz=01;31:.tzo=01;31:.t7z=01;31:.zip=01;31:.z=01;31:.dz=01;31:.gz=01;31:.lrz=01;31:.lz=01;31:.lzo=01;31:.xz=01;31:.zst=01;31:.tzst=01;31:.bz2=01;31:.bz=01;31:.tbz=01;31:.tbz2=01;31:.tz=01;31:.deb=01;31:.rpm=01;31:.jar=01;31:.war=01;31:.ear=01;31:.sar=01;31:.rar=01;31:.alz=01;31:.ace=01;31:.zoo=01;31:.cpio=01;31:.7z=01;31:.rz=01;31:.cab=01;31:.wim=01;31:.swm=01;31:.dwm=01;31:.esd=01;31:.jpg=01;35:.jpeg=01;35:.mjpg=01;35:.mjpeg=01;35:.gif=01;35:.bmp=01;35:.pbm=01;35:.pgm=01;35:.ppm=01;35:.tga=01;35:.xbm=01;35:.xpm=01;35:.tif=01;35:.tiff=01;35:.png=01;35:.svg=01;35:.svgz=01;35:.mng=01;35:.pcx=01;35:.mov=01;35:.mpg=01;35:.mpeg=01;35:.m2v=01;35:.mkv=01;35:.webm=01;35:.ogm=01;35:.mp4=01;35:.m4v=01;35:.mp4v=01;35:.vob=01;35:.qt=01;35:.nuv=01;35:.wmv=01;35:.asf=01;35:.rm=01;35:.rmvb=01;35:.flc=01;35:.avi=01;35:.fli=01;35:.flv=01;35:.gl=01;35:.dl=01;35:.xcf=01;35:.xwd=01;35:.yuv=01;35:.cgm=01;35:.emf=01;35:.ogv=01;35:.ogx=01;35:.aac=00;36:.au=00;36:.flac=00;36:.m4a=00;36:.mid=00;36:.midi=00;36:.mka=00;36:.mp3=00;36:.mpc=00;36:.ogg=00;36:.ra=00;36:.wav=00;36:.oga=00;36:.opus=00;36:.spx=00;36:*.xspf=00;36:', 'GNOME_TERMINAL_SERVICE': ':1.89', 'SSH_AUTH_SOCK': '/run/user/1000/keyring/ssh', 'CONDA_PYTHON_EXE': '/home/mir/anaconda3/bin/python', 'SHELL': '/bin/bash', 'QT_ACCESSIBILITY': '1', 'GDMSESSION': 'ubuntu', 'LESSCLOSE': '/usr/bin/lesspipe %s %s', 'CONDA_DEFAULT_ENV': 'phylophlan', 'GPG_AGENT_INFO': '/run/user/1000/gnupg/S.gpg-agent:0:1', 'QT_IM_MODULE': 'ibus', 'PWD': '/home/mir/VREfm_wd', 'XDG_CONFIG_DIRS': '/etc/xdg/xdg-ubuntu:/etc/xdg', 'CONDA_EXE': '/home/mir/anaconda3/bin/conda', 'NVM_CD_FLAGS': '', 'DBUS_STARTER_ADDRESS': 'unix:path=/run/user/1000/bus,guid=bc9d950cb4f901d071e9a4f761e15d32', 'XDG_DATA_DIRS': '/usr/share/ubuntu:/usr/local/share/:/usr/share/:/var/lib/snapd/desktop', 'CONDA_PREFIX': '/home/mir/anaconda3/envs/phylophlan', 'VTE_VERSION': '6003'}

[e] Command '['/home/mir/anaconda3/envs/phylophlan/bin/diamond', 'blastx', '--quiet', '--threads', '1', '--outfmt', '6', '--more-sensitive', '--id', '50', '--max-hsps', '35', '-k', '0', '--query', '12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00010-21MB787262.consensus.masked.subs.chr.fna', '--db', '/home/mir/VREfm_wd/12_phylophlan/12.2_db/s__Enterococcus_faecium/s__Enterococcus_faecium.dmnd', '--out', '12_phylophlan/12.5_output_isolates/tmp/map_dna/D10Case00010-21MB787262.consensus.masked.subs.chr.b6o.bkp']' died with <Signals.SIGKILL: 9>.

[e] error while mapping {'program_name': '/home/mir/anaconda3/envs/phylophlan/bin/diamond', 'params': 'blastx --quiet --threads 1 --outfmt 6 --more-sensitive --id 50 --max-hsps 35 -k 0', 'input': '--query', 'database': '--db', 'output': '--out', 'version': 'version', 'command_line': '#program_name# #params# #input# #database# #output#'} 12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00010-21MB787262.consensus.masked.subs.chr.fna /home/mir/VREfm_wd/12_phylophlan/12.2_db/s__Enterococcus_faecium/s__Enterococcus_faecium.dmnd 12_phylophlan/12.5_output_isolates/tmp/map_dna D10Case00010-21MB787262.consensus.masked.subs.chr.b6o.bkp True

[e] Command '['/home/mir/anaconda3/envs/phylophlan/bin/diamond', 'blastx', '--quiet', '--threads', '1', '--outfmt', '6', '--more-sensitive', '--id', '50', '--max-hsps', '35', '-k', '0', '--query', '12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00010-21MB787262.consensus.masked.subs.chr.fna', '--db', '/home/mir/VREfm_wd/12_phylophlan/12.2_db/s__Enterococcus_faecium/s__Enterococcus_faecium.dmnd', '--out', '12_phylophlan/12.5_output_isolates/tmp/map_dna/D10Case00010-21MB787262.consensus.masked.subs.chr.b6o.bkp']' died with <Signals.SIGKILL: 9>.

[e] gene_markers_identification crashed

fasnicar commented 2 years ago

Hi,

Have you tried running the diamond command to see what the error output is? (I just removed the --quiet)

home/mir/anaconda3/envs/phylophlan/bin/diamond blastx --threads 1 --outfmt 6 --more-sensitive --id 50 --max-hsps 35 -k 0 --query 12_phylophlan/12.5_output_isolates/tmp/clean_dna/D10Case00010-21MB787262.consensus.masked.subs.chr.fna --db /home/mir/VREfm_wd/12_phylophlan/12.2_db/s__Enterococcus_faecium/s__Enterococcus_faecium.dmnd --out 12_phylophlan/12.5_output_isolates/tmp/map_dna/D10Case00010-21MB787262.consensus.masked.subs.chr.b6o.bkp

I also see you're running version 3.0.60, it would be great if you can pull the newest version (3.0.64) from the Github repository and run that one, so it will be easier in case we need to do some debugging.

Note: Since you installed PhyloPhlAn from conda, you just need to clone the repo and run the scripts with the phylophlan conda env activated to have all the dependencies resolved.

Many thanks, Francesco

llk578496 commented 2 years ago

Hi @fasnicar , We have removed the environment built by conda and recreate the environment again. Phylophlan is working great now! Thank you very much!