Quast Running times.

[migrau]

Hi all, I am trying to use quast with a reference (360Mbp) and one new assembly (450Mbp). Can the size be a problem? The reference come from illumina reads and the new assembly from pacbio. It is running on a cluster (--threads 24) for three days, without results yet. Reading your supplementary information, it should not take that long, right?

Thank you,

[alexeigurevich]

Hi! Normally, the analysis should finish much faster than 3 days for such reference length. So, probably something is going wrong. Could you send us the log files for the detailed analysis (quast.log and contigs_reports/.stdout and .stderr files under )?

Our support email is quast.support@bioinf.spbau.ru

[migrau]

Thanks, Alexei. I sent an email with the log files.

[wl13]

Do you have any answer for this issue? My quast has already run for over a month. The newest file (contigsreport**.mis_contigs.info) was generated about 20 days ago. I have not seen any newer results since then. My code was like:

quast.py contigs.fasta -R reference.fasta -G reference.gff3 -e --threads 8 -o out

The assembled file was ~400Mbp with ~370Mbp reference genome. No error reported in quast.log, only find a warning in contigs_reports/contigsreport*.stderr

WARNING: Unable to run 'gnuplot contigs_reports/nucmer_output/contigs-quiver.gp', Inappropriate ioctl for device

No error found in stderr_part** files either, and quast is still running :-) I pasted several log files here, any suggestion is appreciated. Thanks!

quast.log

/home/wl/lib/python2.7/site-packages/quast-4.5-py2.7.egg/EGG-INFO/scripts/quast.py contigs.fasta -R reference.fasta -G reference.gff3 -e --threads 8 -o out

Version: 4.5

System information: OS: Linux-2.6.32-642.13.1.el6.x86_64-x86_64-with-centos-6.8-Final (linux_64) Python version: 2.7.9 CPUs number: 32

Started: 2017-03-20 19:30:13

Logging to out/quast.log NOTICE: Output directory already exists. Existing Nucmer alignments can be used

CWD: out Main parameters: Threads: 8, eukaryotic: true, minimum contig length: 500, ambiguity: one, threshold for extensive misassembly size: 1000

Reference: reference.fasta ==> reference

Contigs: Pre-processing... contigs.fasta ==> contigs

2017-03-20 19:30:45 Running Basic statistics processor... Reference genome: reference.fasta, length = 373245519, num fragments = 12, GC % = 43.56 Contig files: contigs Calculating N50 and L50... contigs, N50 = 927700, L50 = 132, Total length = 399862497, GC % = 43.72, # N's per 100 kbp = 0.00 Drawing Nx plot... saved to out/basic_stats/Nx_plot.pdf Drawing NGx plot... saved to out/basic_stats/NGx_plot.pdf Drawing cumulative plot... saved to out/basic_stats/cumulative_plot.pdf Drawing GC content plot... saved to out/basic_stats/GC_content_plot.pdf Drawing contigs GC content plot... saved to out/basic_stats/contigs_GC_content_plot.pdf Done.

2017-03-20 19:31:20 Running Contig analyzer... contigs Logging to files out/contigs_reports/contigs_report_contigs.stdout and contigs_report_contigs.stderr... Aligning contigs to the reference Aligning to different chromosomes in parallel (8 threads) /home/wl/lib/python2.7/site-packages/quast-4.5-py2.7.egg/quast_libs/MUMmer/nucmer \ -c 65 -l 65 --maxmatch -p contigs_reports/nucmer_output/contigs_part_1.fasta \ --emem -t 1 quast_corrected_input/split_ref/part_1.fasta quast_corrected_input/contigs.fasta \ >> contigs_reports/contigs_report_contigs.stdout \ 2>> contigs_reports/contigs_report_contigs.stderr_part1 /home/wl/lib/python2.7/site-packages/quast-4.5-py2.7.egg/quast_libs/MUMmer/nucmer \ -c 65 -l 65 --maxmatch -p contigs_reports/nucmer_output/contigs_part_2.fasta \ --emem -t 1 quast_corrected_input/split_ref/part_2.fasta quast_corrected_input/contigs.fasta \ >> contigs_reports/contigs_report_contigs.stdout \ 2>> contigs_reports/contigs_report_contigs.stderr_part2 /home/wl/lib/python2.7/site-packages/quast-4.5-py2.7.egg/quast_libs/MUMmer/nucmer \ -c 65 -l 65 --maxmatch -p contigs_reports/nucmer_output/contigs_part_3.fasta \ --emem -t 1 quast_corrected_input/split_ref/part_3.fasta quast_corrected_input/contigs.fasta \ >> contigs_reports/contigs_report_contigs.stdout \ 2>> contigs_reports/contigs_report_contigs.stderr_part3 /home/wl/lib/python2.7/site-packages/quast-4.5-py2.7.egg/quast_libs/MUMmer/nucmer \ -c 65 -l 65 --maxmatch -p contigs_reports/nucmer_output/contigs_part_4.fasta \ --emem -t 1 quast_corrected_input/split_ref/part_4.fasta quast_corrected_input/contigs.fasta \ >> contigs_reports/contigs_report_contigs.stdout \ 2>> contigs_reports/contigs_report_contigs.stderr_part4 /home/wl/lib/python2.7/site-packages/quast-4.5-py2.7.egg/quast_libs/MUMmer/nucmer \ -c 65 -l 65 --maxmatch -p contigs_reports/nucmer_output/contigs_part_5.fasta \ --emem -t 1 quast_corrected_input/split_ref/part_5.fasta quast_corrected_input/contigs.fasta \ >> contigs_reports/contigs_report_contigs.stdout \ 2>> contigs_reports/contigs_report_contigs.stderr_part5 /home/wl/lib/python2.7/site-packages/quast-4.5-py2.7.egg/quast_libs/MUMmer/nucmer \ -c 65 -l 65 --maxmatch -p contigs_reports/nucmer_output/contigs_part_8.fasta \ --emem -t 1 quast_corrected_input/split_ref/part_8.fasta quast_corrected_input/contigs.fasta \ >> contigs_reports/contigs_report_contigs.stdout \ 2>> contigs_reports/contigs_report_contigs.stderr_part8 /home/wl/lib/python2.7/site-packages/quast-4.5-py2.7.egg/quast_libs/MUMmer/nucmer \ -c 65 -l 65 --maxmatch -p contigs_reports/nucmer_output/contigs_part_7.fasta \ --emem -t 1 quast_corrected_input/split_ref/part_7.fasta quast_corrected_input/contigs.fasta \ >> contigs_reports/contigs_report_contigs.stdout \ 2>> contigs_reports/contigs_report_contigs.stderr_part7 /home/wl/lib/python2.7/site-packages/quast-4.5-py2.7.egg/quast_libs/MUMmer/nucmer \ -c 65 -l 65 --maxmatch -p contigs_reports/nucmer_output/contigs_part_6.fasta \ --emem -t 1 quast_corrected_input/split_ref/part_6.fasta quast_corrected_input/contigs.fasta \ >> contigs_reports/contigs_report_contigs.stdout \ 2>> contigs_reports/contigs_report_contigs.stderr_part6 /home/wl/lib/python2.7/site-packages/quast-4.5-py2.7.egg/quast_libs/MUMmer/nucmer \ -c 65 -l 65 --maxmatch -p contigs_reports/nucmer_output/contigs_part_9.fasta \ --emem -t 1 quast_corrected_input/split_ref/part_9.fasta quast_corrected_input/contigs.fasta \ >> contigs_reports/contigs_report_contigs.stdout \ 2>> contigs_reports/contigs_report_contigs.stderr_part9 /home/wl/lib/python2.7/site-packages/quast-4.5-py2.7.egg/quast_libs/MUMmer/nucmer \ -c 65 -l 65 --maxmatch -p contigs_reports/nucmer_output/contigs_part_10.fasta \ --emem -t 1 quast_corrected_input/split_ref/part_10.fasta quast_corrected_input/contigs.fasta \ >> contigs_reports/contigs_report_contigs.stdout \ 2>> contigs_reports/contigs_report_contigs.stderr_part10 /home/wl/lib/python2.7/site-packages/quast-4.5-py2.7.egg/quast_libs/MUMmer/nucmer \ -c 65 -l 65 --maxmatch -p contigs_reports/nucmer_output/contigs_part_11.fasta \ --emem -t 1 quast_corrected_input/split_ref/part_11.fasta quast_corrected_input/contigs.fasta \ >> contigs_reports/contigs_report_contigs.stdout \ 2>> contigs_reports/contigs_report_contigs.stderr_part11 /home/wl/lib/python2.7/site-packages/quast-4.5-py2.7.egg/quast_libs/MUMmer/delta-filter \ -i 95.0 -l 0 contigs_reports/nucmer_output/contigs.delta \ > contigs_reports/nucmer_output/contigs.fdelta \ 2> contigs_reports/contigs_report_contigs.stderr /home/wl/lib/python2.7/site-packages/quast-4.5-py2.7.egg/quast_libs/MUMmer/mummerplot \ --html --layout -p contigs_reports/nucmer_output/contigs \ contigs_reports/nucmer_output/contigs.delta > contigs_reports/contigs_report_contigs.stdout \ 2> contigs_reports/contigs_report_contigs.stderr /home/wl/lib/python2.7/site-packages/quast-4.5-py2.7.egg/quast_libs/gnuplot/src/gnuplot \ contigs_reports/nucmer_output/contigs.gp > /dev/null \ 2> contigs_reports/contigs_report_contigs.stderr MUMmer plot saved to out/contigs_reports/contigs_mummerplot.html /home/wl/lib/python2.7/site-packages/quast-4.5-py2.7.egg/quast_libs/MUMmer/show-coords \ contigs_reports/nucmer_output/contigs.delta > contigs_reports/nucmer_output/contigs.coords_tmp \ 2> contigs_reports/contigs_report_contigs.stderr /home/wl/lib/python2.7/site-packages/quast-4.5-py2.7.egg/quast_libs/MUMmer/show-snps \ -S -T -H contigs_reports/nucmer_output/contigs.delta \ < contigs_reports/nucmer_output/contigs.coords.headless \ > contigs_reports/nucmer_output/contigs.all_snps \ 2> contigs_reports/contigs_report_contigs.stderr

contigs_report_contigs.stderr:

Writing filtered delta file contigs_reports/nucmer_output/contigs.filter Reading delta file contigs_reports/nucmer_output/contigs.delta Writing plot files contigs_reports/nucmer_output/contigs.fplot, contigs_reports/nucmer_output/contigs.rplot Writing gnuplot script contigs_reports/nucmer_output/contigs.gp Rendering plot contigs_reports/nucmer_output/contigs.html

set terminal canvas jsdir "" ^ "contigs_reports/nucmer_output/contigs.gp", line 1: unknown or ambiguous terminal type; type just 'set terminal' for a list

WARNING: Unable to run 'gnuplot contigs_reports/nucmer_output/contigs.gp', Inappropriate ioctl for device Stderr outputs for reference parts are in: contigs_reports/contigs_report_contigs.stderr_part1 contigs_reports/contigs_report_contigs.stderr_part2 contigs_reports/contigs_report_contigs.stderr_part3 ... ... contigs_reports/contigs_report_contigs.stderr_part11

content of the stderr_part files:

1: PREPARING DATA 2,3: RUNNING e-mem AND CREATING CLUSTERS 4: FINISHING DATA

[wl13]

Here is the CPU and memory usage. It just cost over 100G memory and seems run forever ...

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2386 wl 20 0 114g 113g 2028 R 99.4 15.0 49110:48 quast.py

[alexeigurevich]

Hi Wang,

The problem may be specific for you particular dataset. I will copy here part of my answer to Miquel (the author of this issue) which I wrote after analysing his logs and data. The Miquel problem was in very fragmented reference genome which is not your case (your reference has just 12 fragments, probably chromosomes). However, the following piece of advice may be appropriate for your case too:

"You can use more strict alignment settings to focus on longer and accurate seeds, this also result in faster processing. For doing it, you should use --min-cluster 200 (or another number). Default value for minimal exact match is 65 bp, the aligner finds all such seed and combine them into the full scale alignment. In your case default 65 bp size results in too many matches and slow processing. 200 bp should be fine."

Let me know how it goes. You could also write to our support email (quast.support@bioinf.spbau.ru)

[wl13]

Many thanks Alexei! I have re-run quast with "--min-cluster 200" and "--min-cluster 500". It has now been run for a whole day. I'll return to you once any job is finished.

[blostein]

Hi! I am wondering if I am having a similar problem or something different. Quast has been hanging at 'Running contig analyzer' for ~ 2 days (last logged output and modified files 2 days ago). I'm running on 2 megahit assemblies of metagenomic communities (both 1.1G) and my combined reference is 2.1G. There were no obvious errors in the stderr files (attached below ).

I did get a warning that my reference is fragmented, Quast no longer appears to have --min-cluster option (threw an error when I tried to use it), but I tried rerunning with --fragmented (skipping metaquast steps, and just rerunning from quast .py with the combined reference). Now, I'm getting a Key Error (KeyError: 'Streptococcus_mitis_AP023349.1' ERROR! exception caught!) in contigs_analyzer.py when calling pool.py (log files attached below )

Thanks in advance for any help or suggestions!

Original code: metaquast -o MegahitQuastResults -l megahitdefault,megahitlarge megahit_default/final.contigs.fa megahit_large/final.contigs.fa Original log & error files: RunMetaQuast-19208132.log.txt contigs_report_megahitlarge.stderr.txt quast.log.txt

Code with fragmented option: /home/blostein/.conda/envs/meta_assembly/opt/quast-5.0.2/quast.py --combined-ref --fragmented --ambiguity-usage all megahit_default/final.contigs.fa megahit_large/final.contigs.fa -R /gpfs/accounts/bfoxman_root/bfoxman/blostein/meta_CAVITIES/Files/newPipeline/MegahitQuastResults/quast_corrected_input/combined_reference.fasta -o /gpfs/accounts/bfoxman_root/bfoxman/blostein/meta_CAVITIES/Files/newPipeline/MegahitQuastResults/combined_reference_fragmented --labels "megahitdefault_f, megahitlarge_f”

Log & error files w/fragmented options RunQuastFrag-19293196.log.txt fragemented_quast.log.txt contigs_report_megahitlarge_f.stderr.txt

[benyoung93]

@blostein i know your comment was old but I was also having a similar issue. From the manual I think --min-cluster was replaced with --min alignment. Here is the description of --min-alignmnet and it seems to match the 65bp mentioned above.

--min-alignment (or -i) Minimum length of alignment (in bp). Alignments shorter than this value will be filtered. Note that all alignments shorter than 65 bp will be filtered regardless of this threshold.