search

alekseyzimin / masurca

GNU General Public License v3.0
244 stars 35 forks source link

mega-reads pass 1 on the grid exited, please re-run assemble.sh #279

Open jpinus opened 2 years ago

jpinus commented 2 years ago

I get the following error while running Masurca on the HPC of my University with my Illumina and ONT libraries:

[Mon Mar 14 13:49:13 CET 2022] Processing pe library reads
[Mon Mar 14 13:49:13 CET 2022] Average PE read length 94
[Mon Mar 14 13:49:13 CET 2022] Using kmer size of 65 for the graph
[Mon Mar 14 13:49:13 CET 2022] MIN_Q_CHAR: 33
WARNING: JF_SIZE set too low, increasing JF_SIZE to at least 318298594, this automatic increase may be not enough!
[Mon Mar 14 13:49:13 CET 2022] Estimated genome size: 474706202
[Mon Mar 14 13:49:13 CET 2022] Computing super reads from PE 
[Mon Mar 14 13:49:13 CET 2022] Using CABOG from /cluster/easybuild/broadwell/software/MaSuRCA/4.0.4-foss-2020a-Perl-5.30.2/bin/../bin
[Mon Mar 14 13:49:14 CET 2022] Running mega-reads correction/assembly
[Mon Mar 14 13:49:14 CET 2022] Using mer size 17 for mapping, B=15, d=0.02
[Mon Mar 14 13:49:14 CET 2022] Estimated Genome Size 474706202
[Mon Mar 14 13:49:14 CET 2022] Estimated Ploidy 1
[Mon Mar 14 13:49:14 CET 2022] Using 64 threads
[Mon Mar 14 13:49:14 CET 2022] Output prefix mr.41.17.15.0.02
[Mon Mar 14 13:49:14 CET 2022] Computing mega-reads
[Mon Mar 14 13:49:14 CET 2022] Running on the grid in 512 batches

To submit SLURM jobs, please run

sbatch -D /lustre/project/m2_jgu-symbeetle/Julian/ONT/PTRU/masurca/mr_pass1 -J create_mega_reads -a 1-512 -n 64 -p bigmem -N 1 mr_pass1/create_mega_reads.sh

Please re-run assemble.sh when all jobs finish. If you get this message again, it means that some jobs failed, simply re-submit again using the above command.

[Mon Mar 14 13:49:48 CET 2022] mega-reads pass 1 on the grid exited, please re-run assemble.sh
[Mon Mar 14 13:49:48 CET 2022] mega-reads exited before assembly

I changed the GRID_BATCH_SIZE= multiple times but its always the same error.

(base) [jkiefe01@login22 masurca]$ ls -lth
total 46G
drwxr-sr-x 2 jkiefe01 m2_jgu-symbeetle   32K Mar 14 13:49 mr_pass1
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle  1.6K Mar 14 13:49 out.PTRU_masurca10570085.x0902.log
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle    20 Mar 14 13:49 CA_dir.txt
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle  1022 Mar 14 13:49 environment.sh
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle    11 Mar 14 13:49 meanAndStdevByPrefix.pe.txt
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle  1.8M Mar 14 13:49 pe_data.tmp
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle     2 Mar 14 13:49 PLOIDY.txt
-rwxr-xr-x 1 jkiefe01 m2_jgu-symbeetle  6.5K Mar 14 13:48 assemble.sh
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle  4.4K Mar 14 13:48 config.txt
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle  1.6K Mar 14 13:46 out.PTRU_masurca10569999.x0902.log
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle    22 Mar 14 13:46 FLYE_dir.txt
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle  1.6K Mar 14 13:41 out.PTRU_masurca10569883.x0902.log
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle  1.6K Mar 14 13:38 out.PTRU_masurca10569783.x0902.log
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle  1.6K Mar 14 13:32 out.PTRU_masurca10569642.x0902.log
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle  1.6K Mar 14 13:30 out.PTRU_masurca10569579.x0902.log
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle  1.6K Mar 14 13:23 out.PTRU_masurca10569326.x0902.log
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle  2.0K Mar 14 12:22 out.PTRU_masurca10563335.x0902.log
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle  1.3G Mar 14 12:22 superReadSequences.named.fasta
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle   26K Mar 14 12:22 super1.err
drwxr-sr-x 2 jkiefe01 m2_jgu-symbeetle  4.0K Mar 14 12:22 work1_mr
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle  833M Mar 14 12:17 guillaumeKUnitigsAtLeast32bases_all.41.fasta
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle   10G Mar 14 12:15 longest_reads.25x.fa
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle    31 Mar 14 11:32 create_mega-reads.err
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle  1.6G Mar 14 11:31 k_unitigs.l20.fa
drwxr-sr-x 2 jkiefe01 m2_jgu-symbeetle  4.0K Mar 14 11:28 work1
lrwxrwxrwx 1 jkiefe01 m2_jgu-symbeetle    41 Mar 14 11:11 guillaumeKUnitigsAtLeast32bases_all.jump.fasta -> guillaumeKUnitigsAtLeast32bases_all.fasta
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle 1005M Mar 14 11:11 guillaumeKUnitigsAtLeast32bases_all.fasta
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle    10 Mar 14 11:05 ESTIMATED_GENOME_SIZE.txt
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle  6.0G Mar 14 11:05 k_u_hash_0
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle  7.9G Mar 14 11:03 pe.cor.fa
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle   25M Mar 14 11:03 pe.cor.tmp.log
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle   433 Mar 14 11:03 quorum.err
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle  4.4G Mar 14 10:59 quorum_mer_db.jf
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle   15G Mar 14 10:57 pe.renamed.fastq
-rw-r--r-- 1 jkiefe01 m2_jgu-symbeetle   159 Mar 14 10:53 out.PTRU_masurca10563284.x0901.log
-rwxr--r-- 1 jkiefe01 m2_jgu-symbeetle   267 Mar 14 10:53 masurca.sh
osowiecki commented 2 years ago

I believe you should submit it to slurm, wait for it to finish and then get back to assemble.sh when all jobs finish.