I still got it when I increased -m or any mem parameters.
MEGAHIT v1.2.9
2021-08-06 15:25:27 - Using megahit_core with POPCNT and BMI2 support
2021-08-06 15:25:27 - passing check point 0
2021-08-06 15:25:27 - passing check point 1
Traceback (most recent call last):
File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx512/Core/megahit/1.2.9/bin/megahit", line 1038, in
main()
File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx512/Core/megahit/1.2.9/bin/megahit", line 990, in main
if set_max_k_by_lib():
File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx512/Core/megahit/1.2.9/bin/megahit", line 761, in set_max_k_by_lib
max_read_len = get_max_read_len()
File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx512/Core/megahit/1.2.9/bin/megahit", line 750, in get_max_read_len
with open(opt.read_lib_path + '.lib_info') as read_info:
FileNotFoundError: [Errno 2] No such file or directory: '/home/user/scratch/Contigs/GRDI-ECO_AM-CG-ChampSt2-20161006-WatPhotz-B_MEG.megahit/tmp/reads.lib.lib_info'
Cannot find /home/user/scratch/Contigs/GRDI-ECO_AM-CG-ChampSt2-20161006-WatPhotz-C_MEG.megahit/options.json, switching to normal mode
2021-08-06 15:25:28 - MEGAHIT v1.2.9
2021-08-06 15:25:28 - Using megahit_core with POPCNT and BMI2 support
2021-08-06 15:25:28 - Convert reads to binary library
2021-08-06 15:25:50 - b'INFO sequence/io/sequence_lib.cpp : 75 - Lib 0 (/lustre03/nearline/6033517/Champlain_metagenomic_2016-2020/clean_reads/2016_2017/GRDI-ECO_AM-CG-ChampSt2-20161006-WatPhotz-C_MEG.QC_R1.fastq.gz,/lustre03/nearline/6033517/Champlain_metagenomic_2016-2020/clean_reads/2016_2017/GRDI-ECO_AM-CG-ChampSt2-20161006-WatPhotz-C_MEG.QC_R2.fastq.gz): pe, 23894666 reads, 126 max length'
2021-08-06 15:25:50 - b'INFO utils/utils.h : 152 - Real: 22.1158\tuser: 16.0488\tsys: 3.8649\tmaxrss: 271928'
2021-08-06 15:25:50 - k-max reset to: 141
2021-08-06 15:25:50 - Start assembly. Number of CPU threads 40
2021-08-06 15:25:50 - k list: 21,29,39,59,79,99,119,141
2021-08-06 15:25:50 - Memory used: 729152129433
2021-08-06 15:25:50 - Extract solid (k+1)-mers for k = 21
2021-08-06 15:26:30 - Build graph for k = 21
2021-08-06 15:27:53 - Assemble contigs from SdBG for k = 21
2021-08-06 15:33:53 - Local assembly for k = 21
2021-08-06 15:34:47 - Extract iterative edges from k = 21 to 29
2021-08-06 15:35:48 - Build graph for k = 29
2021-08-06 15:36:16 - Assemble contigs from SdBG for k = 29
2021-08-06 15:40:14 - Local assembly for k = 29
2021-08-06 15:41:29 - Extract iterative edges from k = 29 to 39
2021-08-06 15:41:51 - Build graph for k = 39
2021-08-06 15:42:35 - Assemble contigs from SdBG for k = 39
2021-08-06 15:45:56 - Local assembly for k = 39
2021-08-06 15:47:27 - Extract iterative edges from k = 39 to 59
2021-08-06 15:47:39 - Build graph for k = 59
2021-08-06 15:48:23 - Assemble contigs from SdBG for k = 59
2021-08-06 15:50:10 - Local assembly for k = 59
2021-08-06 15:51:42 - Extract iterative edges from k = 59 to 79
2021-08-06 15:51:47 - Build graph for k = 79
2021-08-06 15:52:06 - Assemble contigs from SdBG for k = 79
2021-08-06 15:53:02 - Local assembly for k = 79
2021-08-06 15:54:17 - Extract iterative edges from k = 79 to 99
2021-08-06 15:54:21 - Build graph for k = 99
2021-08-06 15:54:25 - Assemble contigs from SdBG for k = 99
2021-08-06 15:55:02 - Local assembly for k = 99
2021-08-06 15:56:06 - Extract iterative edges from k = 99 to 119
2021-08-06 15:56:08 - Build graph for k = 119
2021-08-06 15:56:11 - Assemble contigs from SdBG for k = 119
2021-08-06 15:56:40 - Local assembly for k = 119
2021-08-06 15:57:37 - Extract iterative edges from k = 119 to 141
2021-08-06 15:57:38 - Build graph for k = 141
2021-08-06 15:57:41 - Assemble contigs from SdBG for k = 141
2021-08-06 15:58:04 - Merging to output final contigs
2021-08-06 15:58:05 - 19692 contigs, total 42851043 bp, min 1000 bp, max 210336 bp, avg 2176 bp, N50 2158 bp
2021-08-06 15:58:05 - ALL DONE. Time elapsed: 1957.693265 seconds
Hi,
I hope this is not redundant with another issue.
It seems that there might be an issue with passing check point 1 and I am not sure about the reason. No idea if that could impact the final output.
Here is the command used: module load megahit/1.2.9
megahit -1 $sample.QC_R1.fastq.gz -2 $sample.QC_R2.fastq.gz --min-contig-len 1000 -m 0.9 -o /home/user/scratch/Contigs/$sample.megahit -t 40 --continue
I still got it when I increased -m or any mem parameters.
MEGAHIT v1.2.9 2021-08-06 15:25:27 - Using megahit_core with POPCNT and BMI2 support 2021-08-06 15:25:27 - passing check point 0 2021-08-06 15:25:27 - passing check point 1 Traceback (most recent call last): File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx512/Core/megahit/1.2.9/bin/megahit", line 1038, in
main()
File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx512/Core/megahit/1.2.9/bin/megahit", line 990, in main
if set_max_k_by_lib():
File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx512/Core/megahit/1.2.9/bin/megahit", line 761, in set_max_k_by_lib
max_read_len = get_max_read_len()
File "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx512/Core/megahit/1.2.9/bin/megahit", line 750, in get_max_read_len
with open(opt.read_lib_path + '.lib_info') as read_info:
FileNotFoundError: [Errno 2] No such file or directory: '/home/user/scratch/Contigs/GRDI-ECO_AM-CG-ChampSt2-20161006-WatPhotz-B_MEG.megahit/tmp/reads.lib.lib_info'
Cannot find /home/user/scratch/Contigs/GRDI-ECO_AM-CG-ChampSt2-20161006-WatPhotz-C_MEG.megahit/options.json, switching to normal mode
2021-08-06 15:25:28 - MEGAHIT v1.2.9
2021-08-06 15:25:28 - Using megahit_core with POPCNT and BMI2 support
2021-08-06 15:25:28 - Convert reads to binary library
2021-08-06 15:25:50 - b'INFO sequence/io/sequence_lib.cpp : 75 - Lib 0 (/lustre03/nearline/6033517/Champlain_metagenomic_2016-2020/clean_reads/2016_2017/GRDI-ECO_AM-CG-ChampSt2-20161006-WatPhotz-C_MEG.QC_R1.fastq.gz,/lustre03/nearline/6033517/Champlain_metagenomic_2016-2020/clean_reads/2016_2017/GRDI-ECO_AM-CG-ChampSt2-20161006-WatPhotz-C_MEG.QC_R2.fastq.gz): pe, 23894666 reads, 126 max length'
2021-08-06 15:25:50 - b'INFO utils/utils.h : 152 - Real: 22.1158\tuser: 16.0488\tsys: 3.8649\tmaxrss: 271928'
2021-08-06 15:25:50 - k-max reset to: 141
2021-08-06 15:25:50 - Start assembly. Number of CPU threads 40
2021-08-06 15:25:50 - k list: 21,29,39,59,79,99,119,141
2021-08-06 15:25:50 - Memory used: 729152129433
2021-08-06 15:25:50 - Extract solid (k+1)-mers for k = 21
2021-08-06 15:26:30 - Build graph for k = 21
2021-08-06 15:27:53 - Assemble contigs from SdBG for k = 21
2021-08-06 15:33:53 - Local assembly for k = 21
2021-08-06 15:34:47 - Extract iterative edges from k = 21 to 29
2021-08-06 15:35:48 - Build graph for k = 29
2021-08-06 15:36:16 - Assemble contigs from SdBG for k = 29
2021-08-06 15:40:14 - Local assembly for k = 29
2021-08-06 15:41:29 - Extract iterative edges from k = 29 to 39
2021-08-06 15:41:51 - Build graph for k = 39
2021-08-06 15:42:35 - Assemble contigs from SdBG for k = 39
2021-08-06 15:45:56 - Local assembly for k = 39
2021-08-06 15:47:27 - Extract iterative edges from k = 39 to 59
2021-08-06 15:47:39 - Build graph for k = 59
2021-08-06 15:48:23 - Assemble contigs from SdBG for k = 59
2021-08-06 15:50:10 - Local assembly for k = 59
2021-08-06 15:51:42 - Extract iterative edges from k = 59 to 79
2021-08-06 15:51:47 - Build graph for k = 79
2021-08-06 15:52:06 - Assemble contigs from SdBG for k = 79
2021-08-06 15:53:02 - Local assembly for k = 79
2021-08-06 15:54:17 - Extract iterative edges from k = 79 to 99
2021-08-06 15:54:21 - Build graph for k = 99
2021-08-06 15:54:25 - Assemble contigs from SdBG for k = 99
2021-08-06 15:55:02 - Local assembly for k = 99
2021-08-06 15:56:06 - Extract iterative edges from k = 99 to 119
2021-08-06 15:56:08 - Build graph for k = 119
2021-08-06 15:56:11 - Assemble contigs from SdBG for k = 119
2021-08-06 15:56:40 - Local assembly for k = 119
2021-08-06 15:57:37 - Extract iterative edges from k = 119 to 141
2021-08-06 15:57:38 - Build graph for k = 141
2021-08-06 15:57:41 - Assemble contigs from SdBG for k = 141
2021-08-06 15:58:04 - Merging to output final contigs
2021-08-06 15:58:05 - 19692 contigs, total 42851043 bp, min 1000 bp, max 210336 bp, avg 2176 bp, N50 2158 bp
2021-08-06 15:58:05 - ALL DONE. Time elapsed: 1957.693265 seconds
Thanks for your help,
Nico