enriquepola1996 commented 4 months ago

Hello, I am trying to use Flye to assemble a plant genome and I am having the following problem, could you please help me?

My script is the following:

!/bin/bash

PBS -N flye_completo

PBS -l walltime=700:00:00

PBS -l nodes=1:ppn=20,vmem=450gb

PBS -o flye450_output.log

PBS -e flye450_error.log

PBS -q ensam

PBS -V

Carga el módulo de flye

module load Flye/2.9.3

cambia al directorio de trabajo

cd $PBS_O_WORKDIR

flye --pacbio-raw ../../all.pacbio_original.fasta.gz --genome-size 4g --out-dir ./flye_output_completo --threads 20

My flye.log:

[2024-02-03 22:10:52] root: INFO: Starting Flye 2.9.3-b1797 [2024-02-03 22:10:52] root: DEBUG: Cmd: /data/software/Flye/bin/flye --pacbio-raw ../../all.pacbio_original.fasta.gz --genome-size 4g --out-dir ./flye_output_completo --threads 20 [2024-02-03 22:10:52] root: DEBUG: Python version: 3.7.3 (default, Mar 27 2019, 22:11:17) [GCC 7.3.0] [2024-02-03 22:10:52] root: INFO: >>>STAGE: configure [2024-02-03 22:10:52] root: INFO: Configuring run [2024-02-03 23:36:42] root: INFO: Total read length: 301995732522 [2024-02-03 23:36:42] root: INFO: Input genome size: 4000000000 [2024-02-03 23:36:42] root: INFO: Estimated coverage: 75 [2024-02-03 23:36:42] root: INFO: Reads N50/N90: 18904 / 3802 [2024-02-03 23:36:42] root: INFO: Minimum overlap set to 4000 [2024-02-03 23:36:43] root: INFO: >>>STAGE: assembly [2024-02-03 23:36:43] root: INFO: Assembling disjointigs [2024-02-03 23:36:43] root: DEBUG: -----Begin assembly log------ [2024-02-03 23:36:43] root: DEBUG: Running: flye-modules assemble --reads /LUSTRE/usuario/jmvilla/pruebas/all.pacbio_original.fasta.gz --out-asm /LUSTRE/usuario/jmvilla/pruebas/flye_pruebas_1G/flye_output_completo/flye_output_completo/00-assembly/draft_assembly.fasta --config /LUSTRE/storage/data/software/Flye/flye/config/bin_cfg/asm_raw_reads.cfg --log /LUSTRE/usuario/jmvilla/pruebas/flye_pruebas_1G/flye_output_completo/flye_output_completo/flye.log --threads 20 --genome-size 4000000000 --min-ovlp 4000 [2024-02-03 23:36:43] DEBUG: Build date: Jan 31 2024 10:32:47 [2024-02-03 23:36:43] DEBUG: Total RAM: 757 Gb [2024-02-03 23:36:43] DEBUG: Available RAM: 740 Gb [2024-02-03 23:36:43] DEBUG: Total CPUs: 20 [2024-02-03 23:36:43] DEBUG: Loading /LUSTRE/storage/data/software/Flye/flye/config/bin_cfg/asm_raw_reads.cfg [2024-02-03 23:36:43] DEBUG: Loading /LUSTRE/storage/data/software/Flye/flye/config/bin_cfg/asm_defaults.cfg [2024-02-03 23:36:43] DEBUG: big_genome_threshold=29000000 [2024-02-03 23:36:43] DEBUG: meta_read_filter_kmer_freq=100 [2024-02-03 23:36:43] DEBUG: chain_large_gap_penalty=2 [2024-02-03 23:36:43] DEBUG: chain_small_gap_penalty=0.5 [2024-02-03 23:36:43] DEBUG: chain_gap_jump_threshold=100 [2024-02-03 23:36:43] DEBUG: max_jump_gap=500 [2024-02-03 23:36:43] DEBUG: max_coverage_drop_rate=5 [2024-02-03 23:36:43] DEBUG: max_extensions_drop_rate=5 [2024-02-03 23:36:43] DEBUG: chimera_window=100 [2024-02-03 23:36:43] DEBUG: chimera_overhang=1000 [2024-02-03 23:36:43] DEBUG: min_reads_in_disjointig=4 [2024-02-03 23:36:43] DEBUG: max_inner_reads=10 [2024-02-03 23:36:43] DEBUG: max_inner_fraction=0.25 [2024-02-03 23:36:43] DEBUG: aggressive_dup_filter=1 [2024-02-03 23:36:43] DEBUG: max_separation=500 [2024-02-03 23:36:43] DEBUG: unique_edge_length=50000 [2024-02-03 23:36:43] DEBUG: min_repeat_res_support=0.51 [2024-02-03 23:36:43] DEBUG: out_paths_ratio=5 [2024-02-03 23:36:43] DEBUG: graph_cov_drop_rate=5 [2024-02-03 23:36:43] DEBUG: coverage_estimate_window=100 [2024-02-03 23:36:43] DEBUG: max_bubble_length=50000 [2024-02-03 23:36:43] DEBUG: loop_coverage_rate=1.5 [2024-02-03 23:36:43] DEBUG: repeat_edge_cov_mult=1.75 [2024-02-03 23:36:43] DEBUG: weak_detach_rate=5 [2024-02-03 23:36:43] DEBUG: tip_coverage_rate=2 [2024-02-03 23:36:43] DEBUG: tip_length_rate=2 [2024-02-03 23:36:43] DEBUG: output_gfa_before_rr=1 [2024-02-03 23:36:43] DEBUG: remove_alt_edges=0 [2024-02-03 23:36:43] DEBUG: low_cutoff_warning=1 [2024-02-03 23:36:43] DEBUG: kmer_size=17 [2024-02-03 23:36:43] DEBUG: use_minimizers=0 [2024-02-03 23:36:43] DEBUG: reads_base_alignment=0 [2024-02-03 23:36:43] DEBUG: meta_read_top_kmer_rate=0.40 [2024-02-03 23:36:43] DEBUG: maximum_jump=1500 [2024-02-03 23:36:43] DEBUG: maximum_overhang=1500 [2024-02-03 23:36:43] DEBUG: repeat_kmer_rate=100 [2024-02-03 23:36:43] DEBUG: assemble_ovlp_divergence=0.10 [2024-02-03 23:36:43] DEBUG: assemble_divergence_relative=1 [2024-02-03 23:36:43] DEBUG: repeat_graph_ovlp_divergence=0.08 [2024-02-03 23:36:43] DEBUG: read_align_ovlp_divergence=0.25 [2024-02-03 23:36:43] DEBUG: hpc_scoring_on=0 [2024-02-03 23:36:43] DEBUG: add_unassembled_reads=0 [2024-02-03 23:36:43] DEBUG: extend_contigs_with_repeats=0 [2024-02-03 23:36:43] DEBUG: min_read_cov_cutoff=3 [2024-02-03 23:36:43] DEBUG: short_tip_length=20000 [2024-02-03 23:36:43] DEBUG: long_tip_length=100000 [2024-02-03 23:36:43] DEBUG: Running with k-mer size: 17 [2024-02-03 23:36:43] DEBUG: Running with minimum overlap 4000 [2024-02-03 23:36:43] DEBUG: Metagenome mode: N [2024-02-03 23:36:43] DEBUG: Short mode: N [2024-02-03 23:36:43] INFO: Reading sequences [2024-02-04 03:59:04] DEBUG: Building positional index [2024-02-04 03:59:19] DEBUG: Total sequence: 270356027903 bp [2024-02-04 03:59:22] INFO: Counting k-mers: [2024-02-04 05:23:51] DEBUG: Updating k-mer histogram [2024-02-04 05:51:12] DEBUG: Hash size: 3116736766 [2024-02-04 05:51:12] DEBUG: Total k-mers 8505082354 [2024-02-04 05:51:19] INFO: Filling index table (1/2) [2024-02-04 08:43:45] DEBUG: Mean k-mer frequency: 116.698 [2024-02-04 08:43:45] DEBUG: Repetitive k-mer frequency: 11669 [2024-02-04 08:43:45] DEBUG: Filtered 16194367592 repetitive k-mers (0.151781) [2024-02-04 08:45:33] ERROR: Caught unhandled exception: std::bad_alloc [2024-02-04 08:45:33] ERROR: flye-modules(_Z16exceptionHandlerv+0xd0) [0x45fd80] [2024-02-04 08:45:33] ERROR: /opt/pb/gcc-4.8.4/lib64/libstdc++.so.6(+0x5dbf6) [0x2aaaaab26bf6] [2024-02-04 08:45:33] ERROR: /opt/pb/gcc-4.8.4/lib64/libstdc++.so.6(+0x5dc23) [0x2aaaaab26c23] [2024-02-04 08:45:33] ERROR: /opt/pb/gcc-4.8.4/lib64/libstdc++.so.6(+0x5de42) [0x2aaaaab26e42] [2024-02-04 08:45:33] ERROR: /opt/pb/gcc-4.8.4/lib64/libstdc++.so.6(_Znwm+0x7d) [0x2aaaaab2732d] [2024-02-04 08:45:33] ERROR: /opt/pb/gcc-4.8.4/lib64/libstdc++.so.6(_Znam+0x9) [0x2aaaaab273c9] [2024-02-04 08:45:33] ERROR: flye-modules(_ZN11VertexIndex19allocateIndexMemoryEv+0x153) [0x48e563] [2024-02-04 08:45:33] ERROR: flye-modules(_ZN11VertexIndex24buildIndexUnevenCoverageEifi+0x218) [0x491d98] [2024-02-04 08:45:33] ERROR: flye-modules(_Z13assemble_mainiPPc+0xc60) [0x45d6f0] [2024-02-04 08:45:33] ERROR: flye-modules(main+0x87) [0x530907] [2024-02-04 08:45:37] root: ERROR: Command '['flye-modules', 'assemble', '--reads', '/LUSTRE/usuario/jmvilla/pruebas/all.pacbio_original.fasta.gz', '--out-asm', '/LUSTRE/usuario/jmvilla/pruebas/flye_pruebas_1G/flye_output_completo/flye_output_completo/00-assembly/draft_assembly.fasta', '--config', '/LUSTRE/storage/data/software/Flye/flye/config/bin_cfg/asm_raw_reads.cfg', '--log', '/LUSTRE/usuario/jmvilla/pruebas/flye_pruebas_1G/flye_output_completo/flye_output_completo/flye.log', '--threads', '20', '--genome-size', '4000000000', '--min-ovlp', '4000']' died with <Signals.SIGABRT: 6>. [2024-02-04 08:45:37] root: ERROR: Pipeline aborted

mikolmogorov commented 4 months ago

You are likely running out of memory.