runCA3 Cannot allocate memory

[ColyFu]

hi @alekseyzimin,

Thanks for developing this amazing tool! I encountered an error report in runCA3 step by using slurm recently. Error Message:

----------------------------------------START Tue Jan  7 11:24:05 2020
~/.conda/envs/masurca/CA8/Linux-amd64/bin/terminator -g ~/12.pangenome/test3/CA/genome.gkpStore -t ~/12.pangenome/test3/CA/genome.tigStore 16 -c ~/12.pangenome/test3/CA/7-CGW/genome 15 -o ~/12.pangenome/test3/CA/9-terminator/genome > ~/12.pangenome/test3/CA/9-terminator/genome.asm.err 2>&1
----------------------------------------END Tue Jan  7 11:24:05 2020 (0 seconds)
ERROR: Failed with signal HUP (1)
================================================================================

runCA failed.

----------------------------------------
Stack trace:

 at ~/.conda/envs/masurca/bin/../CA8/Linux-amd64/bin/runCA line 1613.
    main::caFailure("terminator failed", "~/12.pangenome/test3/CA/9-ter"...) called at ~/.conda/envs/masurca/bin/../CA8/Linux-amd64/bin/runCA line 5862
    main::terminate() called at ~/.conda/envs/masurca/bin/../CA8/Linux-amd64/bin/runCA line 6534

----------------------------------------
Last few lines of the relevant log file (~/12.pangenome/test3/CA/9-terminator/genome.asm.err):

LoadScaffoldGraphFromCheckpoint()--  Loading checkpoint '~/12.pangenome/test3/CA/7-CGW/genome.ckp.15' at Tue Jan  7 11:24:05 2020
LoadScaffoldGraphFromCheckpoint()--  Loading reads and dists.
LoadScaffoldGraphFromCheckpoint()--  Loading unitigs.
LoadScaffoldGraphFromCheckpoint()--  Loading contigs.
LoadScaffoldGraphFromCheckpoint()--  Loading scaffolds.
LoadScaffoldGraphFromCheckpoint()--  Loading chunk overlaps.
LoadScaffoldGraphFromCheckpoint()--  Rebuilding unitig edges.
ResizeEdgeList()-- sizing list for 'c' to 22108 (currently 0)
RebuildGraphEdges()-- Rebuilding 13592 edges in 11046 unitigs.
LoadScaffoldGraphFromCheckpoint()--  Rebuilding contig edges.
ResizeEdgeList()-- sizing list for 'C' to 23986 (currently 0)
RebuildGraphEdges()-- Rebuilding 13596 edges in 11985 contigs.
LoadScaffoldGraphFromCheckpoint()--  Rebuilding scaffold edges.
ResizeEdgeList()-- sizing list for 'S' to 6622 (currently 0)
RebuildGraphEdges()-- Rebuilding 318 edges in 3303 scaffolds.
* ScaffoldGraph Memory Report:
*           numElements AllocatedElements   AllocatedSize
VA        192 bytes; elements:          3 active;          3 allocated;    64 bytes; 'Dists'
VA    7764096 bytes; elements:      80876 active;      80876 allocated;    96 bytes; 'CIFrags'
VA    1855728 bytes; elements:      11046 active;      11046 allocated;   168 bytes; 'CIs    '
VA    1304832 bytes; elements:      13592 active;      13592 allocated;    96 bytes; 'CIEdges'
VA    2013480 bytes; elements:      11985 active;      11985 allocated;   168 bytes; 'Contigs    '
VA    1305216 bytes; elements:      13596 active;      13596 allocated;    96 bytes; 'ConEdges'
VA     554904 bytes; elements:       3303 active;       3303 allocated;   168 bytes; 'Scaffolds'
VA      30528 bytes; elements:        318 active;        318 allocated;    96 bytes; 'SEdges'
*   TotalMemorySize = 14828976
Writing assembly file
writeMDI()--
ERROR: Write Failure: Cannot allocate memory

----------------------------------------
Failure message:

terminator failed

This error was clear, but I submitted the job to a 3Tb memory node with 56 threads. The genome size is about 5Mb and ~30x depth. I thought maybe slurm limited the memory, so I added the --mem=500GB and --cpus-per-task=48, but didn't work.

I ran the same test in a local machine with 256 GB memory, and worked fine.

Jun

[ColyFu]

By the way, I ran masurca in the local mode(USE_GRID=0), and submitted masurca config.ini & ./assemble.sh to the 3TB large memory node using slurm.

Jun

[ColyFu]

The problem has been resolved by a compilation installation

[lly1214]

can you tell me some details about your solution ? thanks a lot

[ColyFu]

hi, @lly1214

I resolved this problem by recompiling the software in my own computing environment.

[lly1214]

Thank you very much for your reply! Actually, my problem is not as same as yours.Have you met the following problem and do you have solution? I show some details in the attachment.

Thank you a lot and best wishes!

Luo Lingyun

-----原始邮件----- 发件人:"Fu Jun" @.> 发送时间:2022-04-18 11:25:27 (星期一) 收件人: alekseyzimin/masurca @.> 抄送: lly1214 @.>, Mention @.> 主题: Re: [alekseyzimin/masurca] runCA3 Cannot allocate memory (#147)

hi, @lly1214

I resolved this problem by recompiling the software in my own computing environment.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>