Program stops with "Killed" message

[iaposto]

I am running the command unitig-caller --call --refs refs.txt --pyseer --write-graph --threads 12 on a 30GB RAM machine. The .bfg_colors and .gfa files are generated successfully, but after the message Calling unitigs within population, I get a Killed message and the program exits. The refs.txt file contains the paths for 2020 E.coli assemblies. Any idea what might be the issue? Thanks!

[johnlees]

Unfortunately sounds like you ran out of memory. Perhaps take a look with ulimit or /usr/bin/time -v

[iaposto]

The output of /usr/bin/time -v is: Command terminated by signal 9 Command being timed: "unitig-caller --call --graph refs.gfa --colours refs.bfg_colors --threads 12 --pyseer" User time (seconds): 1251.49 System time (seconds): 37.29 Percent of CPU this job got: 92% Elapsed (wall clock) time (h:mm:ss or m:ss): 23:09.66 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 28224880 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 1421117 Minor (reclaiming a frame) page faults: 17509882 Voluntary context switches: 1510737 Involuntary context switches: 985 Swaps: 0 File system inputs: 79176088 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 If this is a memory issue, do you have any idea how much memory would be enough? Alternatively, is there a workaround, like splitting the dataset and joining the outputs?

Thanks for your help!

[johnlees]

Max resident set size looks like it's reaching about 30Gb so yeah I'd think you're running out of memory.

Things you could try:

• Subset to find a smaller sample size that does work, guess scaling from there and find a higher memory machine if feasible
• If not feasible, define unitigs in a subset of the population then use call/simple mode to get counts (though this may miss some variation).
• I can't remember how this is coded and if this would make a difference, but you could also try with just one thread (keeping the graph you've already generated)
• try the older unitig-counter package (though we don't support this any more)

[iaposto]

unitig-counter worked, thanks for the suggestion! Any important differences I should consider compared to unitig-caller? Unitigs will be used as input to pySEER.

EDIT-1: I see that I should also clean up the output with the respective script mentioned in unitig-counter README.md EDIT-2: I managed to successfully run unitig-caller by increasing my swap memory to 100GB (it kept failing up to 50GB).