search

COMBINE-lab / cuttlefish

Building the compacted de Bruijn graph efficiently from references or reads.
BSD 3-Clause "New" or "Revised" License
81 stars 9 forks source link

Building fails for k = 13 or larger #12

Closed jnalanko closed 2 years ago

jnalanko commented 2 years ago

Hello,

I'm running the following command:

bin/cuttlefish build -s ~/data/data.fna -k 13 -o cdbg -w temp/ --ref

This fails when k is 13 or larger, but works for k = 11 or smaller. When k is 13 or larger, the code runs for some seconds, printing a bunch of asterisks, and then crashes with this error message: "Error: Cannot open temporary file ./kmc_01021.bin"

The directory temp/ does exist on disk and the problem also happens if I don't give the -w parameter at all.

Any idea what could be the problem?

Edit: I'm running on the latest commit in the develop branch, which is ec788462346.

rob-p commented 2 years ago

Hi @jnalanko,

This sounds like it's exiting during the KMC3 phase. Perhaps this has to do with KMC's creation of intermediate files and the file limit of the user running the process. Can you try:

ulimit -n 2048

to see if you can raise the file handle limit for your user? We should find a good place to mention this in the documentation.

--Rob

jnalanko commented 2 years ago

Thanks for the quick response. You are right: setting the ulimit 2048 to worked. Thanks!

rob-p commented 2 years ago

Awesome; @jamshed — can we please add a note about this to the documentation, as it seems like it will be a fairly commonly encountered problem? Thanks for the report, @jnalanko!