refresh-bio / KMC

Fast and frugal disk based k-mer counter
266 stars 73 forks source link

Odd error writing temporary file bins - Error: Cannot open temporary file bins/kmc_00253.bin #140

Closed jandreslopez closed 4 years ago

jandreslopez commented 4 years ago

KMC keeps getting stuck while writing the bin files. The bins/ directory is already created and the program starts running and creating temp bin files but at file kmc_00253.bin it reports an error and quits. When I look at the bins directory files 00001 through 00252 were created.

Below is verbose output from one of these failed attempts. I have tried it with a single gzipped file as input as well as with a list with 3 gzipped files. Same behavior in both cases. Changing memory settings has not made a difference.

I am running the OSX binaries currently posted and linked on the KMC web site on a OSX 10.14.6 system with 16GB of RAM. The input files are ~20Gbytes gzipped.

./kmc -k25 -m12 -sm -v 18120XD_bairdi_R1_cleaned.fastq.gz res bins/


** Used parameters: ** No. of input files : 1 Output file name : res No. of working directories : 1 Input format : FASTQ

k-mer length : 25 Max. k-mer length : 256 Signature length : 9 Min. count threshold : 2 Max. count threshold : 1000000000 Max. counter value : 255 Type of counters : direct Both strands : true RAM olny mode : false

Stage 1 configuration:

No. of bins : 512 Bin part size : 65536 Input buffer size : 16777216

No. of readers : 1 No. of splitters : 3

Max. mem. size : 12000MB Max. mem. per storer : 6180MB Max. mem. for single package : 24MB

Max. mem. for PMM (bin parts): 9508MB Max. mem. for PMM (FASTQ) : 1684MB Max. mem. for PMM (reads) : 0MB Max. mem. for PMM (b. reader): 805MB

Error: Cannot open temporary file bins/kmc_00253.bin

marekkokot commented 4 years ago

Hi, Thanks for using kmc. Try to run following command first: ulimit -n 2048

It is mentioned in the README.md in section Additional infromation for MAC OS installation.

jandreslopez commented 4 years ago

Thank you! I missed that bit in the readme. Works great now.

marekkokot commented 4 years ago

Great! In case of any further trouble don't hesitate to ask.