search

refresh-bio / KMC

Fast and frugal disk based k-mer counter
277 stars 72 forks source link

Error: cannot open file: /path/xx.fq.gz #197

Open xibeixingchen opened 2 years ago

xibeixingchen commented 2 years ago

Hi: friend I used code kmc -t1 -m1024 -k31 -ci2 @input_files.txt output_kmc_canon ./ ,but ,however, an error is reported. My input file has 374 lines in total, and the error location appears after 110. I deleted the path of the error file, but the error still occurs. I have checked the integrity and path of the file are correct. May I ask what caused this and how to solve this error? image image My English is not good, I do not know whether to describe my problem clearly, please forgive me, looking forward to your reply. Thank you! Sincerely

marekkokot commented 2 years ago

Hi,

This is strange. Could you send me your input files? Also, your configuration of max memory and number of threads is kinda strange (but of course should work), could you try with default t and m, just: kmc -k31 @input_files.txt output_kmc_canon .?

Best, Marek

xibeixingchen commented 2 years ago

Hi Marek,

I try again your suggestion, but the same error is still reported. However my data size is too large(~1 TB), so I could use for circulation for every sample, sing sample is successful . If not, I will consider part of the data to send to you (how do I send the data to you?).

Best, Zhicheng

marekkokot commented 2 years ago

Hi,

I see, a large dataset indeed. If you could reproduce this behavior on a smaller subset that you could share, it would be perfect. I'm afraid it will be really hard to help without reproducing this. Are you running this on linux? One thing that may cause this is the limitation of the number of input files. On Linux/MAC OS you can increase it with

ulimit -n 10000

or some other number instead of 10000. It may be especially important on MAC OS. Let me know if it helped.

Best, Marek