Open szachn-u opened 1 year ago
Hello, Sorry for my late reply.
Unfortunately, you are encountering a known kmtricks problem. Memory grows with the number of files, regardless of their sizes, which is not expected. I have started investigating and this will be corrected in future releases.
It seems you use kmtricks only to count k-mers without building the matrix (because of --until count
), right? In this case, you can simply do it in several runs by splitting your fof into smaller ones. If you need a matrix, you can use the same strategy and then combine all runs using kmtricks combine
. See https://github.com/tlemane/kmtricks/wiki/combine.
Hope this helps Teo
Hi Teo thanks for the answer, i'll try it. Ugo
hello
I launched this kmtricks command for a file of files with ~ 37000 fastq files
kmtricks pipeline --file "$list_fq" --run-dir "$outDir/out" --kmer-size 31 --hard-min 1 --mode kmer:count:bin --until count --cpr -t 20
i runs nicely but at some point uses all ram available (i have 128Gb) and the script stops
./analysis_10x_MDAMB468_kmtricks.sh : ligne 30 : 21185 Processus arrêté "$kmtricks" pipeline --file "$list_fq" --run-dir "$outDir/out" --kmer-size 31 --hard-min 1 --mode kmer:count:bin --until count --cpr -t 20
Is there a way to limit the RAM usage ?
thanks a lot