olgabot / nf-kmer-similarity

MIT License
1 stars 0 forks source link

Use multiple of total file size for memory for sourmash_compare_sketches #1

Open olgabot opened 5 years ago

olgabot commented 5 years ago

e.g. as here https://github.com/nextflow-io/nextflow/issues/922 they use the file size of a single file to specify the memory.

Something like this:

    memory { sourmash_sketches.size().sum() < 100.MB ? 8.GB * task.attempt :
        sourmash_sketches.size() * 100.MB * task.attempt}

Use 100 MB times the summed file size, times the task attempt, otherwise use 8GB