performance: more aggressive time scaling in `sample_qc` - Githubissues

NCI-CGR / GwasQcPipeline

The CGR GWAS QC processing workflow.

https://nci-cgr.github.io/GwasQcPipeline/

MIT License

0 stars 2 forks source link

performance: more aggressive time scaling in `sample_qc` #297

Closed jaamarks closed 3 months ago

jaamarks commented 3 months ago

We implement two key improvements for resource allocation on rules for sample_qc sub_workflow.

Updated the mem_mb logic to scale memory allocation based on the input file size (input.size_mb).
Implemented a more aggressive scaling mechanism for wall-time allocation. This is so that when processing large samples (>90K), the workflow will more quickly complete successfully.

Fixes #282