The default values (which multiply the requested RAM and CPU usage by retry strategy) are still low for common use cases (such as enrichment in ctDNA samples, which are fairly high depth), and at least in our cluster they consistently fail because the process is killed as it requests more memory than allocated.
For example:
FastQC: try 1 -> requests 4G -> fails
FastQC: try 2 -> requests 8G -> fails
Default maxRetries is 2 -> pipeline fails
Setting a maxRetries in our custom configuration (which we use) > 2 does not seem to have any effect. The only solution was to manually edit the configuration and bump the memory requirements.
Description of the bug
The default values (which multiply the requested RAM and CPU usage by retry strategy) are still low for common use cases (such as enrichment in ctDNA samples, which are fairly high depth), and at least in our cluster they consistently fail because the process is killed as it requests more memory than allocated.
For example:
Setting a maxRetries in our custom configuration (which we use) > 2 does not seem to have any effect. The only solution was to manually edit the configuration and bump the memory requirements.
cc @LMannarino @rikizad
Command used and terminal output
No response
Relevant files
No response
System information
Nextflow version: 24.04.2 Hardware: HPC Executor: SLURM Container engine: Apptainer OS: Debian Buster fastquorum version: latest dev