Closed koushik20 closed 1 year ago
@koushik20 I'm having this same issue - were you able to fix it?
I gave a separate custom nextflow config file and the pipeline was completed without any errors.
process {
withLabel:process_high {
memory = 64.GB
cpus = 52
time = 36.h
}
}
process {
withLabel:process_medium {
memory = 64.GB
cpus = 52
time = 36.h
}
}
process {
withLabel:process_low {
memory = 64.GB
cpus = 52
time = 36.h
}
}
process {
withName:'NFCORE_HIC:HIC:HICPRO:HICPRO_MAPPING:BOWTIE2_ALIGN' {
memory = 64.GB
cpus = 52
time = 36.h
}
}
process {
withName:'NFCORE_HIC:HIC:HICPRO:HICPRO_MAPPING:BOWTIE2_ALIGN_TRIMMED' {
memory = 64.GB
cpus = 52
time = 36.h
}
}
memory = { check_max( 64.GB * task.attempt, 'memory' ) }
// Function to ensure that resource requirements don't go beyond
// a maximum limit
def check_max(obj, type) {
if (type == 'memory') {
try {
if (obj.compareTo(params.max_memory as nextflow.util.MemoryUnit) == 1)
return params.max_memory as nextflow.util.MemoryUnit
else
return obj
} catch (all) {
println " ### ERROR ### Max memory '${params.max_memory}' is not valid! Using default value: $obj"
return obj
}
} else if (type == 'time') {
try {
if (obj.compareTo(params.max_time as nextflow.util.Duration) == 1)
return params.max_time as nextflow.util.Duration
else
return obj
} catch (all) {
println " ### ERROR ### Max time '${params.max_time}' is not valid! Using default value: $obj"
return obj
}
} else if (type == 'cpus') {
try {
return Math.min( obj, params.max_cpus as int )
} catch (all) {
println " ### ERROR ### Max cpus '${params.max_cpus}' is not valid! Using default value: $obj"
return obj
}
}
}
Thank you so much! This worked nicely, though for some samples the bowtie alignment step is taking over 48 hours... seems too long.
Description of the bug
Hi,
Thanks for the detailed documentation! I am running nfcore/hic version 2.0.0 with GRCh38 reference genome but always getting
Process exceeded running time limit (16h)
Below is the terminal output
The pipeline always stops at this particular bowtie2 mapping step. I gave a separate nextflow.config file and assigned greater memory to this specific step.
So My Questions are Why does the pipeline aborts at 16h timestamp even though I gave 240h max time? When I ran some samples earlier with GRCh37 the pipeline was completed successfully so I there an issue with using GRCh38? I tried to run with different
--max_cpus, --max_memory, --max_time
configurations but the pipeline always aborts at this particular step (command executed step) see aboveThank you!
Command used and terminal output
Relevant files
nextflow.log
System information
Nextflow version - 22.10.7 Hardware - Desktop Executor - local Container engine: Docker OS Ubuntu - 20.04.5 Linux Version - nf-core/hic 2.0.0