issues
search
umccr
/
nextflow-stack
AWS stack to run Nextflow pipelines on Batch using shared resources
0
stars
0
forks
source link
Increase initial EBS volume size
#65
Closed
scwatts
closed
1 year ago
scwatts
commented
1 year ago
Background
in the absence of FusionFS, I'm currently using EBS autoscale to increase available scratch space at runtime
the default scale-up threshold of 50% and initial 20 GB EBS volume doesn't provide a sufficient buffer in all cases
e.g. a resource-ramped SvPrep Depth Annotator tasks run out of EBS space before it can be scaled
possibly related to higher instance memory size; larger memory buffer being dumped to disk too quickly
observed in prod for SBJ04367, reproduced in dev
Changes
increase initial volume to default 200 GB to give an effective buffer size of 100 GB
Other notes
an increase to 30 GB resolved the issue described above for SBJ04367 resource-ramped SvPrep Depth Annotator task
however, the error retriggered in subsequent runs at same point with 30 GB and 50 GB initial volume sizes
Background
Changes
Other notes