search

gbif / pipelines

Pipelines for data processing (GBIF and LivingAtlases)
Apache License 2.0
40 stars 28 forks source link

K8s: Use sum of memoryOverhead and config.resources.memory.limit for yunikorn.apache.org/task-groups #1043

Closed muttcg closed 3 months ago

muttcg commented 3 months ago

The TaskGroup resources must account for the memory overhead for Spark drivers and executors.