search

honeycombio / helm-charts

Helm repository and charts for Honeycomb
Apache License 2.0
30 stars 39 forks source link

Consider auto-configuring Refinery memory consumption tunables by default #100

Closed bixu closed 1 year ago

bixu commented 2 years ago

Is your feature request related to a problem? Please describe. I've seen the tuning for MaxAlloc and CacheCapacity cause some confusion for operators of Refinery in Kubernetes.

Since sane default configs are documented here, why not leverage Helm to configure these?

Describe the solution you'd like Since Refinery pods should get Guaranteed QoS in K8s anyway (requests and limits on RAM should be matching values), we could use those RAM allocation values to calculate the default values for both MaxAlloc and CacheCapacity. I think we'd want these to be overridable, however, via something like a custom values.yaml file.

Describe alternatives you've considered I'm considering opening a PR on this myself, but need to negotiate for the time to do so from $job.

seh commented 2 years ago

You can push the memory limit value down into the Refinery container via the Kubernetes Downward API.

For now, we generate the Refinery configuration from the same program that generates a kustomize patch to set the memory request and limit, so we perform the recommended 80% calculation when writing the configuration file, but it would be nice to have Refinery derive these values from a memory limit supplied via environment variable.

MikeGoldsmith commented 2 years ago

Thanks for the suggestion. We'll add this to our backlog for prioritisation, or if you feel inclined we're always open to PRs.

TylerHelmuth commented 1 year ago

With Refinery 2.0's introduction of AvailableMemory and MaxMemoryPercentage the chart is now setting some memory defaults using the resource limits:

Collection:
    # AvailableMemory is the amount of system memory available to the Refinery process.
    AvailableMemory: '{{ .Values.resources.limits.memory }}'
    MaxMemoryPercentage: 75