VPA - Support for custom metrics

[haggrip]

Which component are you using?: Vertical Pod Autoscaler

Is your feature request designed to solve a problem? If so describe the problem this feature should solve.:

We use celery and vertical autoscaling in our app. Celery has a main worker and each for each task a new process is spawned. Sometimes, the tasks get OOMKilled, but because the main worker process has not died, it is not counted as an OOMKill event in kubernetes, and the vertical autoscaler does not update the task deployment. Describe the solution you'd like.:

Support custom metrics in a similar manner to HPA. Ideally, we could send OOM events from our tasks and have the vertical autoscaler consume them to update memory requests

Describe any alternative solutions you've considered.:

Modifying our tasks to update the vertical autoscaling CRDs when memory errors are detected. This seems like an anti-pattern to any kubernetes native application, but I welcome other opinions on this matter.

[adrianmoisey]

/area vertical-pod-autoscaler