Closed or-shachar closed 1 year ago
Having had a dig through the code, the I would be tempted by implementing this using a Go template. In pkg.provider.KedaProvider.GetExternalMetric
to a new CalculateDerivedMetric
function that took the above scaledObject inputs would derive the right context object based on triggers and the formula field would be a template, which would be rendered using that context. Probably using https://masterminds.github.io/sprig/ so KEDA doesn't need to re-invent the wheel, you would end up with something like:
scaledObject.spec:
...
derivedMetric: # better name?
formula: "ceil(mulf(subf(1 .cpu_util) .q_length))"
targetMetricValue: "5"
minMetricValue: "0"
triggers:
- type: aws-cloudwatch
name: q_length
metadata:
namespace: AWS/SQS
... # SQS length
- type: aws-cloudwatch
name: cpu_util
metadata:
namespace: AWS/Neptune
... # neptune writer CPU
The implementation would be rendering that template and converting the resulting string representation into a number to send to the HPA.
Prior discussion: https://github.com/kedacore/keda/issues/373#issuecomment-631487703
2 use case ideas:
desired count = (target each replica handling N requests per sec) + (additional replicas if HTTP 500 errors show in logs)
^-- can't really do something like that with the current or based logic. It could be helpful if someone got their ideal requests per sec wrong.
It's my understanding that currently if you have multiple triggers they OR and take the largest desired count of the multiple triggers, some people mentioned it'd be useful to have AND logic. This formula might be a good opportunity to implement AND logic to cover those scenarios as well.
Proposal
I think that a "Composed Scaler" that allows to put in a simple formula that references multiple other metrics - can be super useful.
This would make Keda a powerful tool not only for collecting external metrics but also for making smart decisions based on multiple conditions.
(discussed original here)
Use-Case
Here's a real-world problem: I have a backend process that consumes "write" tasks from a queue and persists them to DB. My DB has a single writer node that is CPU-bound. A task may implicit varying load on the CPU (depending on the content of the write).
I have 2 relevant metrics to determine the replica count:
Composed metric formula:
ceil((1-cpu) * q)
can promise that replicas would grow if the queue is high but avoid growing if CPU is loaded. Right now the only option for me to do it is to create my own service that exposes that metric.Anything else?
In the original discussion, we started talking about how the ScaledObject spec would look like in the custom logic case.
First I suggested this:
then @zroubalik suggested this (better IMO):
Finally, it's valuable to read @tomkerkhove 's comment about it.