kubernetes-sigs / prometheus-adapter

An implementation of the custom.metrics.k8s.io API using Prometheus
Apache License 2.0
1.9k stars 551 forks source link

how to ignore parameters from hpa request ? #630

Open owanio1992 opened 8 months ago

owanio1992 commented 8 months ago

hello I want use other pod's metric as hpa rule

current I can use this command to get metric from prometheus adapter

kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta2/namespaces/devconn/pods/*/nbl_redis_llen_flx_coll_grpc1" | jq

but I see the prometheus adapter access log (request by hpa)

I1227 12:29:50.512845       1 httplog.go:132] "HTTP" verb="GET" URI="/apis/custom.metrics.k8s.io/v1beta2/namespaces/devconn/pods/%2A/nbl_redis_llen_flx_coll_grpc2?labelSelector=app.kubernetes.io%2Finstance%3Dflx-coll-grpc%2Capp.kubernetes.io%2Fname%3Dworker2" latency="18.291595ms" userAgent="kube-controller-manager/v1.28.4 (linux/amd64) kubernetes/b4ddd68/system:serviceaccount:kube-system:horizontal-pod-autoscaler" audit-ID="295a6e92-142e-440f-8a36-ffaadd3970cb" srcIP="10.111.1.196:48992" resp=200

the request have labelSelector parameters cause I can't get metric how do I config to ignore parameters ? or how do I config can use other pods's metric as pods hpa rule ?

simple architecture image

prometheus adapter config

    - seriesQuery: 'nbl_redis_llen{app_kubernetes_io_instance="flx-coll-grpc", app_kubernetes_io_name="nbl-exporter2"}'
      resources:
        overrides:
          namespace:
            resource: namespace
          pod:
            resource: pod
      name:
        matches: "nbl_redis_llen"
        as: "nbl_redis_llen_flx_coll_grpc2"
      metricsQuery: 'nbl_redis_llen{app_kubernetes_io_instance="flx-coll-grpc", app_kubernetes_io_name="nbl-exporter2"}'

hpa config

spec:
  maxReplicas: 5
  metrics:
  - pods:
      metric:
        name: nbl_redis_llen_flx_coll_grpc2
      target:
        averageValue: "10"
        type: Value
    type: Pods
  minReplicas: 1
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: flx-coll-grpc-worker2
dashpole commented 7 months ago

/assign @dgrisonnet /triage accepted

kaisbaccour commented 2 months ago

Hey! Facing the same problem. Any updates on this?

owanio1992 commented 2 months ago

@kaisbaccour current I change to use KEDA solution, you can try https://keda.sh/