search

kedacore / keda

KEDA is a Kubernetes-based Event Driven Autoscaling component. It provides event driven scale for any container running in Kubernetes
https://keda.sh
Apache License 2.0
8.56k stars 1.08k forks source link

scaledObject name is not specified #6337

Open SadmiB opened 5 days ago

SadmiB commented 5 days ago

Report

I installed keda in EKS 1.29 using helm chart via terraform, using the following snippet:

resource "helm_release" "keda" {
  name       = "keda"
  namespace  = "keda"
  repository = "https://kedacore.github.io/charts"
  chart      = "keda"
  version    = "2.14.0"

  create_namespace = true
}

I tried keda 2.14, 1.15, 2.16

I didn't create any scaled object yet

Expected Behavior

Keda working find and no error logs.

Actual Behavior

I'm getting lot of logs from keda-operator-metrics-apiserver as follows:

E1117 08:36:56.932694       1 provider.go:101] "msg"="please specify scaledObject name, it needs to be set as value of label selector \"scaledobject.keda.sh/name\" on the query" "error"="scaledObject name is not specified" "logger"="keda_metrics_adapter.provider"
E1117 08:36:56.932746       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"scaledObject name is not specified"}: scaledObject name is not specified

Steps to Reproduce the Problem

  1. Install keda using terraform with helm chart
  2. check keda-operator-metrics-apiserver pod logs

Logs from KEDA operator

2024/11/17 08:35:27 maxprocs: Updating GOMAXPROCS=1: determined from CPU quota
2024-11-17T08:35:27Z    INFO    setup   Starting manager
2024-11-17T08:35:27Z    INFO    setup   KEDA Version: 2.14.0
2024-11-17T08:35:27Z    INFO    setup   Git Commit: 6681d5ecbc82293626cab693625c9cc9fb453381
2024-11-17T08:35:27Z    INFO    setup   Go Version: go1.21.9
2024-11-17T08:35:27Z    INFO    setup   Go OS/Arch: linux/amd64
2024-11-17T08:35:27Z    INFO    setup   Running on Kubernetes 1.29+ {"version": "v1.29.8-eks-a737599"}
2024-11-17T08:35:27Z    INFO    starting server {"kind": "health probe", "addr": "[::]:8081"}
I1117 08:35:27.784622       1 leaderelection.go:250] attempting to acquire leader lease keda/operator.keda.sh...
I1117 08:35:44.454586       1 leaderelection.go:260] successfully acquired lease keda/operator.keda.sh
2024-11-17T08:35:44Z    INFO    Starting EventSource    {"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "source": "kind source: *v1alpha1.ScaledObject"}
2024-11-17T08:35:44Z    INFO    Starting EventSource    {"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "source": "kind source: *v2.HorizontalPodAutoscaler"}
2024-11-17T08:35:44Z    INFO    Starting Controller {"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject"}
2024-11-17T08:35:44Z    INFO    Starting EventSource    {"controller": "triggerauthentication", "controllerGroup": "keda.sh", "controllerKind": "TriggerAuthentication", "source": "kind source: *v1alpha1.TriggerAuthentication"}
2024-11-17T08:35:44Z    INFO    Starting Controller {"controller": "triggerauthentication", "controllerGroup": "keda.sh", "controllerKind": "TriggerAuthentication"}
2024-11-17T08:35:44Z    INFO    Starting EventSource    {"controller": "scaledjob", "controllerGroup": "keda.sh", "controllerKind": "ScaledJob", "source": "kind source: *v1alpha1.ScaledJob"}
2024-11-17T08:35:44Z    INFO    Starting Controller {"controller": "scaledjob", "controllerGroup": "keda.sh", "controllerKind": "ScaledJob"}
2024-11-17T08:35:44Z    INFO    Starting EventSource    {"controller": "cloudeventsource", "controllerGroup": "eventing.keda.sh", "controllerKind": "CloudEventSource", "source": "kind source: *v1alpha1.CloudEventSource"}
2024-11-17T08:35:44Z    INFO    Starting Controller {"controller": "cloudeventsource", "controllerGroup": "eventing.keda.sh", "controllerKind": "CloudEventSource"}
2024-11-17T08:35:44Z    INFO    Starting EventSource    {"controller": "clustertriggerauthentication", "controllerGroup": "keda.sh", "controllerKind": "ClusterTriggerAuthentication", "source": "kind source: *v1alpha1.ClusterTriggerAuthentication"}
2024-11-17T08:35:44Z    INFO    Starting Controller {"controller": "clustertriggerauthentication", "controllerGroup": "keda.sh", "controllerKind": "ClusterTriggerAuthentication"}
2024-11-17T08:35:44Z    INFO    cert-rotation   starting cert rotator controller
2024-11-17T08:35:44Z    INFO    Starting EventSource    {"controller": "cert-rotator", "source": "kind source: *v1.Secret"}
2024-11-17T08:35:44Z    INFO    Starting EventSource    {"controller": "cert-rotator", "source": "kind source: *unstructured.Unstructured"}
2024-11-17T08:35:44Z    INFO    Starting EventSource    {"controller": "cert-rotator", "source": "kind source: *unstructured.Unstructured"}
2024-11-17T08:35:44Z    INFO    Starting Controller {"controller": "cert-rotator"}
2024-11-17T08:35:44Z    INFO    cert-rotation   no cert refresh needed
2024-11-17T08:35:44Z    INFO    cert-rotation   certs are ready in /certs
2024-11-17T08:35:44Z    INFO    Starting workers    {"controller": "triggerauthentication", "controllerGroup": "keda.sh", "controllerKind": "TriggerAuthentication", "worker count": 1}
2024-11-17T08:35:44Z    INFO    Starting workers    {"controller": "scaledjob", "controllerGroup": "keda.sh", "controllerKind": "ScaledJob", "worker count": 1}
2024-11-17T08:35:44Z    INFO    Starting workers    {"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "worker count": 5}
2024-11-17T08:35:44Z    INFO    Starting workers    {"controller": "cloudeventsource", "controllerGroup": "eventing.keda.sh", "controllerKind": "CloudEventSource", "worker count": 1}
2024-11-17T08:35:44Z    INFO    Starting workers    {"controller": "clustertriggerauthentication", "controllerGroup": "keda.sh", "controllerKind": "ClusterTriggerAuthentication", "worker count": 1}
2024-11-17T08:35:44Z    INFO    Starting workers    {"controller": "cert-rotator", "worker count": 1}
2024-11-17T08:35:44Z    INFO    cert-rotation   no cert refresh needed
2024-11-17T08:35:44Z    INFO    cert-rotation   Ensuring CA cert    {"name": "keda-admission", "gvk": "admissionregistration.k8s.io/v1, Kind=ValidatingWebhookConfiguration", "name": "keda-admission", "gvk": "admissionregistration.k8s.io/v1, Kind=ValidatingWebhookConfiguration"}
2024-11-17T08:35:44Z    INFO    cert-rotation   Ensuring CA cert    {"name": "v1beta1.external.metrics.k8s.io", "gvk": "apiregistration.k8s.io/v1, Kind=APIService", "name": "v1beta1.external.metrics.k8s.io", "gvk": "apiregistration.k8s.io/v1, Kind=APIService"}
2024-11-17T08:35:44Z    INFO    cert-rotation   no cert refresh needed
2024-11-17T08:35:44Z    INFO    cert-rotation   Ensuring CA cert    {"name": "keda-admission", "gvk": "admissionregistration.k8s.io/v1, Kind=ValidatingWebhookConfiguration", "name": "keda-admission", "gvk": "admissionregistration.k8s.io/v1, Kind=ValidatingWebhookConfiguration"}
2024-11-17T08:35:44Z    INFO    cert-rotation   Ensuring CA cert    {"name": "v1beta1.external.metrics.k8s.io", "gvk": "apiregistration.k8s.io/v1, Kind=APIService", "name": "v1beta1.external.metrics.k8s.io", "gvk": "apiregistration.k8s.io/v1, Kind=APIService"}
2024-11-17T08:35:46Z    INFO    cert-rotation   CA certs are injected to webhooks
2024-11-17T08:35:46Z    INFO    grpc_server Starting Metrics Service gRPC Server    {"address": ":9666"}
2024-11-17T08:37:25Z    INFO    cert-rotation   no cert refresh needed
2024-11-17T08:37:25Z    INFO    cert-rotation   Ensuring CA cert    {"name": "keda-admission", "gvk": "admissionregistration.k8s.io/v1, Kind=ValidatingWebhookConfiguration", "name": "keda-admission", "gvk": "admissionregistration.k8s.io/v1, Kind=ValidatingWebhookConfiguration"}
2024-11-17T08:37:25Z    INFO    cert-rotation   Ensuring CA cert    {"name": "v1beta1.external.metrics.k8s.io", "gvk": "apiregistration.k8s.io/v1, Kind=APIService", "name": "v1beta1.external.metrics.k8s.io", "gvk": "apiregistration.k8s.io/v1, Kind=APIService"}

KEDA Version

2.16.0

Kubernetes Version

1.29

Platform

Amazon Web Services

Scaler Details

No response

Anything else?

No response

SpiritZhou commented 1 day ago

Are there any other HPA instances running in Kubernetes?

SadmiB commented 1 day ago

Are there any other HPA instances running in Kubernetes?

Yes there are old HPAs not created by scaled objects

SpiritZhou commented 1 day ago

These logs are normal since all HPAs will query the metric value in KEDA's metric adapter, and old HPAs will be blocked.

SadmiB commented 1 day ago

Ah, the error message still confusing because there was no scaledobject in the cluster, and the logs polluted our logging platform.

for anyone facing this, I had to scale down keda-operator-metrics-apiserver in my case to test the cron functionality of keda.