Open jark-AB opened 1 month ago
today is a community bi-weekly call, I will add it to the agenda, you are welcome to attend as well :)
I encountered the similar issue in 2.13.1
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x14a5b20]
goroutine 1133 [running]:
github.com/kedacore/keda/v2/pkg/scaling/executor.(*scaleExecutor).RequestScale(0x4000a3b400, {0x41d8d78, 0x401c3c0cd0}, 0x401c390e00, 0x1, 0x0)
/workspace/pkg/scaling/executor/scale_scaledobjects.go:39 +0x140
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).checkScalers(0x40009705b0, {0x41d8d78, 0x401c3c0cd0}, {0x3ab0040?, 0x401c390e00?}, {0x41ae5d0, 0x401c7ce110})
/workspace/pkg/scaling/scale_handler.go:249 +0x334
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).startScaleLoop(0x4015aaecc0?, {0x41d8d78, 0x401c3c0cd0}, 0x401c7de000, {0x3ab0040, 0x401c390e00}, {0x41ae5d0, 0x401c7ce110}, 0xa0?)
/workspace/pkg/scaling/scale_handler.go:182 +0x374
created by github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).HandleScalableObject in goroutine 341
/workspace/pkg/scaling/scale_handler.go:128 +0x468
this was briefly discussed last week, possibly an issue with trigger cache invalidation. It may take some time before it gets fixed.
Report
KEDA Version: 2.15.1 Running on Kubernetes 1.29+ {"version": "v1.29.7-eks-a18cd3a"}
I am running into a transient issue where i have multiple scaledObjects (464) in many namespaces. We use
metricsAPI
scaler to poll against an api pod in each namespace that dynamically scales off an endpoint.This works extremely well and at scale. However, what we noticed is if there is an issue where multiple scaledObjects are failing at once (such as a networking or control plane issue), the Operator will run into a null pointer reference when it runs into many variations of:
Can replace
namespace
with any of the 150 namespaces etc. This results in the Keda Operator restarting and prior to a restart the following nil pointer reference logs are shown:Expected Behavior
I'd expect the Operator to not run into null pointers when Scalers fail at a large amount or run into transient issues.
Actual Behavior
Operator cannot get a scaling decision, spread across 500 scaledObjects and runs into nill pointer panic and restarts
Steps to Reproduce the Problem
Logs from KEDA operator
KEDA Version
2.15.1
Kubernetes Version
1.29
Platform
Amazon Web Services
Scaler Details
MetricsAPI
Anything else?
No response