Closed emirsaidh closed 5 months ago
Hello @emirsaidh , Thanks for reporting! The problem here is that pub/sub still reports the jobs some time after your ACK the message and KEDA doesn't check the messages at all, so we cannot check if the message is the same or not. In this case, I'd suggest 2 options to mitigate the gap:
I know that both aren't the best option, but the problem is that the backend (stackdriver api) still reports the message
Unfortunately both solutions does not work for us but thanks a lot for the response and your time @JorTurFer, I understand that is not about Keda but Pub/Sub.
Report
I have long-running ScaledJobs, lasting approximately 10 minutes. The trigger configuration appears as follows, scaling one job for each message in the PubSub queue. However, upon successful processing and sending an acknowledgment (ack) to PubSub, Keda checks the queue every 30 seconds. Due to PubSub's non-real-time behavior, it updates the UnackedMessageNumber almost after 2 minutes. Consequently, Keda scales new jobs every 30 seconds (3-4 times).
Expected Behavior
Actual Behavior
Steps to Reproduce the Problem
1- Set up a PubSub subscription. 2- Configure Keda to scale jobs based on SubscriptionSize. 3- Ensure that the ScaledJobs have a runtime of few minutes. 4- Start sending messages to the PubSub queue 5- Monitor the behavior of Keda as it scales jobs based on the messages received. 6- Observe that Keda incorrectly scales new jobs every 30 seconds, even though the ack message sent to PubSub.
Logs from KEDA operator
KEDA Version
2.11.2
Kubernetes Version
1.27
Platform
Other
Scaler Details
GCP PubSub
Anything else?
No response