kedacore / http-add-on

Add-on for KEDA to scale HTTP workloads
https://kedacore.github.io/http-add-on/
Apache License 2.0
372 stars 97 forks source link

Deployment replica count loop from 1 to 0 while using cron trigger and keda http add on trigger #1147

Open stefanfvast opened 2 months ago

stefanfvast commented 2 months ago

Report

When using the cron trigger along with the keda http add on trigger, the deployment scales to 1 as expected during the cron interval, then immediately to 0. Used individually, the triggers work as expected.

Expected Behavior

While both triggers are active, I would expect the the deployment to be at 1 desiredReplicas, and outside the cron interval it should scale to 1 only while requests are sent to the ingress.

Actual Behavior

The deployment scales to 1 as results of the cron metric which returns 1, and then immediately scales back to 0.

Steps to Reproduce the Problem

Deploy the httpscaledobject and the scaledobject separately, and then add the cron trigger and the httpscaledobject trigger:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: be-server-keda-scaling
spec:
  cooldownPeriod: 30
  maxReplicaCount: 1
  minReplicaCount: 0
  pollingInterval: 15
  scaleTargetRef:
    name: be-server
  triggers:
  - metadata:
      desiredReplicas: '1'
      end: 0 19 * * 1-5
      start: 0 5 * * 1-5
      timezone: Europe/London
    type: cron
    name: cron_scaler
  - metadata:
      httpScaledObject: be-server-keda-scaling
      scalerAddress: keda-add-ons-http-external-scaler.keda-add-ons-http:9090
    type: external-push
    name: requests_scaler
apiVersion: http.keda.sh/v1alpha1
kind: HTTPScaledObject
metadata:
  annotations:
    httpscaledobject.keda.sh/skip-scaledobject-creation: 'true'
  name: be-server-keda-scaling
spec:
  hosts:
  - beserver.com
  replicas:
    max: 1
    min: 0
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: be-server
    port: 80
    service: be-server

Resulting hpa resource:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  creationTimestamp: '2024-09-20T12:28:07Z'
  labels:
    app.kubernetes.io/managed-by: keda-operator
    app.kubernetes.io/name: keda-hpa-be-server-keda-scaling
    app.kubernetes.io/part-of: be-server-keda-scaling
    app.kubernetes.io/version: 2.15.1
    scaledobject.keda.sh/name: be-server-keda-scaling

  name: keda-hpa-be-server-keda-scaling
  namespace: review-2417---be
  ownerReferences:
    - apiVersion: keda.sh/v1alpha1
      blockOwnerDeletion: true
      controller: true
      kind: ScaledObject
      name: be-server-keda-scaling
      uid: 9fb33d6c-1325-45e0-b19d-0da46c4740ed
  resourceVersion: '191448816'
  uid: c7743f00-75c5-40b0-b058-db7c1598cbeb
spec:
  maxReplicas: 1
  metrics:
    - external:
        metric:
          name: s0-cron-Europe-London-05xx1-5-019xx1-5
          selector:
            matchLabels:
              scaledobject.keda.sh/name: be-server-keda-scaling
        target:
          averageValue: '1'
          type: AverageValue
      type: External
    - external:
        metric:
          name: s1-http-review-2417---be_002Fbe-server-keda-scaling
          selector:
            matchLabels:
              scaledobject.keda.sh/name: be-server-keda-scaling
        target:
          averageValue: '100'
          type: AverageValue
      type: External
  minReplicas: 1
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: be-server
status:
  conditions:
    - lastTransitionTime: '2024-09-20T12:28:17Z'
      message: the HPA controller was able to get the target's current scale
      reason: SucceededGetScale
      status: 'True'
      type: AbleToScale
    - lastTransitionTime: '2024-09-20T13:34:36Z'
      message: scaling is disabled since the replica count of the target is zero
      reason: ScalingDisabled
      status: 'False'
      type: ScalingActive
    - lastTransitionTime: '2024-09-20T12:44:47Z'
      message: the desired count is within the acceptable range
      reason: DesiredWithinRange
      status: 'False'
      type: ScalingLimited
  currentMetrics: null
  desiredReplicas: 0

Logs from KEDA HTTP operator

2024-09-20T13:37:09Z DEBUG scaleexecutor ScaleTarget no change {"scaledobject.Name": "be-server-keda-scaling", "scaledObject.Namespace": "review-2417---be", "scaleTarget.Name": "be-server"} 2024-09-20T13:37:09Z DEBUG scale_handler Getting metrics and activity from scaler {"scaledObject.Namespace": "review-2417---be", "scaledObject.Name": "be-server-keda-scaling", "scaler": "cron_scaler", "metricName": "s0-cron-Europe-London-05xx1-5-019xx1-5", "metrics": [{"metricName":"s0-cron-Europe-London-05xx1-5-019xx1-5","metricLabels":null,"timestamp":"2024-09-20T13:37:09Z","value":"1"}], "activity": true, "scalerError": null} 2024-09-20T13:37:09Z DEBUG scale_handler Scaler for scaledObject is active {"scaledObject.Namespace": "review-2417---be", "scaledObject.Name": "be-server-keda-scaling", "scaler": "cron_scaler", "metricName": "s0-cron-Europe-London-05xx1-5-019xx1-5"} 2024-09-20T13:37:09Z DEBUG scale_handler Getting metrics and activity from scaler {"scaledObject.Namespace": "review-2417---be", "scaledObject.Name": "be-server-keda-scaling", "scaler": "requests_scaler", "metricName": "s1-http-review-2417---be_002Fbe-server-keda-scaling", "metrics": [{"metricName":"s1-http-review-2417---be_002Fbe-server-keda-scaling","metricLabels":null,"timestamp":"2024-09-20T13:37:09Z","value":"0"}], "activity": false, "scalerError": null} 2024-09-20T13:37:09Z INFO scaleexecutor Successfully updated ScaleTarget {"scaledobject.Name": "be-server-keda-scaling", "scaledObject.Namespace": "review-2417---be", "scaleTarget.Name": "be-server", "Original Replicas Count": 0, "New Replicas Count": 1} 2024-09-20T13:37:09Z DEBUG events Scaled apps/v1.Deployment review-2417---be/be-server from 0 to 1, triggered by cron_scaler {"type": "Normal", "object": {"kind":"ScaledObject","namespace":"review-2417---be","name":"be-server-keda-scaling","uid":"9fb33d6c-1325-45e0-b19d-0da46c4740ed","apiVersion":"keda.sh/v1alpha1","resourceVersion":"191451412"}, "reason": "KEDAScaleTargetActivated"} 2024-09-20T13:37:09Z INFO scaleexecutor Successfully set ScaleTarget replicas count to ScaledObject minReplicaCount {"scaledobject.Name": "be-server-keda-scaling", "scaledObject.Namespace": "review-2417---be", "scaleTarget.Name": "be-server", "Original Replicas Count": 1, "New Replicas Count": 0} 2024-09-20T13:37:09Z DEBUG events Deactivated apps/v1.Deployment review-2417---be/be-server from 1 to 0 {"type": "Normal", "object": {"kind":"ScaledObject","namespace":"review-2417---be","name":"be-server-keda-scaling","uid":"9fb33d6c-1325-45e0-b19d-0da46c4740ed","apiVersion":"keda.sh/v1alpha1","resourceVersion":"191451412"}, "reason": "KEDAScaleTargetDeactivated"} 2024-09-20T13:37:09Z DEBUG scaleexecutor ScaleTarget no change {"scaledobject.Name": "be-server-keda-scaling", "scaledObject.Namespace": "review-2417---be", "scaleTarget.Name": "be-server"} 2024-09-20T13:37:10Z DEBUG scaleexecutor ScaleTarget no change {"scaledobject.Name": "be-server-keda-scaling", "scaledObject.Namespace": "review-2417---be", "scaleTarget.Name": "be-server"}



### HTTP Add-on Version

0.8.0

### Kubernetes Version

1.29

### Platform

Amazon Web Services

### Anything else?

_No response_
stale[bot] commented 17 hours ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.