Feature Request: Automatically create and delete a horizontal pod autoscaler

[abatilo]

HPA's in the Kubernetes world are not capable of having a minReplicas of 0. So you always have to have 1 replica no matter what. It would be really amazing if sablier could actually operate by creating an HPA where the number of replicas would be the minReplicas of the HPA, and then the HPA would exist and be able to scale out for more capacity, and then when the sablier session is over, delete the HPA and scale back down to 0

[acouvreur]

Actually the scaling for the HPA will be disabled.

Your HPA will the the following status: Status: ScalingDisabled Reason: scalingis disabled since the replica count of the target is zero

[gorositopablo]

@acouvreur so at the moment I have Sablier deployed and it works great, but only spins up 1 replica, and I understand that so far there is no way to have both in place (Sablier and HPA)

Just wondering how the Keda HTTP add-on does it. It works similarly to the proposal somehow. You got a CRD called HTTPScaledObject, for each of those, they create an HPA. Once you delete the HTTPScaledObject, they delete the HPA.

The generated HPA have that message as you mentioned, when there are zero replicas, but still works when you scale from 0 to X pods.

    - type: ScalingActive
      status: 'False'
      lastTransitionTime: '2024-06-28T13:57:19Z'
      reason: ScalingDisabled
      message: scaling is disabled since the replica count of the target is zero

If you have a HTTPScaledObject like this:

kind: HTTPScaledObject
apiVersion: http.keda.sh/v1alpha1
metadata:
  name: demo-nginx
  namespace: demo-ns
spec:
  hosts:
    - example.com
  pathPrefixes:
    - /
  scaleTargetRef:
    deployment: demo-nginx
    service: demo-nginx
    port: 80
  replicas:
    min: 0
    max: 10
  scaledownPeriod: 120

This is the generated HPA. As you can see they created with a prefix keda-hpa-+ whatever your HTTPScaledObject name is.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: keda-hpa-demo-nginx
  namespace: demo-ns
  uid: 3faaec0b-01d2-4ba3-aed8-001972c70b37
  resourceVersion: '28023995'
  creationTimestamp: '2024-06-28T13:57:18Z'
  labels:
    app.kubernetes.io/managed-by: keda-operator
    app.kubernetes.io/name: keda-hpa-demo-nginx
    app.kubernetes.io/part-of: demo-nginx
    app.kubernetes.io/version: 2.11.2
    scaledobject.keda.sh/name: demo-nginx
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: >
      {"apiVersion":"http.keda.sh/v1alpha1","kind":"HTTPScaledObject","metadata":{"annotations":{},"name":"demo-nginx","namespace":"demo-ns"},"spec":{"hosts":["example.com"],"pathPrefixes":["/"],"replicas":{"max":10,"min":0},"scaleTargetRef":{"deployment":"demo-nginx","port":80,"service":"demo-nginx"},"scaledownPeriod":120}}
  ownerReferences:
    - apiVersion: keda.sh/v1alpha1
      kind: ScaledObject
      name: demo-nginx
      uid: 68883bf5-6484-47b3-b9a2-3839f3bfc488
      controller: true
      blockOwnerDeletion: true
  managedFields:
    - manager: Go-http-client
      operation: Update
      apiVersion: autoscaling/v2
      time: '2024-06-28T13:57:18Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:metadata:
          f:annotations:
            .: {}
            f:kubectl.kubernetes.io/last-applied-configuration: {}
          f:labels:
            .: {}
            f:app.kubernetes.io/managed-by: {}
            f:app.kubernetes.io/name: {}
            f:app.kubernetes.io/part-of: {}
            f:app.kubernetes.io/version: {}
            f:scaledobject.keda.sh/name: {}
          f:ownerReferences:
            .: {}
            k:{"uid":"68883bf5-6484-47b3-b9a2-3839f3bfc488"}: {}
        f:spec:
          f:maxReplicas: {}
          f:metrics: {}
          f:minReplicas: {}
          f:scaleTargetRef:
            f:apiVersion: {}
            f:kind: {}
            f:name: {}
    - manager: kube-controller-manager
      operation: Update
      apiVersion: autoscaling/v2
      time: '2024-06-28T13:57:19Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:status:
          f:conditions:
            .: {}
            k:{"type":"AbleToScale"}:
              .: {}
              f:lastTransitionTime: {}
              f:message: {}
              f:reason: {}
              f:status: {}
              f:type: {}
            k:{"type":"ScalingActive"}:
              .: {}
              f:lastTransitionTime: {}
              f:message: {}
              f:reason: {}
              f:status: {}
              f:type: {}
      subresource: status
  selfLink: >-
    /apis/autoscaling/v2/namespaces/demo-ns/horizontalpodautoscalers/keda-hpa-demo-nginx
status:
  desiredReplicas: 0
  currentMetrics: null
  conditions:
    - type: AbleToScale
      status: 'True'
      lastTransitionTime: '2024-06-28T13:57:19Z'
      reason: SucceededGetScale
      message: the HPA controller was able to get the target's current scale
    - type: ScalingActive
      status: 'False'
      lastTransitionTime: '2024-06-28T13:57:19Z'
      reason: ScalingDisabled
      message: scaling is disabled since the replica count of the target is zero
spec:
  scaleTargetRef:
    kind: Deployment
    name: demo-nginx
    apiVersion: apps/v1
  minReplicas: 1
  maxReplicas: 10
  metrics:
    - type: External
      external:
        metric:
          name: s0-http-demo-ns_002Fdemo-nginx
          selector:
            matchLabels:
              scaledobject.keda.sh/name: demo-nginx
        target:
          type: AverageValue
          averageValue: '100'

[acouvreur]

@acouvreur so at the moment I have Sablier deployed and it works great, but only spins up 1 replica, and I understand that so far there is no way to have both in place (Sablier and HPA)

I believe you can tell your HPA to not do anything when it's scaled to 0. So once it's scaled back to 1, the HPA can kick in and adjust manually.

Did you have any luck with that @gorositopablo ?

[pablogorosito]

Yeah I've tried that today. The problem is, that the HPA itself it forces the minimum replica you have, so it spin up a pod as soon as you deploy the HPA.

[acouvreur]

Have you looked at @pablogorosito https://midbai.com/en/post/hpa-scale-to-zero/ ? Does it solves your issue ?

[gorositopablo]

Have you looked at @pablogorosito https://midbai.com/en/post/hpa-scale-to-zero/ ? Does it solves your issue ?

No, I think that actually, the problem that I have is different.

Sablier and HPA work fine out of the box, but Sablier only deletes the replica if I make at least one request to it, after that first request it gets deleted.

But if I deploy my API and I don't send at least one request to it, Sablier could leave there forever. Is there something that I'm missing? or is that normal behaviour?

Anyway, may not be related here, I could open another issue with the right question.

[acouvreur]

Have you looked at @pablogorosito https://midbai.com/en/post/hpa-scale-to-zero/ ? Does it solves your issue ?

No, I think that actually, the problem that I have is different.

Sablier and HPA work fine out of the box, but Sablier only deletes the replica if I make at least one request to it, after that first request it gets deleted.

But if I deploy my API and I don't send at least one request to it, Sablier could leave there forever. Is there something that I'm missing? or is that normal behaviour?

Anyway, may not be related here, I could open another issue with the right question.

Does that issue properly states what you described ?

https://github.com/acouvreur/sablier/issues/153

I can make a new feature for release 1.8.0 with this behavior.