Allow the upscale but block the downscale in clusterQueue

[kerthcet]

What would you like to be added:

We should :

1. Allow the upscale in clusterQueue, like

   • [ ] Add new resources/flavors
   • [ ] Scale up the size of flavors

2. Block the downscale in clusterQueue, like

   • [ ] Remove resources, including flavors unless the flavor is not in use.
   • [ ] Scale down the size of flavors unless the flavor is not in use.

We can check that in cq.UsedResources.

Why is this needed:

More scaleable.

Completion requirements:

This enhancement requires the following artifacts:

• [x] Design doc
• [ ] API change
• [ ] Docs update

The artifacts should be linked in subsequent comments.

[alculquicondor]

I don't think we should block downscale, or at least it should be more flexible.

Once we have preemption support, when downscaling, we could start suspending some workloads if the quota is being violated.

Maybe we could look at the current state of the cache and only allow dowscale if the quota wouldn't be violated with the active workloads, as you say. However, we would be limiting the ability of a cluster administrator to quickly scale down resources if there is some sort of emergency.

[kerthcet]

Well, it's just a startup and based on the current implements. Maybe we can delay this until we implement the preemption.

[alculquicondor]

/lifecycle frozen /priority backlog