civo / cli

Our Command Line Interface (CLI) for interacting with your Civo resources
Apache License 2.0
178 stars 87 forks source link

Scaling Node Pool via CLI Fails When No Nodes are Present in the Pool #449

Open aakashnagpal4567 opened 2 months ago

aakashnagpal4567 commented 2 months ago

Description

When there is no node in the node pool in the cluster, when we scale that node pool using Civo CLI, it's giving the error. This functionality is working fine in the UI.

Steps to Replicate

  1. Create a Civo Kubernetes cluster with two node pools
  2. Scale one of the node pools to zero using the UI
  3. Using the CLI, scale the nodes in the node pool from zero to the desired amount with this command: civo kubernetes node-pool scale CLUSTER_NAME NODEPOOL_ID -n <NUMBER_OF_NODES>

For example, i tried to scale the node in the node pool that is empty, it gives an error:

$ civo kubernetes node-pool scale node-test 76a32cac-60a3-4a69-999d-251fac214c10 -n 2
Please check if you are using the latest version of CLI and retry the command 
If you are still facing issues, please report it on our community slack or open a GitHub issue (https://github.com/civo/cli/issues) 
Error: Unable to find "76a32cac-60a3-4a69-999d-251fac214c10" node pool inside "node-test" cluster

For reference, I'm attaching a video here:

https://github.com/user-attachments/assets/e4541f2f-03c4-485b-85b7-25661b7385d3

Acceptance Criteria

Praveen005 commented 2 months ago

Adding more context to it.

I tried recreating the issue, but I stumbled upon the following:

When you create the cluster using UI, it is not getting reflected in the CLI, and vice-versa. That's why you got that error.

cluster on the UI:

Screenshot 2024-08-15 005650

CLI giving error:

Screenshot 2024-08-15 004010

cluster created on CLI, not reflecting the UI:

Screenshot 2024-08-15 010332

Nothing on the UI:

Screenshot 2024-08-15 010306

Quota rightly swelling up though:

Screenshot 2024-08-15 010242

Post cluster creation using the UI, I couldn't scale down to zero node node-pool as shown in your video, it gave an error. For n != 0 scaling up and down is going through as intended.

Screenshot 2024-08-15 005400

When you create the cluster using CLI, it gets created successfully, scaling up and down the node-pool also goes as intended, except when you want to scale down to zero nodes.

Screenshot 2024-08-15 010910

Summary:

  1. Scaling down to zero node is not happening either on the UI or the CLI. I am using the latest version of the CLI.
image
  1. Cluster created using UI is not reflecting in the CLI and vice-versa.
fernando-villalba commented 2 months ago

@Praveen005 could it be you were on one region on the UI and in another on the CLI?

aakashnagpal4567 commented 2 months ago

@Praveen005 For this you need to create two node pools, please read the 1st point in "Steps to replicate". Currently, you can't scale the node pool to zero using CLI, it can only be done via the UI. We also have to add this functionality, if there are multiple node pools, we should be able to scale the node pool to zero, and also have to ensure that there should be at least one node in the cluster.

Praveen005 commented 2 months ago

Hi @fernando-villalba, You were right, it was a region mismatch. Thank you.

@aakashnagpal4567 I could recreate the issue now. Thank you.

Praveen005 commented 2 months ago

Hi Sir, @fernando-villalba

I wanted to update you on the issue, I worked on it,

And here is the gist:

  1. We can now scale up a node-pool which has zero nodes using CIVO CLI (fix explained below).
  2. While trying to scale node-pool down to zero-nodes using CIVO CLI, I am faced with the following error, for more context and to make informed changes, I need to see how it is being handled on the API side, to which I don't have access to.
image

Currently, scaling up a node pool with zero nodes using CIVO CLI results in an error because we are searching for the node pool in kubernetesFindCluster.Pools. However, when a node pool is scaled down to zero nodes, it is no longer active and cannot be found, leading to the error.

https://github.com/civo/cli/blob/8a5fa361e070a817a5d07d71d582badf9a274f38/cmd/kubernetes/kubernetes_nodepool_scale.go#L53-L64

To fix this, we can look for the pool in kubernetesFindCluster.RequiredPools. and add it to the nodePool. Like below:

image

This fix solves one part of the issue. I have tested it, and it works fine.

Result:

Initial state: one node pool with zero node

Screenshot 2024-08-15 151236

Can't scale up using CLI:

Screenshot 2024-08-15 152026

Post code changes:

Node pool rightly getting scaled up:

Screenshot 2024-08-15 152108

Change reflecting in UI:

Screenshot 2024-08-15 152202

Up & Running:

Screenshot 2024-08-15 152222

For the second part,

Even when I pass two node-pools in which one clearly has an active node , as evident below:

configKubernetes: &{ NYC1 1 [{ ad8793ba-dbfe-4286-a276-7167fc19ef38 1 g4s.kube.xsmall map[] [] false} { f805408a-de4c-4fb6-9989-fa1010b686d8 0 g4s.kube.xsmall map[] [] false}] }

https://github.com/civo/cli/blob/8a5fa361e070a817a5d07d71d582badf9a274f38/cmd/kubernetes/kubernetes_nodepool_scale.go#L77

I am getting the following error:

Screenshot 2024-08-16 152158

It needs to seen, how validation on the server side is happening.

Regards!