Support master nodegroup scaling

[okozachenko1203]

This function cannot be realized in the scope of mcapi driver itself because Magnum restricts the resize to worker nodegroups at the API level. This issue is just a placeholder to discus master nodegroup scaling across magnum project.

Context

Cluster-API supports KCP resize and it gives us a chance to realize master nodegroup resize in Magnum. We received such requests from customers also these days.

Topics

• [ ] Offload the resize restriction from the magnum API because this is related to driver's capability. It will require the changes of the other drivers' validation/precheck logic. https://github.com/openstack/magnum/blob/b578bd8a78d890ad32aeb6066cc22ece165f54e2/magnum/api/controllers/v1/cluster_actions.py#L117-L120
• [ ] support both scaling up and down? Scaling down is supported by CAPI?
• [ ] autoscaling is needed for controlplane?
• [ ] what else?

[okozachenko1203]

A workaround/hack for now to scale master nodegroup bypassing magnum-api restriction;

• Update master_count in magnum db manually

  
  mysql> select * from nodegroup;
  +---------------------+---------------------+----+--------------------------------------+----------------+--------------------------------------+----------------------------------+--------------------+---------------------------------------------------------+-----------+--------------------------------------+----------------+------------+----------------+----------------+--------+------------+------------+-----------------+---------------+---------+
  | created_at          | updated_at          | id | uuid                                 | name           | cluster_id                           | project_id                       | docker_volume_size | labels                                                  | flavor_id | image_id                             | node_addresses | node_count | max_node_count | min_node_count | role   | is_default | stack_id   | status          | status_reason | version |
  +---------------------+---------------------+----+--------------------------------------+----------------+--------------------------------------+----------------------------------+--------------------+---------------------------------------------------------+-----------+--------------------------------------+----------------+------------+----------------+----------------+--------+------------+------------+-----------------+---------------+---------+
  | 2023-07-18 12:55:56 | 2023-07-18 12:59:54 | 11 | 122a5592-6a4d-4a76-8b9a-98fa3bb955ee | default-master | a0870170-f0ef-4ce0-ac5c-997d854d5268 | 2ffc6c32d8794e3289a34ca890c089f9 |               NULL | {"auto_scaling_enabled": "true", "max_node_count": "2"} | m1.medium | ef107f29-8f26-474e-8f5f-80d269c7d2cd | []             |          1 |           NULL |              0 | master |          1 | kube-7kjbp | CREATE_COMPLETE | NULL          | NULL    |
  | 2023-07-18 12:55:56 | 2023-07-18 13:00:24 | 12 | 1cb81e3c-42c2-40de-bcc9-a62eba34f93a | default-worker | a0870170-f0ef-4ce0-ac5c-997d854d5268 | 2ffc6c32d8794e3289a34ca890c089f9 |               NULL | {"auto_scaling_enabled": "true", "max_node_count": "2"} | m1.medium | ef107f29-8f26-474e-8f5f-80d269c7d2cd | []             |          1 |           NULL |              0 | worker |          1 | kube-7kjbp | CREATE_COMPLETE | NULL          | NULL    |
  +---------------------+---------------------+----+--------------------------------------+----------------+--------------------------------------+----------------------------------+--------------------+---------------------------------------------------------+-----------+--------------------------------------+----------------+------------+----------------+----------------+--------+------------+------------+-----------------+---------------+---------+
  2 rows in set (0.00 sec)

mysql> update nodegroup set node_count=3 where id=11; Query OK, 1 row affected (0.01 sec) Rows matched: 1 Changed: 1 Warnings: 0

mysql> select * from nodegroup; +---------------------+---------------------+----+--------------------------------------+----------------+--------------------------------------+----------------------------------+--------------------+---------------------------------------------------------+-----------+--------------------------------------+----------------+------------+----------------+----------------+--------+------------+------------+-----------------+---------------+---------+ | created_at | updated_at | id | uuid | name | cluster_id | project_id | docker_volume_size | labels | flavor_id | image_id | node_addresses | node_count | max_node_count | min_node_count | role | is_default | stack_id | status | status_reason | version | +---------------------+---------------------+----+--------------------------------------+----------------+--------------------------------------+----------------------------------+--------------------+---------------------------------------------------------+-----------+--------------------------------------+----------------+------------+----------------+----------------+--------+------------+------------+-----------------+---------------+---------+ | 2023-07-18 12:55:56 | 2023-07-18 12:59:54 | 11 | 122a5592-6a4d-4a76-8b9a-98fa3bb955ee | default-master | a0870170-f0ef-4ce0-ac5c-997d854d5268 | 2ffc6c32d8794e3289a34ca890c089f9 | NULL | {"auto_scaling_enabled": "true", "max_node_count": "2"} | m1.medium | ef107f29-8f26-474e-8f5f-80d269c7d2cd | [] | 3 | NULL | 0 | master | 1 | kube-7kjbp | CREATE_COMPLETE | NULL | NULL | | 2023-07-18 12:55:56 | 2023-07-18 13:00:24 | 12 | 1cb81e3c-42c2-40de-bcc9-a62eba34f93a | default-worker | a0870170-f0ef-4ce0-ac5c-997d854d5268 | 2ffc6c32d8794e3289a34ca890c089f9 | NULL | {"auto_scaling_enabled": "true", "max_node_count": "2"} | m1.medium | ef107f29-8f26-474e-8f5f-80d269c7d2cd | [] | 1 | NULL | 0 | worker | 1 | kube-7kjbp | CREATE_COMPLETE | NULL | NULL | +---------------------+---------------------+----+--------------------------------------+----------------+--------------------------------------+----------------------------------+--------------------+---------------------------------------------------------+-----------+--------------------------------------+----------------+------------+----------------+----------------+--------+------------+------------+-----------------+---------------+---------+ 2 rows in set (0.00 sec)

mysql> exit Bye ubuntu@mcapi-large:~$ o coe nodegroup list scale +--------------------------------------+----------------+-----------+--------------------------------------+------------+-----------------+--------+ | uuid | name | flavor_id | image_id | node_count | status | role | +--------------------------------------+----------------+-----------+--------------------------------------+------------+-----------------+--------+ | 122a5592-6a4d-4a76-8b9a-98fa3bb955ee | default-master | m1.medium | ef107f29-8f26-474e-8f5f-80d269c7d2cd | 3 | CREATE_COMPLETE | master | | 1cb81e3c-42c2-40de-bcc9-a62eba34f93a | default-worker | m1.medium | ef107f29-8f26-474e-8f5f-80d269c7d2cd | 1 | CREATE_COMPLETE | worker | +--------------------------------------+----------------+-----------+--------------------------------------+------------+-----------------+--------+ ubuntu@mcapi-large:~$ o coe cluster list +--------------------------------------+-------+---------+------------+--------------+-----------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+-------+---------+------------+--------------+-----------------+---------------+ | a0870170-f0ef-4ce0-ac5c-997d854d5268 | scale | None | 1 | 3 | CREATE_COMPLETE | HEALTHY | +--------------------------------------+-------+---------+------------+--------------+-----------------+---------------+

- Increase the replicas of `Cluster.cluster.x-k8s.io` by setting `spec.topology.controlPlane.replicas`. Then the CAPI will reconcile the KCP
- Finally
```sh
$ kg node
NAME                                          STATUS   ROLES                  AGE     VERSION
kube-7kjbp-control-plane-vxtrz-kkxkw          Ready    control-plane,master   10m     v1.25.3
kube-7kjbp-control-plane-vxtrz-nhjr2          Ready    control-plane,master   3d1h    v1.25.3
kube-7kjbp-control-plane-vxtrz-vth4b          Ready    control-plane,master   8m22s   v1.25.3
kube-7kjbp-default-worker-infra-hnk8x-v6cp9   Ready    worker                 2d20h   v1.25.3

[mnaser]

@ngyenhuukho check this out, fyi this limitation is coming from Magnum at the moment.

[nguyenhuukhoi]

Hello, Did we have plan for this problem?

[mnaser]

There seems to a proposed spec here https://review.opendev.org/c/openstack/magnum-specs/+/905281

[nguyenhuukhoi]

Yes, It sound great. I am looking for a long time for this.

[waipeng]

There's a patch proposed at https://review.opendev.org/c/openstack/magnum/+/906086, do take a look to see if it fits your needs.