harvester / tests

Harvester test cases
Apache License 2.0
10 stars 31 forks source link

[TEST] Rancher v2.9.0 alpha integration test of Harvester charts for K8s v1.29 #1337

Closed TachunLin closed 2 months ago

TachunLin commented 2 months ago

In order to test the K8s v1.29 support for Harvester-cloud-provider and Harvester-csi-driver of the following tickets:

We need to perform Rancher v2.9.0 integration test with Harvester v1.3.1

Test environment

Test plans

TachunLin commented 2 months ago

Verifying on Rancher v2.9.0-alpha5 with Harvester v1.3.1 for K8s v1.29.5

Result

$\color{green}{\textsf{PASS}}$ RKE2: Necessary checks for cloud provider and csi driver working on cluster. $~~$ * Can provision v1.29.5 RKE2 cluster well
$\color{green}{\textsf{PASS}}$: RKE2 running well $~~$ ![image](https://github.com/harvester/tests/assets/29251855/195dffd7-42cb-4fa1-b823-0d60dfd9e08f)
* Check the default cloud provider version:
$\color{orange}{\textsf{Not ready}}$: default cloud provider version is 0.2.3 not 0.2.4 $~~$ ![image](https://github.com/harvester/tests/assets/29251855/281d5886-822d-4c1c-9ec4-4d90e36bba31)
* Check the default csi-driver version:
$\color{green}{\textsf{PASS}}$: default csi driver version is 0.1.1700 $~~$ ![image](https://github.com/harvester/tests/assets/29251855/13695325-1086-4e2d-9cf4-8302243b9511)
* Check the default storage class set to `Harvester`
$\color{green}{\textsf{PASS}}$: set to Harvester 0.1.1700 $~~$ ![image](https://github.com/harvester/tests/assets/29251855/a649599d-91ea-43dc-b5bb-de6ab0c8114b)
* Check the application created pvc and pv
$\color{green}{\textsf{PASS}}$: Can create nginx application pvc $~~$ ![image](https://github.com/harvester/tests/assets/29251855/bef57e78-6848-4e34-a004-30e4adad19f1)
$\color{green}{\textsf{PASS}}$: Can create nginx application pv $~~$ ![image](https://github.com/harvester/tests/assets/29251855/f313c72f-e953-4065-8b61-cbf5a3408f5e)
$\color{green}{\textsf{PASS}}$: nginx pvc created one Harvester $~~$ ![image](https://github.com/harvester/tests/assets/29251855/5d339672-9515-4309-bdef-e7c4aadc779c)
* Check to create the dhcp and pool type load balancer for Harvester cloud provider
$\color{green}{\textsf{PASS}}$: Can create dhcp loadbalancer and route correctly $~~$ ![image](https://github.com/harvester/tests/assets/29251855/dde49bb8-3fe4-4d8b-bc67-818c240b4057)
$\color{green}{\textsf{PASS}}$: Can create pool loadbalancer and route correctly $~~$ ![image](https://github.com/harvester/tests/assets/29251855/ff4c907d-1e6a-4c71-9b24-e52ba201a60f)
$\color{green}{\textsf{PASS}}$ Delete RKE2 guest cluster on Rancher v2.9.0-alpha5 $~~$ * Test automation result
$\color{green}{\textsf{PASS}}$: Can correctly delete RKE2 v1.29.5 cluster $~~$ ![image](https://github.com/harvester/tests/assets/29251855/f37d7426-cef8-4843-bfa9-cb06b2cec7e3) ![image](https://github.com/harvester/tests/assets/29251855/af4fe748-ed6f-4925-bfe9-cbdc0ea4ae0a)
$\color{green}{\textsf{PASS}}$ Execute Rancher integration test backend automation test $~~$ * Test automation result
$\color{green}{\textsf{PASS}}$: all Rancher integration backend test $~~$ - PASS all backend automation Rancher integration test result ![image](https://github.com/harvester/tests/assets/29251855/472b8280-a89a-4033-a448-9765f1ff5e02) ![image](https://github.com/harvester/tests/assets/29251855/c2004b68-c009-4071-9f05-f6e46556d390) - PASS the following Rancher integration test cases - test_import_harvester - test_add_project_owner_user - TestRKE2::test_create_rke2[1] - TestRKE2::test_create_pvc[1] - TestRKE2::test_csi_deployment[1] - TestRKE2::test_delete_deployment[1] - TestRKE2::test_deploy_nginx[1] - TestRKE2::test_load_balancer_service[1-dhcp] - TestRKE2::test_load_balancer_service[1-pool] - TestRKE2::test_delete_rke2[1] - TestRKE1::test_create_rke1[1] - TestRKE1::test_create_pvc[1] - TestRKE1::test_cloud_provider_chart[1] - TestRKE1::test_deploy_nginx[1] - TestRKE1::test_load_balancer_service[1-dhcp] - TestRKE1::test_load_balancer_service[1-pool] - TestRKE1::test_csi_driver_chart[1] - TestRKE1::test_csi_deployment[1] - TestRKE1::test_delete_deployment[1] - TestRKE1::test_delete_rke1[1]
$\color{green}{\textsf{PASS}}$ RKE1: Provision, install, uninstall of cloud-provider and csi-driver $~~$ * Provision RKE1 cluster
$\color{green}{\textsf{PASS}}$: Can correctly provision v1.29.5 RKE1 cluster $~~$ ![image](https://github.com/harvester/tests/assets/29251855/6eaa1fa4-f415-481e-bbbb-3caa95582761)
* Install cloud-provider and csi-driver
$\color{green}{\textsf{PASS}}$: Correctly install cloud-provider and csi-driver $~~$ ![image](https://github.com/harvester/tests/assets/29251855/ce8691fa-2360-48c8-ade6-ee1aafbc8b66)
* Check cloud-provider version on RKE1 cluster
$\color{green}{\textsf{PASS}}$: harvester-cloud-provider:104.0.1+up0.2.4 $~~$ ![image](https://github.com/harvester/tests/assets/29251855/ce8691fa-2360-48c8-ade6-ee1aafbc8b66)
* Check csi-driver version on RKE1 cluster
$\color{green}{\textsf{PASS}}$: harvester-csi-driver:104.0.0+up0.1.17 $~~$ ![image](https://github.com/harvester/tests/assets/29251855/ce8691fa-2360-48c8-ade6-ee1aafbc8b66)
* Uninstall csi-driver
$\color{green}{\textsf{PASS}}$: Can correctly uninstall csi-driver $~~$ ![image](https://github.com/harvester/tests/assets/29251855/e73fd621-90e2-444f-9e7d-0d86a8cb8f91)
* Uninstall cloud-provider
$\color{green}{\textsf{PASS}}$: Can correctly uninstall cloud-provider $~~$ ![image](https://github.com/harvester/tests/assets/29251855/b6be168b-1c7e-419c-b1d6-1b4772b78dd1)
$\color{green}{\textsf{PASS}}$ Delete RKE1 guest cluster on Rancher v2.9.0-alpha5 $~~$ * Test automation result
$\color{green}{\textsf{PASS}}$: Can correctly delete RKE1 v1.29.5 cluster $~~$ ![image](https://github.com/harvester/tests/assets/29251855/74b6d51f-a0f0-4a2d-b4a5-bdce12c3d3f5) ![image](https://github.com/harvester/tests/assets/29251855/1fd2e6e9-2557-4455-abbd-e4c0290fc9cb)
$\color{red}{\textsf{FAIL}}$ Create RKE2 guest cluster using terraform $~~$ * Check can provision v1.29.5 RKE2 cluster well (by automation test)
$\color{red}{\textsf{FAIL}}$: RKE2 cluster stuck in "waiting for cluster agent to connect" $~~$ ![image](https://github.com/harvester/tests/assets/29251855/3e182528-46cc-4a5d-8956-2f260ef4f4bf) ![image](https://github.com/harvester/tests/assets/29251855/ec9955b1-e4d5-467e-abd7-3782d7a596bf)
* Check can provision v1.29.5 RKE2 cluster well (manual test)
$\color{red}{\textsf{FAIL}}$: RKE2 cluster stuck in "waiting for cluster agent to connect" $~~$ ![image](https://github.com/harvester/tests/assets/29251855/4c8ad201-3a11-4931-980c-3b16a356e6bb) ![image](https://github.com/harvester/tests/assets/29251855/8a94b947-f5a2-4af1-b9ad-12fb4e2fa24b)
$\color{green}{\textsf{PASS}}$ Scale down and scale up of guest clusters $~~$ * Check can scale up the RKE2 cluster from 1 to 2 nodes
$\color{green}{\textsf{PASS}}$: Can scale up the RKE2 cluster from 1 to 2 nodes $~~$ ![image](https://github.com/harvester/tests/assets/29251855/d14e4707-760e-4434-b52b-e8f6432b4380)
* Check can scale up the RKE2 cluster from 2 to 3 nodes
$\color{green}{\textsf{PASS}}$: Can scale up the RKE2 cluster from 2 to 3 nodes $~~$ ![image](https://github.com/harvester/tests/assets/29251855/046e7f8d-5fd2-4b56-9c1b-7d09f83767a3)
* Check can scale down the RKE2 cluster from 3 to 2 nodes
$\color{green}{\textsf{PASS}}$: Can scale up the RKE2 cluster from 2 to 3 nodes $~~$ ![image](https://github.com/harvester/tests/assets/29251855/e8184c2c-9c7c-426f-9808-e3ad5e623942) ![image](https://github.com/harvester/tests/assets/29251855/0d764003-ee05-4cb7-8160-f56f43225716) ![image](https://github.com/harvester/tests/assets/29251855/6bc3714b-14b3-4665-991f-d95b89111e3f)
* Check can scale down the RKE2 cluster from 2 to 1 nodes
$\color{green}{\textsf{PASS}}$: Can scale up the RKE2 cluster from 2 to 3 nodes $~~$ ![image](https://github.com/harvester/tests/assets/29251855/f7be2f4c-fd22-4a1d-91ac-2cd6329134f7) ![image](https://github.com/harvester/tests/assets/29251855/648fa97d-d1b0-4eed-a045-de435e320b85) ![image](https://github.com/harvester/tests/assets/29251855/ae2619a4-e14a-4954-b4ea-05e86b55399b)
$\color{green}{\textsf{PASS}}$ RKE2: upgrade v1.28.10 to v1.29.5 $~~$ * Check can upgrade RKE2 v1.28.10 cluster to v1.29.5
$\color{green}{\textsf{PASS}}$: Can correctly upgrade RKE2 cluster from v1.28.10 to v1.29.5 $~~$ ![image](https://github.com/harvester/tests/assets/29251855/4ea10a87-f9ac-4ce3-a11b-30bbad437047)
$\color{green}{\textsf{PASS}}$ RKE2: Upgrade Rancher from v2.8.5 -> v2.9.0-alpha5 with RKE2 guest cluster $~~$ * Check upgrade Rancher from v2.8.5 to v2.9.0-alpha5
$\color{green}{\textsf{PASS}}$: Can upgrade Rancher from v2.8.5 to v2.9.0-alpha5 $~~$ ![image](https://github.com/harvester/tests/assets/29251855/9df6b10e-4965-43e4-8059-5276d6674d7e)
$\color{red}{\textsf{FAIL}}$ Create v1.29.5 RKE2 guest cluster using import cluster $~~$ * Create v1.29.5 RKE2 custom cluster
$\color{green}{\textsf{PASS}}$: Can import v1.29.5 RKE2 guest cluster in running state $~~$ ![image](https://github.com/harvester/tests/assets/29251855/1d243add-b042-4f4f-ad21-e29936bc820f)
* Check can install cloud provider from chart store
$\color{green}{\textsf{PASS}}$: Can install the latest cloud provider 0.2.4 $~~$ ![image](https://github.com/harvester/tests/assets/29251855/401e019b-b251-4577-9ec7-ad2120616fa8) ![image](https://github.com/harvester/tests/assets/29251855/482df123-85b4-4aa9-9f4e-76e1c3fe2f00)
* Check the bundled csi driver version
$\color{green}{\textsf{PASS}}$: Can install the latest csi driver 0.1.17 $~~$ ![image](https://github.com/harvester/tests/assets/29251855/b0d54968-b9a9-4a92-8005-2c366c96ebd0) ![image](https://github.com/harvester/tests/assets/29251855/4c05f403-ca7a-4ace-ae02-6ef4d4f4a564)
* Check the basic functionality of csi driver
$\color{green}{\textsf{PASS}}$: Harvester storage class, pvc and pv works well on application deployment $~~$ ![image](https://github.com/harvester/tests/assets/29251855/8fedb63b-0b15-4a4e-9f70-6a29adf3c29e) ![image](https://github.com/harvester/tests/assets/29251855/34aae962-2d38-4c80-abbe-d7b25e006550) ![image](https://github.com/harvester/tests/assets/29251855/d0c545c5-5da9-4bb3-8bb5-562a960cf29d) ![image](https://github.com/harvester/tests/assets/29251855/a6f84477-963a-44f2-9fca-f51f0d198f14)
* Check the basic functionality of cloud provider
$\color{red}{\textsf{FAIL}}$: The dhcp load balancer stuck in Pending state $~~$ ![image](https://github.com/harvester/tests/assets/29251855/d6f26871-a98d-4ea2-bea8-b0cea358c3ee) The harvester-cloud-provider keep in updating ![image](https://github.com/harvester/tests/assets/29251855/49fad897-46a5-4eae-bf69-5f917df663b2) With the `Crashloopbackoff` failure ![image](https://github.com/harvester/tests/assets/29251855/03345179-4426-4d09-8956-b59865ff8e40)
$\color{green}{\textsf{PASS}}$ Create v1.29.5 RKE2 guest cluster using custom cluster $~~$ * Create v1.29.5 RKE2 custom cluster
$\color{green}{\textsf{PASS}}$: Can create v1.29.5 RKE2 custom cluster in running state $~~$ ![image](https://github.com/harvester/tests/assets/29251855/f1a159c6-6c68-4147-bb01-60ee8b8bbaf0)
* Check the bundled cloud provider version
$\color{green}{\textsf{PASS}}$: The bundled cloud provider version for RKE2 v1.29.5 is 0.2.300 $~~$ ![image](https://github.com/harvester/tests/assets/29251855/02e1af15-a4da-452e-b17b-9a4a3208fb6b)
* Check the bundled csi driver version
$\color{green}{\textsf{PASS}}$: The bundled csi driver version for RKE2 v1.29.5 is 0.1.1700 $~~$ ![image](https://github.com/harvester/tests/assets/29251855/46f20d98-18bc-4dbe-a519-dcb5dbaff952)
* Check the basic functionality of csi driver
$\color{green}{\textsf{PASS}}$: Harvester storage class, pvc and pv works well on application deployment $~~$ ![image](https://github.com/harvester/tests/assets/29251855/1c7c2177-52c8-4fca-8f3e-71a1b98c7516) ![image](https://github.com/harvester/tests/assets/29251855/125e771b-dc4a-45e5-973a-2c7607e60b9e) ![image](https://github.com/harvester/tests/assets/29251855/f30616cf-2ce3-4563-a795-8675d68e57fe) ![image](https://github.com/harvester/tests/assets/29251855/38362042-6605-4f98-af85-73b43ccdd3ee) ![image](https://github.com/harvester/tests/assets/29251855/98428196-a4a6-456a-a247-b71e89fd2d22)
* Check the basic functionality of cloud provider
$\color{green}{\textsf{PASS}}$: Check the load balancer service can running well $~~$ ![image](https://github.com/harvester/tests/assets/29251855/b57a62e2-3f28-4fd2-bf21-cf25eacbbd9a)

Test Information

Verify Steps

Necessary checks for cloud provider and csi driver
TachunLin commented 2 months ago

After complete the full Rancher integration on Rancher v2.9.0-alpha5 with Harvester v1.3.1

Most of the provision of RKE2, RKE1 cluster, bundle chart of Harvester cloud-provider and csi-driver are working as expected.

As the above test result stated: https://github.com/harvester/tests/issues/1337#issuecomment-2192059083

We can pass the following test scenarios

While we found the harvester-cloud-provider chart bundled with RKE2 v1.29.5+rke2r1 is not yet updated to the latest version v0.2.4

According to the investigation result in comment https://github.com/rancher/rancher/issues/45470#issuecomment-2190719068, we need to wait next release for v1.29.6+rke2r1 to have v0.2.4 cloud provider bundled on it.

And we encounter the following failure while executing the following cases

  1. Create RKE2 guest cluster using terraform-provider-rancher2

  2. Create v1.29.5 RKE2 guest cluster using import cluster

    • When we have RKE2 cluster imported to Rancher and try to install csi-driver and cloud-provider, the cloud-provider can't fully deployed which cause failed to create the load balancer service
    • We have create following issue to track the failure
TachunLin commented 2 months ago

We also encounter issue when access the virtualization management in Rancher

Access Harvester in Rancher virtualization management, most of the UI text are displayed with the incorrect %variable-reference%

image

We also created the following issue for a track: https://github.com/rancher/dashboard/issues/11349 https://github.com/rancher/dashboard/issues/11350

TachunLin commented 2 months ago

Close this since we have finished all task activities.