Resize volume: add pool capacity disablethreshold for resize and allow volume auto migration

[weizhouapache]

Description

Currently, when resize a volume, ACS checks if the current storage pool can accommodate the new size, and simply return true if pool has enough capacity, or false if there are not enough capacity.

This PR

• add global and zone setting volume.resize.allowed.beyond.allocation and pool.storage.allocated.resize.capacity.disablethreshold
• add API parameter automigrate to resizeVolume API.

when resize a volume

• check if the pool capacity (with new volume size) is above disable threshold pool.storage.allocated.capacity.disablethreshold
• if pool capacity is below disable threshold, operation succeeds.
• if pool capacity is above disable threshold
  • if volume.resize.allowed.beyond.allocation is false, go to migration step
  • if volume.resize.allowed.beyond.allocation is true, it will check pool capacity for resize threshold pool.storage.allocated.resize.capacity.disablethreshold.
  • if yes, operation succeeds.
  • if not, go to migration step
• migrate volume to another pool
  • find storage pools by tags, storage types, pool allocation capacity (consider current volume size), and etc
  • filter storage pool by resize capacity threshold (consider new volume size)
  • if no pool found, operation fails.
  • if automigrate is false, operation fails
  • if automigrate is true, try to migrate volume to another pool
  • if migration succeeds (for example vm is stopped), operation succeeds
  • if migration fails (for example vm is running on KVM), operation succeeds

prior to this change

With this change

Types of changes

• [ ] Breaking change (fix or feature that would cause existing functionality to change)
• [ ] New feature (non-breaking change which adds functionality)
• [ ] Bug fix (non-breaking change which fixes an issue)
• [x] Enhancement (improves an existing feature and functionality)
• [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
• [ ] build/CI
• [ ] test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• [ ] Major
• [ ] Minor

Bug Severity

• [ ] BLOCKER
• [ ] Critical
• [ ] Major
• [ ] Minor
• [ ] Trivial

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

[weizhouapache]

@blueorangutan package

[blueorangutan]

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 50.76923% with 32 lines in your changes missing coverage. Please review.

Project coverage is 15.81%. Comparing base (046870e) to head (e92f38e). Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
...n/java/com/cloud/storage/VolumeApiServiceImpl.java	40.00%	19 Missing and 8 partials :warning:
...ain/java/com/cloud/storage/StorageManagerImpl.java	78.57%	3 Missing :warning:
...stack/api/command/user/volume/ResizeVolumeCmd.java	33.33%	2 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #9761 +/- ## ============================================ + Coverage 15.78% 15.81% +0.02% - Complexity 12551 12568 +17 ============================================ Files 5625 5625 Lines 491972 492051 +79 Branches 63764 60723 -3041 ============================================ + Hits 77662 77821 +159 + Misses 405850 405716 -134 - Partials 8460 8514 +54 ``` | [Flag](https://app.codecov.io/gh/apache/cloudstack/pull/9761/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | | |---|---|---| | [uitests](https://app.codecov.io/gh/apache/cloudstack/pull/9761/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `4.03% <ø> (-0.01%)` | :arrow_down: | | [unittests](https://app.codecov.io/gh/apache/cloudstack/pull/9761/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `16.63% <50.76%> (+0.03%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11261

[weizhouapache]

@blueorangutan test

[blueorangutan]

@weizhouapache a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

[weizhouapache]

@blueorangutan package

[blueorangutan]

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

[SF] Trillian test result (tid-11602) Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8 Total time taken: 380 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9761-t11602-kvm-ol8.zip Smoke tests completed. 0 look OK, 0 have errors, 81 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
all_test_deploy_vm	Skipped	---	test_deploy_vm.py
all_test_escalations_templates	Skipped	---	test_escalations_templates.py
all_test_vm_ha	Skipped	---	test_vm_ha.py
all_test_vm_sync	Skipped	---	test_vm_sync.py
all_test_accounts	Skipped	---	test_accounts.py
all_test_affinity_groups_projects	Skipped	---	test_affinity_groups_projects.py
all_test_affinity_groups	Skipped	---	test_affinity_groups.py
all_test_async_job	Skipped	---	test_async_job.py
all_test_certauthority_root	Skipped	---	test_certauthority_root.py
all_test_create_list_domain_account_project	Skipped	---	test_create_list_domain_account_project.py
all_test_deploy_vgpu_enabled_vm	Skipped	---	test_deploy_vgpu_enabled_vm.py
all_test_deploy_virtio_scsi_vm	Skipped	---	test_deploy_virtio_scsi_vm.py
all_test_deploy_vm_iso	Skipped	---	test_deploy_vm_iso.py
all_test_deploy_vm_root_resize	Skipped	---	test_deploy_vm_root_resize.py
all_test_deploy_vms_with_varied_deploymentplanners	Skipped	---	test_deploy_vms_with_varied_deploymentplanners.py
all_test_deploy_vm_with_userdata	Skipped	---	test_deploy_vm_with_userdata.py
all_test_diagnostics	Skipped	---	test_diagnostics.py
all_test_direct_download	Skipped	---	test_direct_download.py
all_test_disk_offerings	Skipped	---	test_disk_offerings.py
all_test_domain_disk_offerings	Skipped	---	test_domain_disk_offerings.py
all_test_domain_network_offerings	Skipped	---	test_domain_network_offerings.py
all_test_domain_service_offerings	Skipped	---	test_domain_service_offerings.py
all_test_domain_vpc_offerings	Skipped	---	test_domain_vpc_offerings.py
all_test_dynamicroles	Skipped	---	test_dynamicroles.py
all_test_global_settings	Skipped	---	test_global_settings.py
all_test_guest_vlan_range	Skipped	---	test_guest_vlan_range.py
all_test_host_annotations	Skipped	---	test_host_annotations.py
all_test_hostha_simulator	Skipped	---	test_hostha_simulator.py
all_test_internal_lb	Skipped	---	test_internal_lb.py
all_test_iso	Skipped	---	test_iso.py
all_test_list_ids_parameter	Skipped	---	test_list_ids_parameter.py
all_test_loadbalance	Skipped	---	test_loadbalance.py
all_test_login	Skipped	---	test_login.py
all_test_metrics_api	Skipped	---	test_metrics_api.py
all_test_migration	Skipped	---	test_migration.py
all_test_multipleips_per_nic	Skipped	---	test_multipleips_per_nic.py
all_test_nested_virtualization	Skipped	---	test_nested_virtualization.py
all_test_network_acl	Skipped	---	test_network_acl.py
all_test_network	Skipped	---	test_network.py
all_test_nic_adapter_type	Skipped	---	test_nic_adapter_type.py
all_test_nic	Skipped	---	test_nic.py
all_test_non_contigiousvlan	Skipped	---	test_non_contigiousvlan.py
all_test_outofbandmanagement_nestedplugin	Skipped	---	test_outofbandmanagement_nestedplugin.py
all_test_outofbandmanagement	Skipped	---	test_outofbandmanagement.py
all_test_over_provisioning	Skipped	---	test_over_provisioning.py
all_test_password_server	Skipped	---	test_password_server.py
all_test_portable_publicip	Skipped	---	test_portable_publicip.py
all_test_portforwardingrules	Skipped	---	test_portforwardingrules.py
all_test_primary_storage	Skipped	---	test_primary_storage.py
all_test_privategw_acl	Skipped	---	test_privategw_acl.py
all_test_projects	Skipped	---	test_projects.py
all_test_public_ip_range	Skipped	---	test_public_ip_range.py
all_test_pvlan	Skipped	---	test_pvlan.py
all_test_regions	Skipped	---	test_regions.py
all_test_reset_vm_on_reboot	Skipped	---	test_reset_vm_on_reboot.py
all_test_resource_accounting	Skipped	---	test_resource_accounting.py
all_test_resource_detail	Skipped	---	test_resource_detail.py
all_test_router_dhcphosts	Skipped	---	test_router_dhcphosts.py
all_test_router_dns	Skipped	---	test_router_dns.py
all_test_router_dnsservice	Skipped	---	test_router_dnsservice.py
all_test_routers_iptables_default_policy	Skipped	---	test_routers_iptables_default_policy.py
all_test_routers_network_ops	Skipped	---	test_routers_network_ops.py
all_test_routers	Skipped	---	test_routers.py
all_test_scale_vm	Skipped	---	test_scale_vm.py
all_test_secondary_storage	Skipped	---	test_secondary_storage.py
all_test_service_offerings	Skipped	---	test_service_offerings.py
all_test_snapshots	Skipped	---	test_snapshots.py
all_test_ssvm	Skipped	---	test_ssvm.py
all_test_staticroles	Skipped	---	test_staticroles.py
all_test_templates	Skipped	---	test_templates.py
all_test_usage_events	Skipped	---	test_usage_events.py
all_test_usage	Skipped	---	test_usage.py
all_test_vm_deployment_planner	Skipped	---	test_vm_deployment_planner.py
all_test_vm_life_cycle	Skipped	---	test_vm_life_cycle.py
all_test_vm_snapshots	Skipped	---	test_vm_snapshots.py
all_test_volumes	Skipped	---	test_volumes.py
all_test_vpc_redundant	Skipped	---	test_vpc_redundant.py
all_test_vpc_router_nics	Skipped	---	test_vpc_router_nics.py
all_test_vpc_vpn	Skipped	---	test_vpc_vpn.py
all_test_host_maintenance	Skipped	---	test_host_maintenance.py
all_test_hostha_kvm	Skipped	---	test_hostha_kvm.py

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11264

[sureshanaparti]

@blueorangutan test

[blueorangutan]

@sureshanaparti a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

[weizhouapache]

@blueorangutan test

[blueorangutan]

@weizhouapache a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian test result (tid-11607) Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8 Total time taken: 52087 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9761-t11607-kvm-ol8.zip Smoke tests completed. 140 look OK, 1 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_secure_vm_migration	Error	377.75	test_vm_life_cycle.py

[blueorangutan]

[SF] Trillian test result (tid-11603) Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8 Total time taken: 65044 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9761-t11603-kvm-ol8.zip Smoke tests completed. 139 look OK, 2 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_secure_vm_migration	Error	134.20	test_vm_life_cycle.py
test_01_secure_vm_migration	Error	134.21	test_vm_life_cycle.py
test_04_deploy_vnf_appliance	Error	477.32	test_vnf_templates.py
test_04_deploy_vnf_appliance	Error	477.32	test_vnf_templates.py
test_05_delete_vnf_template	Error	1.22	test_vnf_templates.py
ContextSuite context=TestVnfTemplates>:teardown	Error	1.43	test_vnf_templates.py

[sureshanaparti]

@blueorangutan package

[blueorangutan]

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11284

[rohityadavcloud]

@blueorangutan test

[blueorangutan]

@rohityadavcloud a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian test result (tid-11643) Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8 Total time taken: 53040 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9761-t11643-kvm-ol8.zip Smoke tests completed. 141 look OK, 0 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

[blueorangutan]

[SF] Trillian test result (tid-11645) Environment: kvm-debian12 (x2), Advanced Networking with Mgmt server d12 Total time taken: 52845 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9761-t11645-kvm-debian12.zip Smoke tests completed. 140 look OK, 1 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
ContextSuite context=TestClusterDRS>:setup	Error	0.00	test_cluster_drs.py

[blueorangutan]

[SF] Trillian test result (tid-11646) Environment: kvm-ubuntu24 (x2), Advanced Networking with Mgmt server u24 Total time taken: 52213 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9761-t11646-kvm-ubuntu24.zip Smoke tests completed. 141 look OK, 0 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File