:rocket: Feature request: shutdown vm's with specific tags before cluster update

[ddemlow]

Is your feature request related to a problem? Please describe.

this is somewhat a corner case for now (in some cases testing potential future capabilities) - but I have clusters with specific VM's that can't live migrate to other nodes in the cluster causing cluster rolling updates to fail if they are running. examples include vm's running certain nested virtualization stacks (like nested hypercore nodes), with device passthrough like gpu or usb or could even be if strict affinity is set only allowing vm to run on specific node either due to licensing restrictions or where failover is not required or desired - for example a kubernetes node in multi-node kubernetes cluster

Describe the solution you'd like

some way to automate the shutdown of those VM's before rest of cluster update is initiated

Describe alternatives you've considered

add a task or role (either independent or into existing update roles) that loops through running VM's matching specified tag(or possibly tags?) - goes through shutdown process (initiate apci, wait timeout,force off if needed) - then continues

Additional context

really just looking for help / best practice for including this in existing update roles / examples or creating new example

[justinc1]

Likely we want to have same approach as in update single node cluster - shutdown VMs before upgrade, boot VMs after upgrade.

Maybe we could auto-detect the problematic VMs. We have 3 types: A. nested virtualization B. device passthrough (USB, GPU) C. strict node affinity D. VM does not need to be live migrate (like multinode kubernetes - HA is implemented on another layer)

A B C could be somehow autodetected (run virsh dumpxml and look at output). But D really depends only on what is inside VM, and maybe also on what other related VMs (other kubernetes nodes) are doing (if kubernetes cluster in not fully running, then maybe user would prefer to not upgrade).

I guess we cannot autodetect in usecases, so special tag on VMs will be needed. Tag name - maybe ansible_shutdown_during_upgrade=true/false.

User will then need to tag all relevant VMs. TODO - provide example how to assign only a tag to VM (vm_params module), and list there also VMs with specific tag (since it can be painful to check which VMs are already tagged).

For VM ACPI shutdown first, next force shutdown if needed - this needs to be reusable tasks files, we should already have it in version_update_single_node role.

If we implement this as another role, then we add to existing version_update_single_node some new role version_update_multi_node. Maybe we unify both roles into a single cluster_version_update (string version_update is already used for module name, reusing it for role name could be confusing).

[ddemlow]

the nested virt and passthrough cases currently would be "set" in hypercore using specific tags on the VM so either specifying the tags that indicate vm should be shut down (or adding extra "special" tag as you suggested) would work.

the strict node affinity is likely how you would indicate a case like D - or again could be done like tags. but strict affinity itself is available in REST API ... strict affinity is not commonly used - especially strict affinity with only a preferredNodeUUID because it will raise some strange errors (basically because that vm can't failover)

"affinityStrategy": {
  "strictAffinity": false,
  "preferredNodeUUID": "6c57dd62-3a4f-4ff5-b931-8cb538fcf4f3",
  "backupNodeUUID": ""