Enhance `snapshot_additional_test.go` to allow any node role to be replaced, not just worker node role

[slickwarren]

    """
    - This test create 1 cluster with s3 backups enabled
        - 1 ControlPlane/worker node
        - 2 worker nodes
        - 3 etcd nodes
    - Creates an Ingress pointing to a workload
    - Snapshots the cluster and checks the backup is in S3
    - Stops the etcd nodes in ec2
    - Waits for the cluster to go into unavailable state
    - Removes all 3 etcd nodes
    - Creates 3 new etcd nodes and waits until the cluster
      asks to restore from backup.
    - Restores from S3 backup
    - Cluster is validated after it gets in Active state.
    - Checks the Ingress created before the backup is functional after restore.
    - Removes cluster if RANCHER_CLEANUP_CLUSTER=True
    """

this test would only work in AWS for go, because that's the only place we have the ability to directly talk to the nodes through a provider.

[markusewalker]

Here are the PRs that fix general flakiness seen in the snapshot tests:

Rancher PRs 2.9: https://github.com/rancher/rancher/pull/45668 2.8: https://github.com/rancher/rancher/pull/45858 2.7: https://github.com/rancher/rancher/pull/45859

Shepherd PRs 2.9: https://github.com/rancher/shepherd/pull/205 2.8: https://github.com/rancher/shepherd/pull/209 2.7: https://github.com/rancher/shepherd/pull/210

[markusewalker]

This PR is being split into two parts. The first will be to provide general stability fixes to the snapshot package. There were multiple complications with both found during the review process and the original PRs are quickly becoming bloated.

Current failures are specific to the snapshot_additional_tests.go, with flakiness experienced. More time is needed to consistently get passing results.

[markusewalker]

Initial PRs merged.

[markusewalker]

Rancher PRs: Main: https://github.com/rancher/rancher/pull/47072 2.9: https://github.com/rancher/rancher/pull/47093 2.8: https://github.com/rancher/rancher/pull/47094 2.7: https://github.com/rancher/rancher/pull/47095

[markusewalker]

Merged.