Use Equinix Metal LB service to provision the Kubernetes API VIP

[displague]

User Story

As a [developer/user/operator] I would like to have CAPP provision an IP via Metal LBaaS to create a Kubernetes API IP that will balance traffic between the control plane nodes. This would be a simpler to manage process than the existing K8s controller managing via KubeVIP or Metal API calls that juggle a Metal EIP.

Detailed Description

TODO: [A clear and concise description of what you want to happen.]

Anything else you would like to add:

TODO: [Miscellaneous information that will assist in solving the issue.]

• https://deploy.equinix.com/developers/docs/metal/networking/load-balancers/

• /kind feature

[cprivitere]

Current Status:

• [x] Add "EMLB" VIPManager type
• [x] Create emlb based template that creates a cluster with an EMLB-based VIP
• [x] Add LBaaS SDK to codebase (since it's not publically available yet)
• [x] Add emlb internal package
• [x] Can create new load balancer
• [x] Can create new origins
• [x] Can create new pools
• [x] Can create new listener ports
• [x] Can add origins to pools
• [x] Can add pool to listener port
• [x] Creates new load balancer, pool, port, and origin with new cluster using EMLB VIPManager type.
• [x] Deletes load balancer, pool, port, origin when deleting cluster.
• [x] Test coverage for new functions
• [x] Update e2e test to check emlb template
• [x] Update packetmachine and packetcluster to do the same things for kube-vip as they do for cpem, as it needs those as well.
• [x] ~Update packetmachine to handle worker node load balancer annotations.~ Exclude worker nodes from EMLB pool deletion processing.
• [x] Update e2e cleanup to clean up old emlbs too.
• [x] Update README to have an example in using EMLB