:sparkles: Add Equinix Metal Load Balancer support

[cprivitere]

What this PR does / why we need it: This PR adds support to CAPP to create Equinix Metal Load Balancers and configure the cluster to use them for its API Server. It does not configure service load balancing and users are encouraged to configure CPEM in their clusters to enable that particular feature.

Fixes: #693

Current Status:

• [x] Add "EMLB" VIPManager type
• [x] Create emlb based template that creates a cluster with an EMLB-based VIP
• [x] Add LBaaS SDK to codebase (since it's not publically available yet)
• [x] Add emlb internal package
• [x] Can create new load balancer
• [x] Can create new origins
• [x] Can create new pools
• [x] Can create new listener ports
• [x] Can add origins to pools
• [x] Can add pool to listener port
• [x] Creates new load balancer, pool, port, and origin with new cluster using EMLB VIPManager type.

Edit: Splitting this into multiple PRs. Please see #693 for the full checklist of what must be done.

How to Run You can use the cluster-template-emlb.yaml template file to deploy a cluster with EMLB. When we do a new release, you can just use --flavor emlb to get that template. For now, it'll be more like:

clusterctl generate cluster quickstart --from go/src/sigs.k8s.io/cluster-api-provider-packet/templates/cluster-template-emlb.yaml > quickstart.yaml

[displague]

If CAPP is creating LBaaS instances (rather than just accessing them by a referenced/existing ID), then it is important for CAPP to cleanup after itself.

Deletion of controller-created resources is a common problem with multiple solutions. I've seen this implemented as labels, annotations, and spec. persistentVolumeReclaimPolicy and deletionPolicy come to mind.

The E2E tests should include a test that creates an LBaaS managed VIP for the control-plane. Upon deleting that cluster, a test should ensure that the loadbalancer was actually deleted.

A near term approach may be to include LBaaS cleanup to the existing post-test sweeper/cleanup code.

If we have existing warning text about indirect CAPP's cluster deletion, we may need to add to that warning that LBaaS instances may also be orphaned. https://docs.crossplane.io/latest/software/uninstall/ as an example.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cprivitere, ctreatma, displague

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/kubernetes-sigs/cluster-api-provider-packet/blob/main/OWNERS)~~ [cprivitere,displague] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

[displague]

This PR lacks any documentation on the new capabilities (ignoring the -emlb.yaml example). This is intentional to avoid usage until the remaining components of #693 can be incorporated.

[displague]

/lgtm