Use rancher/wharfie code for registries.yaml support

[brandond]

1. Remove code from k3s that has been moved to rancher/wharfie; use wharfie instead.
2. Add support for --image-credential-provider-(bin-dir|config) config as top-level k3s flags - enable kubelet featuregate if set and pass through to kubelet-args. These flags will be piggybacked upon by RKE2 to enable credential provider plugin for the bootstrap image pull.

Alternately, do we want to just have users pass --image-credential-provider-(bin-dir|config) and --featuregate=KubeletCredentialProviders=true as kubelet-args if they want to use this in K3s or RKE2? RKE2 could pick image-credential-provider-(bin-dir|config) out of kubelet-args and pass them in to the wharfie code if found, which would prevent needing another set of top-level CLI flags.

Final decision as per Slack conversation with @ibuildthecloud and @Oats87:

• try /var/lib/rancher/credentialprovider/bin and /var/lib/rancher/credentialprovider/config.yaml if they exist
• allow for changing those paths with config flags
• k3s, rke2, and rancher agent will all honor this behavior contract

[rancher-max]

I've validated standard airgap testing in v1.21.1-rc1+k3s1. This continues to work with tarball method, private registry in registries.yaml, and now also works with system-default-registry flag.

The image-credential-provider stuff on the kubelet is not working, even with the featuregate turned on. This appears to be an upstream issue, as using the same configurations with wharfie directly works. The error I'm seeing is a 401 Unauthorized error when trying to pull the images. Using config file:

kind: CredentialProviderConfig
apiVersion: kubelet.config.k8s.io/v1alpha1
providers:
  - name: ecr-credential-provider-amd64
    matchImages:
    - "*.dkr.ecr.*.amazonaws.com"
    - "*.dkr.ecr.*.amazonaws.cn"
    - "*.dkr.ecr-fips.*.amazonaws.com"
    - "*.dkr.ecr.us-iso-east-1.c2s.ic.gov"
    - "*.dkr.ecr.us-isob-east-1.sc2s.sgov.gov"
    defaultCacheDuration: "12h"
    apiVersion: credentialprovider.kubelet.k8s.io/v1alpha1
    args:
    - get-credentials

With that ecr-credential-provider-amd64 binary pulled from: https://github.com/rancher/wharfie/releases/tag/v0.3.5.

Bringing up k3s with flag: --system-default-registry=<account>.dkr.ecr.<region>.amazonaws.com where all the necessary k3s images are present in that registry.

[rancher-max]

Moved back to working to address the issues. Likely this is upstream and requires a fix there or will have to sit in our backlog for a bit.

[davidnuzik]

Bumped to 1.21.2 as more issues here need to be addressed.

[cjellick]

@cjellick to decide what to do with this one.

[brandond]

Tracking the remaining issue with kubelet credential provider support in #3463. The wharfie code works, and we are correctly enabling the feature-gate and setting the kubelet CLI args, but for some reason the kubelet is not using the plugins.