search

kubernetes-sigs / cloud-provider-equinix-metal

Kubernetes Cloud Provider for Equinix Metal (formerly Packet Cloud Controller Manager)
https://deploy.equinix.com/labs/cloud-provider-equinix-metal
Apache License 2.0
74 stars 26 forks source link

The CCM is no longer assigning IPs to services #148

Closed thebsdbox closed 3 years ago

thebsdbox commented 3 years ago

The only change to the default configuration is:

        env:
          - name: PACKET_LB
            value: "kube-vip://"
E0203 12:50:44.392227       1 cloud.go:258] failed to update and sync service for add default/nginx: elastic ip tag is empty. Nothing to do

regardless of running kube-vip or not the CCM is no longer doing an API call to apply an EIP to the service object.

deitch commented 3 years ago

This is for a service of type=LoadBalancer?

thebsdbox commented 3 years ago

Full v5 log -> E0203 12:50:44.392227 is an error for a service of type LoadBalancer.

root@k8s:~# kubectl logs -n kube-system   cloud-provider-equinix-metal-54c9fbb596-mj6pm -f
I0203 12:44:01.469229       1 main.go:235] authToken: '<masked>'
I0203 12:44:01.469317       1 main.go:235] projectID: 'f2a2d7ad-886e-4207-bf38-10ebdf49cf84'
I0203 12:44:01.469329       1 main.go:235] load balancer config: ''%!s(MISSING)
I0203 12:44:01.469341       1 main.go:235] metallb-system:config
I0203 12:44:01.469352       1 main.go:235] facility: 'ams1'
I0203 12:44:01.469362       1 main.go:235] peer ASN: '65530'
I0203 12:44:01.469377       1 main.go:235] local ASN: '65000'
I0203 12:44:01.469387       1 main.go:235] Elastic IP Tag: ''
I0203 12:44:01.469398       1 main.go:235] API Server Port: '6443'
I0203 12:44:01.469408       1 main.go:235] BGP Node Selector: ''
I0203 12:44:01.469456       1 plugins.go:61] Registered cloud provider "equinixmetal"
I0203 12:44:01.469667       1 flags.go:59] FLAG: --add-dir-header="false"
I0203 12:44:01.469697       1 flags.go:59] FLAG: --address="0.0.0.0"
I0203 12:44:01.469741       1 flags.go:59] FLAG: --allocate-node-cidrs="false"
I0203 12:44:01.469765       1 flags.go:59] FLAG: --allow-untagged-cloud="false"
I0203 12:44:01.469779       1 flags.go:59] FLAG: --alsologtostderr="false"
I0203 12:44:01.469793       1 flags.go:59] FLAG: --authentication-kubeconfig=""
I0203 12:44:01.469831       1 flags.go:59] FLAG: --authentication-skip-lookup="true"
I0203 12:44:01.469852       1 flags.go:59] FLAG: --authentication-token-webhook-cache-ttl="10s"
I0203 12:44:01.469870       1 flags.go:59] FLAG: --authentication-tolerate-lookup-failure="false"
I0203 12:44:01.469938       1 flags.go:59] FLAG: --authorization-always-allow-paths="[/healthz]"
I0203 12:44:01.470000       1 flags.go:59] FLAG: --authorization-kubeconfig=""
I0203 12:44:01.470015       1 flags.go:59] FLAG: --authorization-webhook-cache-authorized-ttl="10s"
I0203 12:44:01.470040       1 flags.go:59] FLAG: --authorization-webhook-cache-unauthorized-ttl="10s"
I0203 12:44:01.470055       1 flags.go:59] FLAG: --bind-address="0.0.0.0"
I0203 12:44:01.470095       1 flags.go:59] FLAG: --cert-dir=""
I0203 12:44:01.470110       1 flags.go:59] FLAG: --cidr-allocator-type="RangeAllocator"
I0203 12:44:01.470124       1 flags.go:59] FLAG: --client-ca-file=""
I0203 12:44:01.470138       1 flags.go:59] FLAG: --cloud-config=""
I0203 12:44:01.470174       1 flags.go:59] FLAG: --cloud-provider="equinixmetal"
I0203 12:44:01.470189       1 flags.go:59] FLAG: --cluster-cidr=""
I0203 12:44:01.470202       1 flags.go:59] FLAG: --cluster-name="kubernetes"
I0203 12:44:01.470216       1 flags.go:59] FLAG: --concurrent-service-syncs="1"
I0203 12:44:01.470276       1 flags.go:59] FLAG: --configure-cloud-routes="true"
I0203 12:44:01.470292       1 flags.go:59] FLAG: --contention-profiling="false"
I0203 12:44:01.470307       1 flags.go:59] FLAG: --controller-start-interval="0s"
I0203 12:44:01.470344       1 flags.go:59] FLAG: --controllers="[*]"
I0203 12:44:01.470361       1 flags.go:59] FLAG: --external-cloud-volume-plugin=""
I0203 12:44:01.470375       1 flags.go:59] FLAG: --feature-gates=""
I0203 12:44:01.470413       1 flags.go:59] FLAG: --help="false"
I0203 12:44:01.470428       1 flags.go:59] FLAG: --http2-max-streams-per-connection="0"
I0203 12:44:01.470447       1 flags.go:59] FLAG: --kube-api-burst="30"
I0203 12:44:01.470461       1 flags.go:59] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
I0203 12:44:01.470494       1 flags.go:59] FLAG: --kube-api-qps="20"
I0203 12:44:01.470517       1 flags.go:59] FLAG: --kubeconfig=""
I0203 12:44:01.470532       1 flags.go:59] FLAG: --leader-elect="false"
I0203 12:44:01.470546       1 flags.go:59] FLAG: --leader-elect-lease-duration="15s"
I0203 12:44:01.470596       1 flags.go:59] FLAG: --leader-elect-renew-deadline="10s"
I0203 12:44:01.470619       1 flags.go:59] FLAG: --leader-elect-resource-lock="endpointsleases"
I0203 12:44:01.470640       1 flags.go:59] FLAG: --leader-elect-resource-name="cloud-controller-manager"
I0203 12:44:01.470691       1 flags.go:59] FLAG: --leader-elect-resource-namespace="kube-system"
I0203 12:44:01.470712       1 flags.go:59] FLAG: --leader-elect-retry-period="2s"
I0203 12:44:01.470761       1 flags.go:59] FLAG: --log-backtrace-at=":0"
I0203 12:44:01.470793       1 flags.go:59] FLAG: --log-dir=""
I0203 12:44:01.470840       1 flags.go:59] FLAG: --log-file=""
I0203 12:44:01.470863       1 flags.go:59] FLAG: --log-file-max-size="1800"
I0203 12:44:01.470893       1 flags.go:59] FLAG: --log-flush-frequency="5s"
I0203 12:44:01.470940       1 flags.go:59] FLAG: --logtostderr="true"
I0203 12:44:01.470961       1 flags.go:59] FLAG: --master=""
I0203 12:44:01.470982       1 flags.go:59] FLAG: --min-resync-period="12h0m0s"
I0203 12:44:01.471032       1 flags.go:59] FLAG: --node-monitor-period="5s"
I0203 12:44:01.471053       1 flags.go:59] FLAG: --node-status-update-frequency="5m0s"
I0203 12:44:01.471099       1 flags.go:59] FLAG: --node-sync-period="0s"
I0203 12:44:01.471123       1 flags.go:59] FLAG: --permit-port-sharing="false"
I0203 12:44:01.471144       1 flags.go:59] FLAG: --port="0"
I0203 12:44:01.471188       1 flags.go:59] FLAG: --profiling="true"
I0203 12:44:01.471212       1 flags.go:59] FLAG: --provider-config="/etc/cloud-sa/cloud-sa.json"
I0203 12:44:01.471234       1 flags.go:59] FLAG: --requestheader-allowed-names="[]"
I0203 12:44:01.471304       1 flags.go:59] FLAG: --requestheader-client-ca-file=""
I0203 12:44:01.471325       1 flags.go:59] FLAG: --requestheader-extra-headers-prefix="[x-remote-extra-]"
I0203 12:44:01.471377       1 flags.go:59] FLAG: --requestheader-group-headers="[x-remote-group]"
I0203 12:44:01.471419       1 flags.go:59] FLAG: --requestheader-username-headers="[x-remote-user]"
I0203 12:44:01.471471       1 flags.go:59] FLAG: --route-reconciliation-period="10s"
I0203 12:44:01.471493       1 flags.go:59] FLAG: --secure-port="10258"
I0203 12:44:01.471539       1 flags.go:59] FLAG: --skip-headers="false"
I0203 12:44:01.471562       1 flags.go:59] FLAG: --skip-log-headers="false"
I0203 12:44:01.471581       1 flags.go:59] FLAG: --stderrthreshold="2"
I0203 12:44:01.471626       1 flags.go:59] FLAG: --tls-cert-file=""
I0203 12:44:01.471649       1 flags.go:59] FLAG: --tls-cipher-suites="[]"
I0203 12:44:01.471716       1 flags.go:59] FLAG: --tls-min-version=""
I0203 12:44:01.471737       1 flags.go:59] FLAG: --tls-private-key-file=""
I0203 12:44:01.471756       1 flags.go:59] FLAG: --tls-sni-cert-key="[]"
I0203 12:44:01.471806       1 flags.go:59] FLAG: --use-service-account-credentials="false"
I0203 12:44:01.471827       1 flags.go:59] FLAG: --v="5"
I0203 12:44:01.471874       1 flags.go:59] FLAG: --version="false"
I0203 12:44:01.471904       1 flags.go:59] FLAG: --vmodule=""
I0203 12:44:04.573193       1 serving.go:331] Generated self-signed cert in-memory
W0203 12:44:05.518620       1 client_config.go:608] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0203 12:44:05.528483       1 controllermanager.go:127] Version: v0.0.0-master+$Format:%h$
I0203 12:44:05.529247       1 healthz.go:158] No default health checks specified. Installing the ping handler.
I0203 12:44:05.529273       1 healthz.go:162] Installing health checkers for (/healthz): "ping"
I0203 12:44:05.533505       1 tlsconfig.go:200] loaded serving cert ["Generated self signed cert"]: "localhost@1612356244" [serving] validServingFor=[127.0.0.1,localhost,localhost] issuer="localhost-ca@1612356242" (2021-02-03 11:44:01 +0000 UTC to 2022-02-03 11:44:01 +0000 UTC (now=2021-02-03 12:44:05.533459092 +0000 UTC))
I0203 12:44:05.534194       1 named_certificates.go:53] loaded SNI cert [0/"self-signed loopback"]: "apiserver-loopback-client@1612356245" [serving] validServingFor=[apiserver-loopback-client] issuer="apiserver-loopback-client-ca@1612356244" (2021-02-03 11:44:04 +0000 UTC to 2022-02-03 11:44:04 +0000 UTC (now=2021-02-03 12:44:05.534164226 +0000 UTC))
I0203 12:44:05.534263       1 secure_serving.go:197] Serving securely on [::]:10258
I0203 12:44:05.534357       1 tlsconfig.go:240] Starting DynamicServingCertificateController
2021/02/03 12:44:05 [DEBUG] GET https://api.equinix.com/metal/v1/projects/f2a2d7ad-886e-4207-bf38-10ebdf49cf84/bgp-config?
I0203 12:44:06.095426       1 reflector.go:207] Starting reflector *v1.Node (0s) from pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156
I0203 12:44:06.095478       1 reflector.go:243] Listing and watching *v1.Node from pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156
E0203 12:44:06.102442       1 cloud.go:202] failed to update and sync node for add k8s.bgp01 for handler: no provider ID given
E0203 12:44:06.102479       1 cloud.go:202] failed to update and sync node for add k8s.bgp01 for handler: control plane loadbalancer elastic ip tag is empty. Nothing to do
I0203 12:44:06.195404       1 shared_informer.go:270] caches populated
I0203 12:44:06.195540       1 cloud.go:238] nodes watcher started
I0203 12:44:06.195736       1 reflector.go:207] Starting reflector *v1.Service (0s) from pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156
I0203 12:44:06.195768       1 reflector.go:243] Listing and watching *v1.Service from pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156
E0203 12:44:06.199768       1 cloud.go:258] failed to update and sync service for add default/kubernetes: elastic ip tag is empty. Nothing to do
E0203 12:44:06.199817       1 cloud.go:258] failed to update and sync service for add kube-system/kube-dns: elastic ip tag is empty. Nothing to do
I0203 12:44:06.295871       1 shared_informer.go:270] caches populated
I0203 12:44:06.295942       1 cloud.go:293] services watcher started
I0203 12:44:06.296014       1 controllermanager.go:244] Starting "cloud-node"
I0203 12:44:06.300112       1 node_controller.go:108] Sending events to api server.
W0203 12:44:06.300206       1 cloud.go:152] The Equinix Metal cloud provider does not support InstancesV2
I0203 12:44:06.300290       1 controllermanager.go:254] Started "cloud-node"
I0203 12:44:06.300328       1 controllermanager.go:244] Starting "cloud-node-lifecycle"
I0203 12:44:06.304099       1 node_lifecycle_controller.go:77] Sending events to api server
W0203 12:44:06.304208       1 cloud.go:152] The Equinix Metal cloud provider does not support InstancesV2
I0203 12:44:06.304248       1 controllermanager.go:254] Started "cloud-node-lifecycle"
I0203 12:44:06.304287       1 controllermanager.go:244] Starting "service"
I0203 12:44:06.304819       1 node_controller.go:224] This node k8s.bgp01 is still tainted. Will not process.
E0203 12:44:06.308452       1 core.go:97] Failed to start service controller: the cloud provider does not support external load balancers
W0203 12:44:06.308498       1 controllermanager.go:251] Skipping "service"
I0203 12:44:06.308529       1 controllermanager.go:244] Starting "route"
I0203 12:44:06.308552       1 core.go:108] Will not configure cloud provider routes for allocate-node-cidrs: false, configure-cloud-routes: true.
W0203 12:44:06.308579       1 controllermanager.go:251] Skipping "route"
I0203 12:44:06.312908       1 reflector.go:207] Starting reflector *v1.Service (30s) from pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156
I0203 12:44:06.312947       1 reflector.go:243] Listing and watching *v1.Service from pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156
I0203 12:44:06.312908       1 reflector.go:207] Starting reflector *v1.Node (22h36m21.362184607s) from pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156
I0203 12:44:06.312981       1 reflector.go:243] Listing and watching *v1.Node from pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156
I0203 12:44:06.316953       1 node_controller.go:329] Initializing node k8s.bgp01 with cloud provider
W0203 12:44:06.321084       1 cloud.go:152] The Equinix Metal cloud provider does not support InstancesV2
2021/02/03 12:44:06 [DEBUG] GET https://api.equinix.com/metal/v1/projects/f2a2d7ad-886e-4207-bf38-10ebdf49cf84/devices?include=facility
2021/02/03 12:44:08 [DEBUG] GET https://api.equinix.com/metal/v1/projects/f2a2d7ad-886e-4207-bf38-10ebdf49cf84/devices?page=2&include=facility
W0203 12:44:10.034319       1 cloud.go:152] The Equinix Metal cloud provider does not support InstancesV2
2021/02/03 12:44:10 [DEBUG] GET https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501?include=facility
2021/02/03 12:44:10 [DEBUG] GET https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501?include=facility
I0203 12:44:10.700568       1 node_controller.go:483] Adding node label from cloud provider: beta.kubernetes.io/instance-type=t1.small.x86
I0203 12:44:10.700617       1 node_controller.go:484] Adding node label from cloud provider: node.kubernetes.io/instance-type=t1.small.x86
2021/02/03 12:44:10 [DEBUG] GET https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501?include=facility
I0203 12:44:10.995163       1 node_controller.go:511] Adding node label from cloud provider: failure-domain.beta.kubernetes.io/region=ams1
I0203 12:44:10.995218       1 node_controller.go:512] Adding node label from cloud provider: topology.kubernetes.io/region=ams1
W0203 12:44:11.027366       1 cloud.go:152] The Equinix Metal cloud provider does not support InstancesV2
2021/02/03 12:44:11 [DEBUG] GET https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501?include=facility
I0203 12:44:11.345611       1 node_controller.go:397] Successfully initialized node k8s.bgp01 with cloud provider
I0203 12:44:36.315441       1 reflector.go:369] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: forcing resync
E0203 12:45:06.296396       1 cloud.go:310] failed to update and sync services: elastic ip tag is empty. Nothing to do
2021/02/03 12:45:06 [DEBUG] POST https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501/bgp/sessions
I0203 12:45:06.315724       1 reflector.go:369] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: forcing resync
2021/02/03 12:45:06 [DEBUG] GET https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501/bgp/neighbors?
E0203 12:45:07.186169       1 cloud.go:319] failed to update and sync nodes: control plane loadbalancer elastic ip tag is empty. Nothing to do
I0203 12:45:36.316029       1 reflector.go:369] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: forcing resync
I0203 12:46:06.316357       1 reflector.go:369] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: forcing resync
E0203 12:46:07.186418       1 cloud.go:310] failed to update and sync services: elastic ip tag is empty. Nothing to do
2021/02/03 12:46:07 [DEBUG] POST https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501/bgp/sessions
2021/02/03 12:46:07 [DEBUG] GET https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501/bgp/neighbors?
E0203 12:46:08.054288       1 cloud.go:319] failed to update and sync nodes: control plane loadbalancer elastic ip tag is empty. Nothing to do
I0203 12:46:36.316606       1 reflector.go:369] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: forcing resync
I0203 12:47:06.316894       1 reflector.go:369] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: forcing resync
E0203 12:47:08.054520       1 cloud.go:310] failed to update and sync services: elastic ip tag is empty. Nothing to do
2021/02/03 12:47:08 [DEBUG] POST https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501/bgp/sessions
2021/02/03 12:47:08 [DEBUG] GET https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501/bgp/neighbors?
E0203 12:47:08.654764       1 cloud.go:319] failed to update and sync nodes: control plane loadbalancer elastic ip tag is empty. Nothing to do
I0203 12:47:36.317182       1 reflector.go:369] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: forcing resync
I0203 12:48:06.317441       1 reflector.go:369] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: forcing resync
E0203 12:48:08.655095       1 cloud.go:310] failed to update and sync services: elastic ip tag is empty. Nothing to do
2021/02/03 12:48:08 [DEBUG] POST https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501/bgp/sessions
2021/02/03 12:48:09 [DEBUG] GET https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501/bgp/neighbors?
E0203 12:48:09.396419       1 cloud.go:319] failed to update and sync nodes: control plane loadbalancer elastic ip tag is empty. Nothing to do
E0203 12:48:26.421453       1 cloud.go:202] failed to update and sync node for add k8s.bgp02 for handler: no provider ID given
E0203 12:48:26.421490       1 cloud.go:202] failed to update and sync node for add k8s.bgp02 for handler: control plane loadbalancer elastic ip tag is empty. Nothing to do
I0203 12:48:26.422026       1 controller.go:659] Ignoring node k8s.bgp02 with Ready condition status False
I0203 12:48:26.422097       1 node_controller.go:329] Initializing node k8s.bgp02 with cloud provider
W0203 12:48:26.461505       1 cloud.go:152] The Equinix Metal cloud provider does not support InstancesV2
2021/02/03 12:48:26 [DEBUG] GET https://api.equinix.com/metal/v1/projects/f2a2d7ad-886e-4207-bf38-10ebdf49cf84/devices?include=facility
2021/02/03 12:48:27 [DEBUG] GET https://api.equinix.com/metal/v1/projects/f2a2d7ad-886e-4207-bf38-10ebdf49cf84/devices?page=2&include=facility
W0203 12:48:28.442416       1 cloud.go:152] The Equinix Metal cloud provider does not support InstancesV2
2021/02/03 12:48:28 [DEBUG] GET https://api.equinix.com/metal/v1/devices/be6900e6-a28c-4da7-866c-cbca22f773e7?include=facility
E0203 12:48:28.606947       1 cloud.go:202] failed to update and sync node for add k8s.bgp03 for handler: no provider ID given
E0203 12:48:28.609988       1 cloud.go:202] failed to update and sync node for add k8s.bgp03 for handler: control plane loadbalancer elastic ip tag is empty. Nothing to do
I0203 12:48:28.609023       1 controller.go:659] Ignoring node k8s.bgp02 with Ready condition status False
I0203 12:48:28.610218       1 controller.go:659] Ignoring node k8s.bgp03 with Ready condition status False
2021/02/03 12:48:28 [DEBUG] GET https://api.equinix.com/metal/v1/devices/be6900e6-a28c-4da7-866c-cbca22f773e7?include=facility
I0203 12:48:29.119980       1 node_controller.go:483] Adding node label from cloud provider: beta.kubernetes.io/instance-type=t1.small.x86
I0203 12:48:29.120026       1 node_controller.go:484] Adding node label from cloud provider: node.kubernetes.io/instance-type=t1.small.x86
2021/02/03 12:48:29 [DEBUG] GET https://api.equinix.com/metal/v1/devices/be6900e6-a28c-4da7-866c-cbca22f773e7?include=facility
I0203 12:48:29.416263       1 node_controller.go:511] Adding node label from cloud provider: failure-domain.beta.kubernetes.io/region=ams1
I0203 12:48:29.416303       1 node_controller.go:512] Adding node label from cloud provider: topology.kubernetes.io/region=ams1
W0203 12:48:29.434664       1 cloud.go:152] The Equinix Metal cloud provider does not support InstancesV2
2021/02/03 12:48:29 [DEBUG] GET https://api.equinix.com/metal/v1/devices/be6900e6-a28c-4da7-866c-cbca22f773e7?include=facility
I0203 12:48:29.446892       1 controller.go:659] Ignoring node k8s.bgp03 with Ready condition status False
I0203 12:48:29.446930       1 controller.go:659] Ignoring node k8s.bgp02 with Ready condition status False
I0203 12:48:29.760912       1 node_controller.go:397] Successfully initialized node k8s.bgp02 with cloud provider
I0203 12:48:29.760961       1 node_controller.go:329] Initializing node k8s.bgp02 with cloud provider
I0203 12:48:29.767329       1 node_controller.go:329] Initializing node k8s.bgp02 with cloud provider
I0203 12:48:29.771344       1 node_controller.go:320] This node k8s.bgp03 is registered without the cloud taint. Will not process.
W0203 12:48:31.314699       1 cloud.go:152] The Equinix Metal cloud provider does not support InstancesV2
2021/02/03 12:48:31 [DEBUG] GET https://api.equinix.com/metal/v1/devices/be6900e6-a28c-4da7-866c-cbca22f773e7?include=facility
W0203 12:48:31.628921       1 cloud.go:152] The Equinix Metal cloud provider does not support InstancesV2
2021/02/03 12:48:31 [DEBUG] GET https://api.equinix.com/metal/v1/devices/be6900e6-a28c-4da7-866c-cbca22f773e7?include=facility
W0203 12:48:31.933152       1 cloud.go:152] The Equinix Metal cloud provider does not support InstancesV2
2021/02/03 12:48:31 [DEBUG] GET https://api.equinix.com/metal/v1/projects/f2a2d7ad-886e-4207-bf38-10ebdf49cf84/devices?include=facility
2021/02/03 12:48:32 [DEBUG] GET https://api.equinix.com/metal/v1/projects/f2a2d7ad-886e-4207-bf38-10ebdf49cf84/devices?page=2&include=facility
2021/02/03 12:48:33 [DEBUG] GET https://api.equinix.com/metal/v1/devices/259f8392-21f0-4724-9625-59802a81cb12?include=facility
W0203 12:48:34.124579       1 cloud.go:152] The Equinix Metal cloud provider does not support InstancesV2
E0203 12:48:34.124653       1 node_lifecycle_controller.go:178] error checking if node k8s.bgp03 is shutdown: providerID cannot be empty string
I0203 12:48:36.317739       1 reflector.go:369] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: forcing resync
I0203 12:48:36.501494       1 controller.go:659] Ignoring node k8s.bgp03 with Ready condition status False
I0203 12:48:36.501555       1 controller.go:708] Detected change in list of current cluster nodes. New node set: map[k8s.bgp02:{}]
I0203 12:48:36.505200       1 controller.go:716] Successfully updated 0 out of 0 load balancers to direct traffic to the updated set of nodes
I0203 12:48:38.685425       1 controller.go:708] Detected change in list of current cluster nodes. New node set: map[k8s.bgp02:{} k8s.bgp03:{}]
I0203 12:48:38.685479       1 controller.go:716] Successfully updated 0 out of 0 load balancers to direct traffic to the updated set of nodes
W0203 12:49:06.312466       1 cloud.go:152] The Equinix Metal cloud provider does not support InstancesV2
2021/02/03 12:49:06 [DEBUG] GET https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501?include=facility
I0203 12:49:06.318049       1 reflector.go:369] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: forcing resync
W0203 12:49:06.745416       1 cloud.go:152] The Equinix Metal cloud provider does not support InstancesV2
2021/02/03 12:49:06 [DEBUG] GET https://api.equinix.com/metal/v1/devices/be6900e6-a28c-4da7-866c-cbca22f773e7?include=facility
W0203 12:49:07.034741       1 cloud.go:152] The Equinix Metal cloud provider does not support InstancesV2
2021/02/03 12:49:07 [DEBUG] GET https://api.equinix.com/metal/v1/projects/f2a2d7ad-886e-4207-bf38-10ebdf49cf84/devices?include=facility
2021/02/03 12:49:07 [DEBUG] GET https://api.equinix.com/metal/v1/projects/f2a2d7ad-886e-4207-bf38-10ebdf49cf84/devices?page=2&include=facility
E0203 12:49:09.396674       1 cloud.go:310] failed to update and sync services: elastic ip tag is empty. Nothing to do
2021/02/03 12:49:09 [DEBUG] POST https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501/bgp/sessions
2021/02/03 12:49:09 [DEBUG] GET https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501/bgp/neighbors?
2021/02/03 12:49:10 [DEBUG] POST https://api.equinix.com/metal/v1/devices/be6900e6-a28c-4da7-866c-cbca22f773e7/bgp/sessions
2021/02/03 12:49:10 [DEBUG] GET https://api.equinix.com/metal/v1/devices/be6900e6-a28c-4da7-866c-cbca22f773e7/bgp/neighbors?
E0203 12:49:10.638851       1 cloud.go:319] failed to update and sync nodes: no provider ID given
E0203 12:49:10.638997       1 cloud.go:319] failed to update and sync nodes: control plane loadbalancer elastic ip tag is empty. Nothing to do
I0203 12:49:23.317174       1 reflector.go:515] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: Watch close - *v1.Node total 37 items received
I0203 12:49:36.318380       1 reflector.go:369] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: forcing resync
I0203 12:50:06.318728       1 reflector.go:369] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: forcing resync
E0203 12:50:10.639184       1 cloud.go:310] failed to update and sync services: elastic ip tag is empty. Nothing to do
2021/02/03 12:50:10 [DEBUG] POST https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501/bgp/sessions
2021/02/03 12:50:10 [DEBUG] GET https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501/bgp/neighbors?
2021/02/03 12:50:11 [DEBUG] POST https://api.equinix.com/metal/v1/devices/be6900e6-a28c-4da7-866c-cbca22f773e7/bgp/sessions
2021/02/03 12:50:11 [DEBUG] GET https://api.equinix.com/metal/v1/devices/be6900e6-a28c-4da7-866c-cbca22f773e7/bgp/neighbors?
E0203 12:50:11.742316       1 cloud.go:319] failed to update and sync nodes: no provider ID given
E0203 12:50:11.742360       1 cloud.go:319] failed to update and sync nodes: control plane loadbalancer elastic ip tag is empty. Nothing to do
I0203 12:50:16.201971       1 reflector.go:515] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: Watch close - *v1.Service total 0 items received
I0203 12:50:36.319004       1 reflector.go:369] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: forcing resync
I0203 12:50:37.104356       1 reflector.go:515] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: Watch close - *v1.Node total 38 items received
E0203 12:50:44.392227       1 cloud.go:258] failed to update and sync service for add default/nginx: elastic ip tag is empty. Nothing to do
I0203 12:51:06.319287       1 reflector.go:369] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: forcing resync
E0203 12:51:11.742601       1 cloud.go:310] failed to update and sync services: elastic ip tag is empty. Nothing to do
2021/02/03 12:51:11 [DEBUG] POST https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501/bgp/sessions
2021/02/03 12:51:12 [DEBUG] GET https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501/bgp/neighbors?
2021/02/03 12:51:12 [DEBUG] POST https://api.equinix.com/metal/v1/devices/be6900e6-a28c-4da7-866c-cbca22f773e7/bgp/sessions
2021/02/03 12:51:12 [DEBUG] GET https://api.equinix.com/metal/v1/devices/be6900e6-a28c-4da7-866c-cbca22f773e7/bgp/neighbors?
E0203 12:51:12.914516       1 cloud.go:319] failed to update and sync nodes: no provider ID given
E0203 12:51:12.914574       1 cloud.go:319] failed to update and sync nodes: control plane loadbalancer elastic ip tag is empty. Nothing to do
I0203 12:51:36.319608       1 reflector.go:369] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: forcing resync
I0203 12:52:06.319881       1 reflector.go:369] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: forcing resync

E0203 12:52:12.914806       1 cloud.go:310] failed to update and sync services: elastic ip tag is empty. Nothing to do
2021/02/03 12:52:12 [DEBUG] POST https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501/bgp/sessions
2021/02/03 12:52:13 [DEBUG] GET https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501/bgp/neighbors?
2021/02/03 12:52:13 [DEBUG] POST https://api.equinix.com/metal/v1/devices/be6900e6-a28c-4da7-866c-cbca22f773e7/bgp/sessions
2021/02/03 12:52:13 [DEBUG] GET https://api.equinix.com/metal/v1/devices/be6900e6-a28c-4da7-866c-cbca22f773e7/bgp/neighbors?
E0203 12:52:13.958707       1 cloud.go:319] failed to update and sync nodes: no provider ID given
E0203 12:52:13.958742       1 cloud.go:319] failed to update and sync nodes: control plane loadbalancer elastic ip tag is empty. Nothing to do
I0203 12:52:36.320176       1 reflector.go:369] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: forcing resync
I0203 12:53:06.320473       1 reflector.go:369] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: forcing resync
E0203 12:53:13.958920       1 cloud.go:310] failed to update and sync services: elastic ip tag is empty. Nothing to do
2021/02/03 12:53:13 [DEBUG] POST https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501/bgp/sessions
2021/02/03 12:53:14 [DEBUG] GET https://api.equinix.com/metal/v1/devices/16ac7d50-798f-41f4-ba0f-2d3c992c9501/bgp/neighbors?
2021/02/03 12:53:14 [DEBUG] POST https://api.equinix.com/metal/v1/devices/be6900e6-a28c-4da7-866c-cbca22f773e7/bgp/sessions
2021/02/03 12:53:14 [DEBUG] GET https://api.equinix.com/metal/v1/devices/be6900e6-a28c-4da7-866c-cbca22f773e7/bgp/neighbors?
E0203 12:53:14.731460       1 cloud.go:319] failed to update and sync nodes: no provider ID given
E0203 12:53:14.731511       1 cloud.go:319] failed to update and sync nodes: control plane loadbalancer elastic ip tag is empty. Nothing to do
I0203 12:53:26.316603       1 reflector.go:515] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: Watch close - *v1.Service total 6 items received
I0203 12:53:36.320703       1 reflector.go:369] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: forcing resync
deitch commented 3 years ago

How is this configured?

thebsdbox commented 3 years ago

As mentioned above.. from the deployment.yaml for the latest release and with the environment variable for the loadbalancer added. kube-vip://