Do `server-acl-init` and `inject-connect` support consul-server discovery from a k8s config in 1.x?

[nabadger]

Question

I've upgraded from https://artifacthub.io/packages/helm/hashicorp/consul/0.49.8 to https://artifacthub.io/packages/helm/hashicorp/consul/1.17 and noticed that server-acl-init no longer seems to support discovering consul-servers using the k8s provider.

Previously we were running consul-servers in 1 cluster, and consul-clients in another cluster (peered over vpc).

We were using the job to configure auth methods per client like so

 consul-k8s-control-plane server-acl-init  \
 -resource-prefix=consul-client-eu-qa1 -k8s-namespace=hybrid-consul \
 -auth-method-host=${EKS_CLUSTER_ENDPOINT}
 -server-address='provider=k8s kubeconfig=/consul/userconfig/consul-server-kubeconfig/kubeconfig
         label_selector="app=consul,component=server" namespace="hybrid-consul"'

However in the new version (which uses -addresses) the provider format here is not supported.

I was hoping it had moved to the new go-discover option since the changelog mentions this at https://github.com/hashicorp/consul-k8s/blob/main/CHANGELOG.md#100-november-17-2022

However server-acl-init seems to be using netaddrs.IPAddr directly, so not sure this will work (although looks like it supports exec=discover ... too)

Are you able to confirm any workarounds here?

I'm trying consul-k8s 1.1.7 but the code looks similar the latest release too.

edit Might the issue be that by default the k8s provider is not registered with discover, and calling netaddrs.IPAddr by-passes the util function newDiscover

[nabadger]

Furthermore, I'm finding that inject-connect deployment also does not seem to support the use of consol-server via kubeconfig (provider=k8s).

This seems to make use of discovery helpers that don't appear to inject the k8s provider support.

The second point here is that it would also require the new app containers which get the sidecars injected to have knowledge of this, and then volumemount the kubeconfig as well...

[nabadger]

I've also noticed that when the consul-dataplane sidecars come up, they run distroless so various custom exec options that one might use (at the point of defining inject-connect) are unlikely to be available.

I did an experiment where we discovered consul server IPs using an intermediate service I wrote...although this relied on wget which is not in the control-dataplane image.

Quite reluctant to build a custom container here, so not sure which direction to go in.

- name: CONSUL_ADDRESSES
  value: exec=wget -qO - http://get-consul-server-pod-ipds:8080

edit

I got around this using a better option, which is just to have external-dns publish consul-addresses to route53 - at which point I can just use netaddrs with DNS lookups.