kafka-controller dns lookup fails in istio ambient mode

[joke]

Describe the bug

The kafka-controller isn't working in istio ambient mode. The DNS entries can't be resolved properly.

level: error
ts: '2024-10-14T09:48:21.865Z'
logger: kafka-broker-controller
caller: 'prober/prober.go:105'
msg: Failed probe
commit: d641892-dirty
knative.dev/pod: kafka-controller-5fdcdfc9bf-vbtf8
scope: prober
port: ''
IP: bulk-updater-kn-channel.pass.svc.cluster.local
address: 'http://bulk-updater-kn-channel.pass.svc.cluster.local'
error: >-
  Get "http://bulk-updater-kn-channel.pass.svc.cluster.local": dial tcp: lookup
  bulk-updater-kn-channel.pass.svc.cluster.local on 172.20.0.10:53: no such host
stacktrace: "knative.dev/eventing-kafka-broker/control-plane/pkg/prober.probe\n\tknative.dev/eventing-kafka-broker/control-plane/pkg/prober/prober.go:105\nknative.dev/eventing-kafka-broker/control-plane/pkg/prober.(*asyncProber).probe.func2\n\tknative.dev/eventing-kafka-broker/control-plane/pkg/prober/async_prober.go:136"

The kafka-controller can't do the lookups on the channel services even though Istio Ambient DNS Capture is activated.

As far as I can tell this might be caused by missing service entries for the services with ExternalNames. Knative eventing does not create these service entries. But maybe it should based on this documentation.

Expected behavior

No error reports.

The System

To Reproduce

If the cluster as a working istio ambient setup just add these pod labels to the deployment:

istio.io/dataplane-mode: ambient
sidecar.istio.io/inject: 'false'

Knative release version

Knative: 1.15 Eventing Kafka Broker: 1.15.2

[pierDipi]

@joke are you using the eventing-istio controllers? https://knative.dev/docs/eventing/features/istio-integration/

we didn't update istio versions for a while in our testing https://github.com/knative-extensions/eventing-istio/blob/main/third_party/istio/manifest.yaml but we've not tried ambient mode either, is this ambient mode specific issue or generally 1.21+ istio doesn't work?

[joke]

@pierDipi I'm not using the eventing-istio controller. But I can try.

I looked at the source code and from what I gathered the controller only creates DestinationRules. According to the release notes DestinationRules can't be applied to ExternalName services without a ServiceEntry thus they shouldn't have any effect.

[pierDipi]

I read it the same, with 1.21 eventing-istio is not necessary anymore

[joke]

@pierDipi Some component would need to create the ServiceEntrys tough. Maybe eventing-istio should do that.