[BUG] Destination rule proxyProtocol in Ambient enabled environment

[sando38]

Is this the right place to submit this?

• [X] This is not a security vulnerability or a crashing bug
• [X] This is not a question about how to use Istio

Bug Description

Today I switch from sidecar mode to ambient. Actually it was a smooth transfer, however, one thing does not want to work. I run a XMPP server (TCP/TLS connections, no HTTP) behind an istio-ingressgateway and I use a destinationRule to send the proxyProtocol header upstream to the XMPP server:

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: ejabberd-xmpp
  namespace: ejabberd
spec:
  host: ejabberd.ejabberd.svc.cluster.local
  trafficPolicy:
    loadBalancer:
      simple: LEAST_REQUEST
    proxyProtocol:
      version: V2

The XMPP server complains, that the proxy protocol header cannot be parsed. When I removed the proxyProtocol section in the destinationRule, all runs fine (of course I do not see the actual client public IP address)

Version

~/istio-1.21.0$ ./bin/istioctl version
client version: 1.21.0
control plane version: 1.21.0
data plane version: 1.21.0 (2 proxies)

~/istio-1.21.0$ kubectl version
Client Version: v1.29.0
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.2+k3s1

~/istio-1.21.0$ helm version --short
v3.14.0+g3fc9f4b

Additional Information

bug-report.tar.gz

Just let me know, if I can help anywhere with further information!

[howardjohn]

Are you using a waypoint here or just ztunnel?

[sando38]

Just ztunnel.

[sando38]

Ah, I just remember something, which I read somewhere, maybe slack:

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: "no-mtls-for-tls-passthrough"
  namespace: ejabberd
spec:
  mtls:
    mode: STRICT
  selector:
    matchLabels:
      app.kubernetes.io/name: ejabberd
  portLevelMtls:
    "5222": # plain TCP with StartTLS
      mode: DISABLE
    "5223": # TLS passthrough - directTLS
      mode: DISABLE

With sidecar mode I needed to disable mTLS to make the proxyProtocol work. I think what I read was, that Ambient does not allow to disable mTLS, right?

[howardjohn]

Ztunnel does not read DestinationRule or support this configuration. Only AuthorizationPolicy and PeerAuthentication

[linsun]

Could you deploy a waypoint for your workload (or its namespace) to see if it works?

[sando38]

Thanks to both for the feedback. I was not aware, Likely, because not everything is yet documented, or I just have not seen it ;)

Sure, will deploy a waypoint and let you know. I suppose I find related documentation in istio.io?

[linsun]

btw, how did you "Today I switch from sidecar mode to ambient"? Did your ingress gateway HBONE enablement as well?

[linsun]

Sure, will deploy a waypoint and let you know. I suppose I find related documentation in istio.io?

Yes - the ambient get started doc shows you how. There is also an istioctl (x) waypoint cmd.

[sando38]

btw, how did you "Today I switch from sidecar mode to ambient"? Did your ingress gateway HBONE enablement as well?

Basically I checked this guide and saw that no special flag was used for the istio-ingressgateway, therefore, I did not change anything on my gateway config.

Here is what I did:

1. Installed istio-cni on k3s with --set profile=ambient (and CNI paths (https://github.com/istio/istio.io/pull/14826))
2. Re-deployed istiod with --set profile=ambient (I did/do not have any non-default configs running)
3. Deployed Ztunnel
4. Re-labeled all relevant namespaces from istio-injection: enabled to istio.io/dataplane-mode: ambient
5. Restarted all relevant pods.

[linsun]

Looking at your step, I think you'd need to restart your gateway (or deploy a new revision) to get HBONE enabled. Unless you created a new gateway deployment... @ilrudie @stevenctl thoughts?

[stevenctl]

https://github.com/istio/istio/blob/f21041a1bfeeded6b57aa509bcc2bc12ed81a103/pilot/pkg/networking/core/cluster_traffic_policy.go#L59-L62

I don't think this will work right now even with HBONE enabled.

[sando38]

okay, then I am waiting for your guys' go ;) Thanks nonetheless for the quick feedback.

[istio-policy-bot]

🧭 This issue or pull request has been automatically marked as stale because it has not had activity from an Istio team member since 2024-04-03. It will be closed on 2024-07-17 unless an Istio team member takes action. Please see this wiki page for more information. Thank you for your contributions.

Created by the issue and PR lifecycle manager.