search

kumahq / kuma

🐻 The multi-zone service mesh for containers, Kubernetes and VMs. Built with Envoy. CNCF Sandbox Project.
https://kuma.io/install
Apache License 2.0
3.61k stars 332 forks source link

Kong ingress can't reach service when mTLS is enabled in Mesh in AWS EKS #6636

Closed adriano-brito-ts closed 1 year ago

adriano-brito-ts commented 1 year ago

What happened?

Hi I'm trying to deploy Kuma Service Mesh and Kong Ingress Controller as delegated Gateway in an EKS cluster.

Everything goes fine until I enable mTLS in the Mesh, I start seeing the following errors in kong proxy logs:

[error] 1198#0: *1015 upstream prematurely closed connection while reading response header from upstream

This only happens when I enable mTLS. (it works fine without mTLS and Permissive mode)

Kong is installed in a namespace with "kuma.io/sidecar-injection: enabled" and I confirmed it has the sidecar container running inside its pod, I can also see it in Kuma's Gui as a delegated gateway.

image image

I would appreciate any help! Thank you!

jakubdyszkiewicz commented 1 year ago

Triage: Hey, please send stats (+clusters) from Kong's DPP (you can execute it via GUI). Also please send a Kubernetes service definition of a service that you are trying to consume. Ingress (or Gateway API's equivalent) resources. Are you using any CNI? Our guess so far is that Kong's is using Pod IP instead of Cluster IP therefore Envoy does not know that it should use mTLS.

bartsmykla commented 1 year ago

@adriano-brito-ts could you send us information @jakubdyszkiewicz asked for below?

slonka commented 1 year ago

@adriano-brito-ts asking again for ☝️

michaelbeaumont commented 1 year ago

I'll close this. @adriano-brito-ts feel free to reply here with more information if you're running into this still.