kumahq / kuma

🐻 The multi-zone service mesh for containers, Kubernetes and VMs. Built with Envoy. CNCF Sandbox Project.
https://kuma.io/install
Apache License 2.0
3.61k stars 332 forks source link

Failed Connection to Azure SQL from Zone CP running on AKS #5082

Closed ravish-kumar-maersk closed 1 year ago

ravish-kumar-maersk commented 1 year ago

Hi Team,

Am using Multi-zone deployment in AKS. One K8s is Global CP and the second one is Zone CP.

From Zone CP, am trying to connect to Azure SQL with connection string as "sqlserver://sqlservertest.database.windows.net"

In the deployment YAML, the exclude annotation is added - traffic.kuma.io/exclude-outbound-ports: "1433"

Even though, the connection is failing.

Am I missing some configuration ??

Deployment yaml and cm for db connection string --

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test-ipw-svc
  namespace: test-ui-ns
  labels:
    product: test
    app: test-ipw-svc
spec:
  replicas: 1
  selector:
    matchLabels:
      app: test-ipw-svc
  template:
    metadata:
      annotations:
        traffic.kuma.io/exclude-outbound-ports: "1433,443"
      labels:
        kuma.io/sidecar-injection: enabled
        app: test-ipw-svc
        product: test
    spec:
      imagePullSecrets:
        - name: acr-secret
      containers:
        - name: test-ipw-svc
          image: "__DOCKER_IMAGE_NAME__"
          imagePullPolicy: IfNotPresent
          envFrom:
          - configMapRef:
              name: test-ipw-svc-configmap
          ports:
            - containerPort: 3000
              protocol: TCP
          resources:
            requests:
              cpu: 100m
              memory: 256Mi
            limits:
              cpu: 150m
              memory: 512Mi
        - name: auth-sidecar
          image: "__AUTH_SIDECAR_IMAGE_NAME__"
          imagePullPolicy: IfNotPresent
          envFrom:
          - configMapRef:
              name: test-ipw-svc-configmap
          ports:
            - containerPort: 8086
              protocol: TCP
          resources:
            requests:
              cpu: 100m
              memory: 256Mi
            limits:
              cpu: 150m
              memory: 512Mi

apiVersion: v1
kind: ConfigMap
metadata:
  name: test-ipw-svc-configmap
  namespace: test-ui-ns
data:
  USER: "sqladmin"
  DATABASE: "test_DB"
  HOST: "sqlservertest.database.windows.net"
jakubdyszkiewicz commented 1 year ago

Triage: Does it work without sidecar injection? Could you provide Config Dump and Stats of the client Envoy? (use GUI or port-forward 9901 and execute curl localhost:9901/stats and localhost:9901/config_dump)

ravish-kumar-maersk commented 1 year ago

Triage: Does it work without sidecar injection?

Yes.. It's working without sidecar injection.

Could you provide Config Dump and Stats of the client Envoy? (use GUI or port-forward 9901 and execute curl localhost:9901/stats and localhost:9901/config_dump)

Attaching the config dump and stats.

config_dump.txt stats.txt

ravish-kumar-maersk commented 1 year ago

Hi @jakubdyszkiewicz

I can see a similar issue was solved at Istio - https://github.com/istio/istio/issues/12390

What should be the Kuma configuration for this issue?

ravish-kumar-maersk commented 1 year ago

Hi @jakubdyszkiewicz

I can see a similar issue was solved at Istio - istio/istio#12390

What should be the Kuma configuration for this issue?

Found a better solution at - https://github.com/istio/istio/issues/6587#issuecomment-454267919

michaelbeaumont commented 1 year ago

we have found the issue . Added the annotation with the below ports

traffic.kuma.io/exclude-outbound-ports: "1433,11000-11999"

Connection Policy: https://learn.microsoft.com/en-us/azure/azure-sql/database/connectivity-architecture?view=azuresql#connection-policy

I'll close this issue with the above solution