uswitch / yggdrasil

Envoy Control Plane for Kubernetes Multi-cluster Ingress
Apache License 2.0
192 stars 17 forks source link

Ingress controllers under loadbalancer #49

Open Bujail opened 4 years ago

Bujail commented 4 years ago

Hi folks,

I was trying to test yggdrasil to achieve loadbalancing across two k8s clusters. Since yggdrasil using ingress controller's IP/Host name, I can't use my ELB here. Do we have any work around for this scenario?

Environment:

Cloud: AWS Clusters in East and West Nginx ingress ASG under internal Classic ELB. External DNS service will update Route53 from ingress rules.

envoy.yaml

`admin: access_log_path: /tmp/admin_access.log address: socket_address: { address: 0.0.0.0, port_value: 9901 }

dynamic_resources: lds_config: api_config_source: api_type: GRPC grpc_services: envoy_grpc: cluster_name: dev cds_config: api_config_source: api_type: GRPC grpc_services: envoy_grpc: cluster_name: dev

static_resources: clusters:

yggdrasil.conf

{ "nodeName": "k8s-envoy-agt-w2-1", "ingressClasses": ["nginx-internal"], "clusters": [ { "token": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", "apiServer": "https://west.dev.master.kube.com:6443", "ca": "ca.crt" } ] }

ingress.yaml

`apiVersion: extensions/v1beta1 kind: Ingress metadata: annotations: external-dns.alpha.kubernetes.io/hostname: envoy.dev.kube.com external-dns.alpha.kubernetes.io/target: internal-dev-k8s-ing-int-w2-xxxxxx.us-west-2.elb.amazonaws.com kubernetes.io/ingress.class: nginx-internal nginx.ingress.kubernetes.io/backend-protocol: HTTP yggdrasil.uswitch.com/healthcheck-path: / yggdrasil.uswitch.com/timeout: 30s name: hello-world namespace: default spec: rules: