Using Bridge to Kubernetes with Private AKS Cluster

[richardpz]

We would like to start using Bridge to Kubernetes for solving our slow inner-loop problem and avoid the hassle of having to install all services and dependencies locally for testing and debugging.

We are currently using AKS in a hub-spoke network topology with AKS setup as a private cluster. https://docs.microsoft.com/en-us/azure/aks/private-clusters

Our dev workstations sit on the corporate LAN and can only access AKS services in the AKS virtual network via web requests routed through the hub firewalls or via a Linux SSH jump box.

It is not fully clear from the documentation what network routes need to be made available for it to work.

I would like to know if:

1. There are any firewall considerations we should be aware of?
2. What protocols and ports need to be open to enable communication between the Kubernetes cluster and Dev workstation?
3. Will Bridge to Kubernetes work with a private AKS cluster?

Thanks for any help in advance.

[rakeshvanga]

@richardpz Yes, Private AKS clusters are supported with Bridge to Kubernetes. To answer your questions:

• There are any firewall considerations we should be aware of? For Bridge to Kubernetes to work it pulls three images from Azure Container Registry onto your private aks cluster. You should whitelist ACR on your firewall or Linux SSH jump box. For example, if your firewall supports service tags you could whitelist ACR public endpoints using service tags. More information here: https://docs.microsoft.com/en-us/azure/virtual-network/service-tags-overview Also, the above link provides the way to get the public IP addresses for ACR per region to whitelist them in cases where service tags are not supported. Additionally, when routing is enabled (isolated scenarios), envoyproxy/envoy:v1.14.1 is also required, so this one image from dockerhub needs to be made available on the private cluster.
• What protocols and ports need to be open to enable communication between the Kubernetes cluster and Dev workstation? Bridge to Kubernetes uses port-forwarding to talk to the pods running on the cluster. So, the aks cluster's kube-api server should be able to talk to the dev box and vice versa. If you can issue kubectl commands from your dev box against the private aks clusters then Bridge to Kubernetes should also work properly.
• Will Bridge to Kubernetes work with a private AKS cluster? Yes. Bridge to Kubernetes respects all properties defined on the ingresses in your cluster to enable routing, so if you expose an ingress to a local network then routing should also be enabled locally to that network.

With respect to images, we are planning to make our images available on MCR including the envoy so that users can just whitelist MCR and be done with it. This work is in our backlog.

Please let me know if you face any issues and I would be happy to provide more information to unblock your scenarios.

[richardpz]

Thanks @rakeshvanga for taking the time to respond. Will come back with any further questions if needed.

[mark-vw]

With respect to images, we are planning to make our images available on MCR including the envoy so that users can just whitelist MCR and be done with it. This work is in our backlog.

@rakeshvanga, is there an ETA on moving to MCR? Current state makes the Bridge to Kubernetes a non-starter for teams in our company.