search

Azure / AKS

Azure Kubernetes Service
https://azure.github.io/AKS/
1.95k stars 304 forks source link

[Feature] Hubble in Azure CNI Powered by Cilium #3978

Open illrill opened 10 months ago

illrill commented 10 months ago

I'd like to see support for Hubble (hubble-relay and hubble-ui) in Azure CNI Powered by Cilium

siegenthalerroger commented 9 months ago

Our team is also looking to have hubble enabled in Azure CNI Powered by Cilium. Less interested in hubble-ui tbh, as that is deployable in standalone mode and could easily be left to the user.

Being able to configure Cilium the same way e.g. CoreDNS is configured would be great, allow us to enable Hubble and Service Mesh as we see fit.

mac-kul commented 8 months ago

is there any other good solution that could help us with network observability? On the docs we can see the benefit:

How this can be achieved? I would like to be able to monitor the traffic inside the cluster.

PixelRobots commented 8 months ago

is there any other good solution that could help us with network observability? On the docs we can see the benefit:

  • Better observability of cluster traffic

How this can be achieved? I would like to be able to monitor the traffic inside the cluster.

Hi @mac-kul

Check out this doc's page. https://learn.microsoft.com/en-us/azure/aks/network-observability-managed-cli?tabs=cilium.

It is currently in preview but could be a good starting point for you to look at. It also supports non cilium clusters.

siegenthalerroger commented 8 months ago

@PixelRobots from what I understand, Network Observability only provides metrics about aggregate traffic (bandwidth, latency, etc). There's no capability for a "Flow" type view, where you can see/analyse individual requests/packets right?

PixelRobots commented 8 months ago

@PixelRobots from what I understand, Network Observability only provides metrics about aggregate traffic (bandwidth, latency, etc). There's no capability for a "Flow" type view, where you can see/analyse individual requests/packets right?

According to the docs it currently only supports node level metrics. So it suggests that pod level metrics will hopefully come soon.

In the meantime it could also be good to look at https://www.inspektor-gadget.io/. To fill the gap when troubleshooting.

illrill commented 6 months ago

Being able to configure Cilium the same way e.g. CoreDNS is configured would be great, allow us to enable Hubble and Service Mesh as we see fit.

I like this idea. By providing a way for users to override the kube-system/cilium-config CM, a lot of other Cilium-related issues (e.g. #3450 #3797) could also be closed, since we would have the ability to toggle these things on ourselves.

One approach could be to use CiliumNodeConfig as the config override mechanism. AKS team would just need to install the CRD. We could use the CR with a nodeSelector that selects all nodes.

JitseHijlkema commented 5 months ago

Just announced on KubeCon 2024 in Paris, which will gives us the Hubble functionality:

Advanced Networking Observability which includes Hubble (CLI+UI) and with Retina (made OpenSource) also supports other CNI's and other platforms.

JitseHijlkema commented 5 months ago

Just announced on KubeCon 2024 in Paris, which will gives us the Hubble functionality:

Advanced Networking Observability which includes Hubble (CLI+UI) and with Retina (made OpenSource) also supports other CNI's and other platforms.

20240319_152300.jpg

bartwitkowski commented 4 months ago

@JitseHijlkema do you remember from which session is this photo?

JitseHijlkema commented 4 months ago

@JitseHijlkema do you remember from which session is this photo?

This is presented during Azure Day with Kubernetes on March 19 in Paris:

Link to presentation -Starting from slide 43

PixelRobots commented 4 months ago

@JitseHijlkema do you remember from which session is this photo?

You can find the video on the AKS youtube Channel. here is where it starts. https://youtu.be/hb__fHnx11s?si=jJderXVXRwWpHPfh&t=2013

I would advise you to watch all the sessions if you have not.

bartwitkowski commented 4 months ago

I was on the KubeCon in Paris but not on the Azure day but the Cilium :-). I'm watching Azure Day sessions in the meantime. Great stuff there!

Thanks @JitseHijlkema @PixelRobots

CalleB3 commented 2 months ago

Hey, any updates on this issue?

illrill commented 2 months ago

It was released in public preview in May: https://learn.microsoft.com/en-us/azure/aks/advanced-network-observability-cli

AKS bundles hubble-relay, and then you'll need to install hubble-ui yourself.

PixelRobots commented 2 months ago

I did a blog post on it too. It might help. https://pixelrobots.co.uk/2024/06/advanced-network-observability-supercharging-container-network-observability-in-azure-kubernetes-service-aks/

bartwitkowski commented 1 week ago

Well, Advanced Network Observability is a part of Advanced Container Networking Services and it is a paid feature: https://azure.microsoft.com/en-us/pricing/details/azure-container-networking-services/. Is there any way to do "BYO Hubble" and look through the metrics/flows/networkpolicies?