cilium / hubble-ui

Observability & Troubleshooting for Kubernetes Services
https://www.cilium.io
Apache License 2.0
395 stars 62 forks source link

Verify Service Map for connectivity check #246

Open genbit opened 3 years ago

genbit commented 3 years ago

I'm trying hubble-ui 0.7.5 with connectivity-check and here is how it's looking on a service map:

Screen Shot 2020-11-16 at 11 31 38 AM

I'm looking for an expert opinion to verify if this looks correct or not and if not - what's missing.

joestringer commented 3 years ago

The intended destinations for most of these boxes is embedded into the name of the source pod, such as pod-to-a means it communicates with echo-a, pod-to-external-fqdn talks to an FQDN outside the cluster, etc.

Unexpected observations:

tgraf commented 3 years ago

echo-b is rendered outside the namespace. echo-b is a regular pod in the namespace

The reason for that is likely this:

        ports:
        - containerPort: 8080
          hostPort: 40000

This means that there is connectivity to a host IP that will end up in echo-b. It's still incorrect though. It's definitely a corner case as almost nobody is using hostPort. The connectivity check is using it to test pod to host connectivity.

Some pods like pod-to-a appear to be reaching out to the world, but they should only reach out to the echo-a pod via name. Maybe this is a race condition related to deployment? Does this persist over time?

I'm assuming this is traffic to k8s worker nodes. Depending on whether remote node identities are enabled, it will show up as world.