linkerd / linkerd2

Ultralight, security-first service mesh for Kubernetes. Main repo for Linkerd 2.x.
https://linkerd.io
Apache License 2.0
10.71k stars 1.28k forks source link

Test how linkerd and cillium interact #6246

Open dadjeibaah opened 3 years ago

dadjeibaah commented 3 years ago

Based on https://github.com/linkerd/linkerd2/issues/6238#issuecomment-858089072 and https://github.com/linkerd/linkerd2/issues/6238#issuecomment-858796177, it appears that there may be issues between linkerd proxies and Cillium's load balancing features. In order to get a better understanding on what these issues are and possibly how we can fix them, we should write new integration tests that exercise running Linkerd with Cillium.

We should make sure that some of the following features work in a cillium environment:

There may be other issues like multicluster but that might be out of scope for these tests.

kleimkuhler commented 3 years ago

Another issue I have observed this interaction in is #5932

Not all users who have commented on the issue use Cilium, but if you look at my comment I was able to observe this behavior and fix it by disabling hostServices on Cilium.

alex-berger commented 3 years ago

For reference these are the corresponding Cilium issue/PR for this problem

jastBytes commented 3 years ago

I am using cilium and tried to set up multicluster with linkerd today. My problem is that the linkerd-service-mirror proxy is using the external ip address of the 2nd cluster to contact the multicluster gateway instead of the local probe service. That’s why the identity is not correct when connecting to the 2nd cluster and the 2nd cluster's gateway refuses the connection. Special thanks to @mateiidavid which tried to help me here. Maybe this is a cilium issue as well. https://linkerd.slack.com/archives/C89RTCWJF/p1629357555443900

jastBytes commented 3 years ago

So I tried the exact same configuration but switched both clusters to calico and it works right away. So there is definitely something going on with cilium and multicluster linkerd.