Support Istio-compatible deployment with Cilium

juliusvonkohout commented 1 month ago

Validation Checklist

[X] Is this a Kubeflow issue?
[X] Are you posting in the right repository ?
[X] Did you follow the installation guide https://github.com/kubeflow/manifests?tab=readme-ov-file ?
[X] Is the issue report properly structured and detailed with version numbers?
[X] Is this for Kubeflow development ?
[ ] Would you like to work on this issue?
[X] Join our slack channel using wg-manifests.

Version

master

Describe your issue

See https://github.com/kubeflow/kubeflow/issues/7553#issue-2251830611

@adriantorrie @AndersBennedsgaard @Apsu

Steps to reproduce the issue

Try cilium with Kubeflow

Put here any screenshots or videos (optional)

No response

thesuperzapper commented 1 month ago

As one of the @kubeflow/wg-notebooks-leads I want to highlight that many of the components have a fairly hard dependency on specifically Istio including Notebooks, Profiles, Volumes, TensorBoards, and KServe.

Given this, we would have to see a specific need from users/distributions that cant be achieved with Istio for it to make sense from a developer time perspective.

And I don't see an easy way to remove the need for a service mesh altogether (in multi-user, AKA "kubeflow platform" mode), as this would require implementing authentication into every app.

krom commented 1 month ago

Did you really want to mention me?

adriantorrie commented 3 weeks ago

As one of the @kubeflow/wg-notebooks-leads I want to highlight that many of the components have a fairly hard dependency on specifically Istio including Notebooks, Profiles, Volumes, TensorBoards, and KServe.

Given this, we would have to see a specific need from users/distributions that cant be achieved with Istio for it to make sense from a developer time perspective.

And I don't see an easy way to remove the need for a service mesh altogether (in multi-user, AKA "kubeflow platform" mode), as this would require implementing authentication into every app.

The intention of the request isn't to remove the need for a service mesh. It's to not force the need for Istio's implementation of a service mesh (in favour of using a Cilium service mesh for example, or any other project that supports the underlying Ingress/Gateway APIs).

Just to further clarify from my original request:

Cilium supports cluster mesh and service mesh
Cilium supports both K8S Gateway API and K8S Ingress
Cilium can integrate with Keycloak via K8S Gateway API

From my perspective, the above would show that Cilium supports what you've called the "kubeflow platform" mode, and it would seem my request is merely asking for abstraction instead of hard-coding the use of Istio objects?

Once Cilium is installed on a cluster it's capabilities make the need for Istio being on that same cluster redundant, from what I can tell.

juliusvonkohout commented 3 weeks ago

@adriantorrie @AndersBennedsgaard are you willing to implement cilium as an option? I need someone responsible for driving this and willing to put in the time and effort. Only then i can review it.

adriantorrie commented 2 weeks ago

@adriantorrie @AndersBennedsgaard are you willing to implement cilium as an option? I need someone responsible for driving this and willing to put in the time and effort. Only then i can review it.

I'm happy to pitch in and help

juliusvonkohout commented 2 weeks ago

Hello, are you both on the new Slack in #kubeflow-platform ? See https://www.kubeflow.org/docs/about/community/

AndersBennedsgaard commented 2 weeks ago

I will most likely not be able to contribute in this matter (but yes, I am on the new Slack channels)

adriantorrie commented 2 weeks ago

Hello, are you both on the new Slack in #kubeflow-platform ? See https://www.kubeflow.org/docs/about/community/

@juliusvonkohout Yes

juliusvonkohout commented 2 weeks ago

@adriantorrie then please write up a proposal as we have it for rootless Kubeflow etc.

kubeflow / manifests