search

kubeflow-kale / kale

Kubeflow’s superfood for Data Scientists
http://kubeflow-kale.github.io
Apache License 2.0
632 stars 128 forks source link

How to integrate kubeflow-kale extension to run pipelines on a seperate standalone cluster of Kubeflow pipelines like GCP AI pipeline #193

Open rituraj17 opened 4 years ago

rituraj17 commented 4 years ago

Hi Team, I am currently trying to use the kubeflow kale jupyter extension on my local jupyterlab server without Kubernetes and kubeflow installed and trying to run my code pipeline on GCP AI pipeline server or any other Cloud Kubeflow pipeline server. I am able to do it through kubeflow pipeline SDK(As it has a feature to add hostname details). But when trying to achieve through the kubeflow-kale extension it does not work. As I am aware we need to provide the hostname of the Kubeflow pipeline server which I was not able to add on kubeflow-kale UI extension drop-down fields. I have explored a lot of kubeflow-kale materials and blogs but was not able to find the solution. Almost all the blogs and material about Kubeflow-kale implementation has been done on the Kubeflow hosted notebook server

Can anyone help me with the following doubts about Kubeflow-kale:-

  1. Kubeflow-kale is only supported for kubeflow hosted notebook server?
  2. If No, How can we provide the option to run the pipeline on a remote Server like GCP AI Pipelines?
yaliqin commented 4 years ago

Hi Team, I am currently trying to use the kubeflow kale jupyter extension on my local jupyterlab server without Kubernetes and kubeflow installed and trying to run my code pipeline on GCP AI pipeline server or any other Cloud Kubeflow pipeline server. I am able to do it through kubeflow pipeline SDK(As it has a feature to add hostname details). But when trying to achieve through the kubeflow-kale extension it does not work. As I am aware we need to provide the hostname of the Kubeflow pipeline server which I was not able to add on kubeflow-kale UI extension drop-down fields. I have explored a lot of kubeflow-kale materials and blogs but was not able to find the solution. Almost all the blogs and material about Kubeflow-kale implementation has been done on the Kubeflow hosted notebook server

Can anyone help me with the following doubts about Kubeflow-kale:-

  1. Kubeflow-kale is only supported for kubeflow hosted notebook server?
  2. If No, How can we provide the option to run the pipeline on a remote Server like GCP AI Pipelines?

1) I integrated Kale into Jupyterlab as an extension. Then it can be used to generate the pipeline file.
2) use shared storage to make the generated pipeline file available from Kubeflow namespace 3) then you can find the file from Kubeflow namespace 4) I add a virtual service to make istio-ingress gateway direct the link in Jupyterlab Kale Deployment Panel to Kubeflow pipeline UI, so i can use the pipeline file by clicking the link in the Kale deployment panel. Istio-ingress gateway is already installed when we install kubeflow. There is a virtual service to handle kubeflow related links. You can write a new virtual service to navigate the link in Kale deployment panel(generated by Kale library) .