search

RhodiumGroup / docker_images

Docker images for Rhodium's jupyterlab deployments
2 stars 2 forks source link

Use Elyra for kubeflow notebook pipelines #143

Open delgadom opened 3 years ago

delgadom commented 3 years ago

This is ridiculous: https://medium.com/codait/creating-notebook-pipelines-using-elyra-and-kubeflow-pipelines-f9606449cc53

https://github.com/elyra-ai/elyra

brews commented 3 years ago

Could we consider kubeflow-pipelines? Does that get us the features we want or do we want all of elyra?

Edit: clarifying

Edit2: I don't think we get the notebook pipeline UI extension with kubeflow-pipeline without installing all of kubeflow. So, elyra is needed if we want to shortcut this.

bolliger32 commented 3 years ago

whoa this looks like a pretty cool project. @brews I don't know much about kubeflow-pipelines, but I would imagine one of the key features of this elyra thing is the fact that it uses notebooks natively, so that would mesh with (I think) a lot of different teams' workflows. I can certainly imagine this being a useful tool for organizing our coastal pipelines. Might load it onto the coastal image and start playing around with it to check it out. It wasn't totally obvious to me whether it would be easy to just plug in our GCS bucket or not. At the very least, it would be helpful for us to have a visual flowchart of notebooks even if we never actually executed them through this tool (tho obviously there are easier ways to make a flowchart lol)