search

kubeflow / community

Information about the Kubeflow community including proposals and governance information.
Apache License 2.0
159 stars 220 forks source link

Sky computing #787

Open hongbo-miao opened 1 year ago

hongbo-miao commented 1 year ago

/kind feature

Why you need this feature: This is just an early idea. It would be awesome to support sky computing - training a machine learning model cross multiple clouds (different vendors). It will help potentially lower the cost. ☺️

Describe the solution you'd like: If possible, somehow integrating with SkyPilot would be awesome. For example, a naive idea is somehow allowing different Kubeflow pipeline component can run on individual instances on different clouds (AWS, Google Cloud, Azure, etc.) and then collect the experiment results back to the cluster hosting Kubeflow for example AWS.

Here is screenshot from YouTube explaining how SkyPilot works for a cross-cloud machine job:

image

andreyvelich commented 4 weeks ago

Hi @hongbo-miao, I think this great idea! Any thoughts on how SkyPilot can be integrated with Kubeflow Components in addition to KFP ?

juliusvonkohout commented 4 days ago

/transfer community