Open miguelvr opened 5 years ago
We don't have it added into pipeline yet
@johnugeorge Hi! Do you have any roadmap about integrating into pipelines?
Looking at TFJob pipeline integration it seems not to be hard.
/kind question /area pipelines /priority p2
@jtfogarty: The label(s) area/pipelines
cannot be applied, because the repository doesn't have them
/area engprod
I've seen discussion on launchers here and on Kubeflow/Pipelines, but nothing concrete. Has anyone built a solution for this? If not, I plan to create one based on the TFJob launcher and can share
@ca-scribner Thanks! I do not think there is progress from our side.
I'm completely new to kubeflow, but the main advantage of it from my perspective is the usage of pipelines to setup production ready end to end machine learning workflows.
I'm trying to port production code that we have into a kubeflow pipeline. However, one of the steps is to perform distributed training. I understand how to set up a pipeline that uses a small number of GPUs within the same pod, but still don't get how to integrate distributed training with a
PyTorchJob
into a kubeflow pipeline.Is this possible? What are the current solutions for this?