search

kubeflow / pipelines

Machine Learning Pipelines for Kubeflow
https://www.kubeflow.org/docs/components/pipelines/
Apache License 2.0
3.6k stars 1.62k forks source link

Render static graph when compiling a pipeline in a notebook #504

Closed amygdala closed 6 months ago

amygdala commented 5 years ago

Feature request: When compiling a pipeline in a notebook, render the static graph as output, same as the graph shown when you upload a pipeline.

gaoning777 commented 5 years ago

Why is the compiler generating a static graph important when we already have the pipeline UI supporting it? Would it be a common scenario that users author the pipeline without a pipeline deployment(we do support on-prem minikube) such that the DSL generating the static graph is beneficial? Thanks

yebrahim commented 5 years ago

I can see that providing a nice feedback loop to iterate visually on the pipeline. You make a change, compile, see the output graph, then iterate. It's easier to debug high level issues just by looking at the graph than dozens of Python lines.

It won't be easy to support though, since the graph is built on the fly by our UI layer. It does make one more argument for enabling editing the pipeline source code in our UI though? @Ark-kun @qimingj

Ark-kun commented 5 years ago

Yes, visual pipeline editing would be great.

As for the graph, I could generate it, but I'm, a bit hesitant to add more dependencies (to graphing packages) to the SDK.

Ark-kun commented 5 years ago

Another solution would be to add IFrame to Jupyter (easy - I've checked) that shows the run page (without navigation). See https://github.com/kubeflow/pipelines/issues/182

gaoning777 commented 4 years ago

@amygdala is this still a wanted feature?

gaoning777 commented 4 years ago

/unassign @gaoning777 /assign @numerology @Ark-kun

Bobgy commented 4 years ago

This sounds useful. @Ark-kun Can you provide me an example python notebook that compiles a pipeline, uploads it and shows the iframe of pipeline detail page, so I can try to iterate UI on it.

Bobgy commented 4 years ago

It seems keeping uploading versions to the same pipeline and maybe auto refresh the pipeline detail page showing latest result is a good enough UX.

Bobgy commented 4 years ago

WDUT?

amygdala commented 4 years ago

Yeah, I think we can consider it lower-priority. (though if we get some breathing room it would be a 'nice to have').

rmgogogo commented 4 years ago

Middle/long term plan: this one can be handled together with IR topic. when we have an IR, it can help decouple the runner with visualizer.

In short term, we may bring the argo based render to jupyter world but I think it has lower priority as it will be replaced by IR based sooner or later. SG?

Bobgy commented 4 years ago

@rmgogogo I think @Ark-kun's main proposal is that putting something to jupyter is minimal cost, jupyter just takes a url to show something. If we have a page that only shows the content, it will work by itself.

So it doesn't really have any conflict with IR.

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Bobgy commented 4 years ago

/lifecycle frozen

rimolive commented 6 months ago

Closing this issue. No activity for more than a year.

/close

google-oss-prow[bot] commented 6 months ago

@rimolive: Closing this issue.

In response to [this](https://github.com/kubeflow/pipelines/issues/504#issuecomment-2039387460): >Closing this issue. No activity for more than a year. > >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.