[feature] Should be able to set _DEFAULT_LAUNCHER_IMAGE using environment variable in notebook pod

kubeflow / pipelines

Machine Learning Pipelines for Kubeflow

https://www.kubeflow.org/docs/components/pipelines/

Apache License 2.0

3.53k stars 1.59k forks source link

[feature] Should be able to set _DEFAULT_LAUNCHER_IMAGE using environment variable in notebook pod #7225

Closed typhoonzero closed 4 months ago

typhoonzero commented 2 years ago

Feature Area

What feature would you like to see?

Currently the default launcher image is set using a constant: sdk/python/kfp/compiler/v2_compat.py:_DEFAULT_LAUNCHER_IMAGE = "gcr.io/ml-pipeline/kfp-launcher:1.8.7", however, if we want to use a private registry when launching a pipeline in Kubeflow notebook, we may need to use an environment variable to set this, so that we can just configure that in the Kubeflow notebook YAML file.

Is there a workaround currently?

I can set it directly in the function call here: https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.client.html#kfp.Client.create_run_from_pipeline_func, but I'd like to configure a default for all users.

Love this idea? Give it a 👍. We prioritize fulfilling features with the most 👍.

chensun commented 2 years ago

Hi @typhoonzero, v2 compatible mode is being deprecated in the next SDK release: https://github.com/kubeflow/pipelines/issues/6829

We would have a similar launcher/driver image for v2. And we plan to version it the same way as we version the rest KFP backend images. That being said, I'm still interested in why would you want to use use private registry for the launcher image. Are you making custom images? Can you share some details to help us understand your use case? Thanks!

typhoonzero commented 2 years ago

Thanks for the reply @chensun , my case is simple, I can't access gcr.io in China, so I'm hoping to host the images in a private registry.

dbg-raghulkrishna commented 1 year ago

any update on this?

wang-mask commented 7 months ago

how to set the image in create_run_from_pipeline_func?

xin053 commented 6 months ago

https://github.com/kubeflow/pipelines/pull/10269#issuecomment-1916254578

rimolive commented 4 months ago

Closing this issue as it was implemented. Feel free to reopen it if issue persists.

/close

google-oss-prow[bot] commented 4 months ago

@rimolive: Closing this issue.

In response to [this](https://github.com/kubeflow/pipelines/issues/7225#issuecomment-1991040653): >Closing this issue as it was implemented. Feel free to reopen it if issue persists. > >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.