kubeflow / pipelines

Machine Learning Pipelines for Kubeflow
https://www.kubeflow.org/docs/components/pipelines/
Apache License 2.0
3.56k stars 1.61k forks source link

[feature] deploy pipelines on local K8S with private network environment. #10235

Closed oldthreefeng closed 3 months ago

oldthreefeng commented 10 months ago

Feature Area

/area components

What feature would you like to see?

deploy pipelines on private network environment.

from a private registry (For security reasons we are not allowed to use public registries such as gcr.io - so we pull the images and push them to our environment).

What is the use case or pain point?

some situation, we canit use public registries such as gcr.io .

Is there a workaround currently?

no more work on this

i find a issue about this. but no solution for this problem . https://github.com/kubeflow/pipelines/issues/6931


Love this idea? Give it a 👍.

oldthreefeng commented 10 months ago

now , first to sync image to my private registry.

# get image list 
kustomize build env/platform-agnostic-pns | grep "gcr.io/" |awk -F 'gcr' '{ print "gcr"$2}' | sort -u 
gcr.io/google-containers/busybox
gcr.io/ml-pipeline/api-server:2.0.3
gcr.io/ml-pipeline/argoexec:v3.3.10-license-compliance
gcr.io/ml-pipeline/cache-deployer:2.0.3
gcr.io/ml-pipeline/cache-server:2.0.3
gcr.io/ml-pipeline/frontend:2.0.3
gcr.io/ml-pipeline/metadata-envoy:2.0.3
gcr.io/ml-pipeline/metadata-writer:2.0.3
gcr.io/ml-pipeline/minio:RELEASE.2019-08-14T20-37-41Z-license-compliance
gcr.io/ml-pipeline/mysql:8.0.26
gcr.io/ml-pipeline/persistenceagent:2.0.3
gcr.io/ml-pipeline/scheduledworkflow:2.0.3
gcr.io/ml-pipeline/viewer-crd-controller:2.0.3
gcr.io/ml-pipeline/visualization-server:2.0.3
gcr.io/ml-pipeline/workflow-controller:v3.3.10-license-compliance
gcr.io/tfx-oss-public/ml_metadata_store_server:1.14.0
# docker pull and docker tag 

second use

./kustomize build pipelines/manifests/kustomize/env/platform-agnostic-pns | sed  's@gcr.io/ml-pipeline/@example.com/mlops-pipeline@g'  | sed 's@gcr.io/tfx-oss-public@example.com/mlops-pipeline@g'  | kubectl apply -f -

but when we use the pipelines . this two image are hard coded... gcr.io/ml-pipeline/kfp-launcher@sha256:50151a8615c8d6907aa627902dce50a2619fd231f25d1e5c2a72737a2ea4001e gcr.io/ml-pipeline/kfp-driver@sha256:8e60086b04d92b657898a310ca9757631d58547e76bbbb8bfc376d654bef1707

github-actions[bot] commented 7 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

rimolive commented 6 months ago

Currently there is no plan to support disconnected environments. I recommend you to use a Kubeflow distribution or look for paid consulting.

github-actions[bot] commented 4 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] commented 3 months ago

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.