search

kubeflow / manifests

A repository for Kustomize manifests
Apache License 2.0
810 stars 874 forks source link

Provision Minio bucket lifecycle rules when deploying Kubeflow #2375

Closed wooleybugger closed 3 months ago

wooleybugger commented 1 year ago

Is it possible to provision minio bucket lifecycle rules when deploying Kubeflow and if so where/how?

I am looking for the manifest that will allow me to provision lifecycle rules on kubeflow minio storage buckets. The desire is to have the rules in place prior to deployment of Kubeflow. The closest possibility appears to be apps/pipeline/upstream/third-party/argo/base/workflow-controller-configmap-patch.yaml or apps/pipeline/upstream/third-party/argo/upstream/manifests/quick-start/base/overlays/workflow-controller-configmap.yaml, but no "cigar".

Thanks

juliusvonkohout commented 1 year ago

We are moving away from minio and we ship a 3 year old insecure version.

wooleybugger commented 1 year ago

Can you say what storage solution you are moving to and when?

juliusvonkohout commented 1 year ago

There are some bugs here regarding the minio replacement https://github.com/kubeflow/pipelines/pull/7725

wooleybugger commented 1 year ago

Well there is alot to read there. My takeaway is that Zenko cloudserver and S3Proxy are the choices, depending if AWS/GCS or on-premises. Not sure if S3proxy equates to "file backend".

" So to summarize we try Zenko cloudserver (Apache 2 licence and actively maintained and proposed by james Liu) in gateway mode for AWS and GCP and with the file backend for on-premises deployments."

wooleybugger commented 1 year ago

@juliusvonkohout , Is there a roadmap/timeline for the replacement of Minio?

juliusvonkohout commented 1 year ago

@juliusvonkohout , Is there a roadmap/timeline for the replacement of Minio?

according to google after Q2 2023 the will start working on it.

wooleybugger commented 1 year ago

Thanks

juliusvonkohout commented 10 months ago

@wooleybugger can you join the KFP community meetings and discuss this with google again?

github-actions[bot] commented 4 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] commented 3 months ago

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.