Open davidspek opened 3 years ago
@karlschriek @matty1979 I'm expecting you guys will run into a lot more things while working on this deployment so it would be good to document that here.
Ping @samdobson In case you are interesting in helping out.
Also ping @PatrickXYS in response to https://github.com/kubeflow/manifests/pull/1832#issuecomment-822859477 if anybody from AWS would like to help out with this effort.
Use RDS for pipelines database
This is part of KFP support, if we want to enable this, this should be added in kubeflow/pipelines instead.
Use S3 for pipelines instead of MinIO
Same for KFP
For other parts of Cognito stuff, we will explore those in AWS distribution, so I don't think we will be able to contribute to argoflow.
@PatrickXYS The AWS integrations for things like Cognito in ArgoFlow would all be kustomize manifests, so they would likely be the same as for the AWS distribution.
@DavidSpek I agree, so you can simply check out AWS side best practice after we're done with those features.
Somewhat related PR/issue about enabling AWS ALB authentication in KFP SDK Client: https://github.com/kubeflow/pipelines/pull/4182
@DavidSpek Are all of the AWS integrations relatively complete besides Cognito? We currently have the standard KF 1.2 distribution deployed on AWS and I’m wondering if this distribution is at a good place for us to try it.
AWS plugin for profile controller
Is this required if we’re not using Cognito?
@soleares Most of the AWS integrations are complete. Cognito we just haven’t gotten around to yet, but from what I’ve read it shouldn’t be difficult to setup with Oauth2-Proxy.
The plug-in for the profile controller is just to delegate IAM roles to users and their workloads, it shouldn’t be necessary for anything else.
@DavidSpek What Kubeflow components use this plug-in? Is it required for basic pipeline functionality?
@soleares The profile controller uses the plugin for service accounts. It is not necessary for functionality of the components.
Creating a quick list of things that still need to be done so we can track progress.
What still needs to be done: