Launch Gen3 using Helm/K8s

[lbeckman314]

Gen3-Helm Deployment Instructions: https://github.com/uc-cdis/gen3-helm

[lbeckman314]

• [x] Deploy on https://aced-training.compbio.ohsu.edu
• [x] Successfully log in as Mock user (test@example.com)
• [x] Add Google and Microsoft logins
• [x] Fix /data/schema 404 error
• [x] Add data upload/ETL workflow
  • [x] Mount volumes in ETL pod
  • S3:// data (e.g. aced.json)
  • Sheepdog credentials (e.g. sheepdog-credentials.yaml)
  • User config (e.g. user.yaml)
  • User credentials (e.g. fence.yaml)
  • [x] Add service connections in ETL pod
  • Arborist
  • PostgreSQL
  • Elasticsearch
  • [x] Build and push ETL image to quay.io
  • [x] Add ETL upload script with Gen3 SDK
  • [x] Connect OHSU Minio bicket
  • [x] Connect AWS Minio buckets
  • [x] Add RDS and connect to Helm
  • [x] Add OpenSearch and connect to Helm

---

ETL:

• [ ] Map Fence/ETL Bot AWS Access Keys to ETL pod
• [ ] Map Fence.ALLOWED_DATA_UPLOAD_BUCKETS to ETL pod
• [ ] Create an ETL Bot IAM user modeled after Fence Bot in AWS (different policies?)
• [ ] How best to separate development/staging environments for ETL pod?
  • Separate etl.yaml files with different configmaps?
  • Switch or variable to tell Helm to use a particular environment
  • Current differences center on studies/output location and NodeSelector

[tbaxter73]

meeting with Jawad(UC) to talk about issues loading images

[lbeckman314]

Tracking PR in gen3-helm: https://github.com/ACED-IDP/gen3-helm/pull/2