Open ranchodeluxe opened 10 months ago
The "Short-Term Goal" mentioned above is over here (https://github.com/NASA-IMPACT/veda-pforge-job-runner) and has been an invaluable exercise in understanding user interaction, user needs and where flink/pangeo-forge is feeling flunky
Problem
If you're a k8s fan then using
pangeo-forge-runner.FlinkOperatorBakery
isn't that gross out of the box. But the k8s interface is a hard pill to swallow for those who are less familiar. Plus there is only the terminal UI/UX.Long-Term Goal
Create a UI/UX experience that approximates what GCP DataFlow does for Apache Beam. Tools such as
Argo[ CD || Workflows]
should be able to handle most of this (especially the auth and RBAC for workflows) but maybe others have different ideas on how to accommodate this:Shopping list:
Short-Term Goal
Just use any CI/CD system such as GH Actions (possibly codify it in GH Actions plugin) and we can skip some of the above requirements and user experience for now