itkpi / trembita

Model complex data transformation pipelines easily
Apache License 2.0
46 stars 3 forks source link

Deploy Pipelines #20

Open vitaliihonta opened 5 years ago

vitaliihonta commented 5 years ago

Sometimes in distributed systems, there is a need to deploy components as a result of some user interactions. For instance, deploy spark cluster for distributive computations when a user wants so. Need to research about something I'll call "Deploy pipelines". The idea is simple: write an OutputT which will (based on input pipeline elements) deploy something (for instance docker containers, k8s pods, spark clusters, etc.)