Next-gen data orchestration
Dagster is our data orchestrator. Every ETL step takes place here.
Dagster Cloud is a hosted front-end for our Dagster servers where you can observe and run integration jobs.
Dagster hosts multiple "code locations", one for each of our business units, including a separate one for our CMO:
Each code location hosts and runs the code and configurations for each respective business unit. Behind-the-scenes, these are containers run on Google Cloud Kubernetes. Each code location has it's own respective jobs, schedules, sensors, and assets.
Before you merge: