While doing some research into potential software for this budget season, I came across dagster. It's an orchestration platform similar to Airflow that uses a declarative Python model similar to dbt. It integrates with dbt natively and also shares dbt's open-core licensing. IMO, this could be a great potential fit for replacing our ad-hoc Python/R ETL scripts. If we ever get the time, I think it would be worthwhile to do a local spike of dagster using the CLI.
While doing some research into potential software for this budget season, I came across dagster. It's an orchestration platform similar to Airflow that uses a declarative Python model similar to dbt. It integrates with dbt natively and also shares dbt's open-core licensing. IMO, this could be a great potential fit for replacing our ad-hoc Python/R ETL scripts. If we ever get the time, I think it would be worthwhile to do a local spike of dagster using the CLI.