Set up initial CI/CD - Githubissues

alexischicoinedatadeveloper / aws-databaricks-sqlmesh

A portfolio data pipeline project using Databricks in AWS with sqlmesh.

MIT License

1 stars 1 forks source link

Set up initial CI/CD #3

Closed alexischicoinedatadeveloper closed 6 months ago

alexischicoinedatadeveloper commented 6 months ago

Setup the basic CI/CD with a github action that has access to both AWS and the Databricks account. Will need to create an IAM setup for AWS and a service principal with a token for the Databricks account. The credentials will be saved in Github Secrets. Will also setup an S3 bucket for the terraform state. This small amount of manual work is needed to get the ball rolling with terraform.