astronomer / astro-cli

CLI that makes it easy to create, test and deploy Airflow DAGs to Astronomer
https://www.astronomer.io
Other
350 stars 70 forks source link

Add `astro dbt deploy` command #1670

Closed jeremybeard closed 3 months ago

jeremybeard commented 3 months ago

Description

This change adds a new command astro dbt deploy for deploying a dbt project to a deployment independently of the Astro project of the deployment. This uses the new bundle deploy type, where arbitrary files can be mounted on the Airflow containers at a selected mount path.

The selection of the deployment is in line with astro deploy, where a deployment id argument can be provided, or one can be selected from a list.

By default the dbt project will be mounted on the Airflow containers at /usr/local/airflow/dbt/{dbt project name}, where the project name is extracted from the dbt project's dbt_project.yml file. This mount path can be overridden with --mount-path.

The command is currently hidden behind the CLI config dbt_deploys_enabled until the feature is GA.

The goerr113 lint rule is also removed because it was repeatedly being overridden, and doesn't seem to add a lot of value.

🧪 Functional Testing

📸 Screenshots

Screenshot 2024-06-24 at 3 35 28 PM

📋 Checklist

codecov[bot] commented 3 months ago

Codecov Report

Attention: Patch coverage is 63.52941% with 124 lines in your changes missing coverage. Please review.

Project coverage is 86.35%. Comparing base (d79dc21) to head (a7d14af).

Files Patch % Lines
cloud/deploy/bundle.go 60.60% 33 Missing and 19 partials :warning:
pkg/git/git.go 32.20% 36 Missing and 4 partials :warning:
cmd/cloud/dbt.go 78.04% 9 Missing and 9 partials :warning:
pkg/fileutil/files.go 72.72% 3 Missing and 3 partials :warning:
cloud/deploy/deploy.go 83.33% 3 Missing and 1 partial :warning:
airflow/docker_image.go 0.00% 1 Missing :warning:
cmd/cloud/deploy.go 50.00% 0 Missing and 1 partial :warning:
cmd/software/deploy.go 0.00% 0 Missing and 1 partial :warning:
houston/houston.go 0.00% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1670 +/- ## ========================================== - Coverage 86.74% 86.35% -0.40% ========================================== Files 114 116 +2 Lines 16712 16977 +265 ========================================== + Hits 14497 14660 +163 - Misses 1324 1394 +70 - Partials 891 923 +32 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

kushalmalani commented 3 months ago

@rujhan-arora-astronomer Looping you in since there are some changes to the software code because of change in a common util function.