dbt-labs / dbt-external-tables

dbt macros to stage external sources
https://hub.getdbt.com/dbt-labs/dbt_external_tables/latest/
Apache License 2.0
294 stars 119 forks source link

update test command #197

Closed jeremyyeo closed 1 year ago

jeremyyeo commented 1 year ago

Description & motivation

Tests are failing because --var can't be used anymore with dbt 1.5. Need to use the right flag here --vars.

Also making a quick small change while we're at it - previously an error in one of the individual shell commands (individual dbt invocations) results in a passing overall CircleCI step. This means if:

dbt deps --target $1 ✅
dbt seed --full-refresh --target $1 ✅
dbt run-operation prep_external --target $1 ✅
dbt run-operation stage_external_sources --vars 'ext_full_refresh: true' --target $1 ❌
dbt run-operation stage_external_sources --target $1 ✅
dbt test --target $1 ✅

All of those shell commands / dbt invocations are considered a single CircleCI step (see command:): https://github.com/dbt-labs/dbt-external-tables/blob/34c1cc4bbcf28ccb0243807441bf642e801ad158/.circleci/config.yml#L39-L43

The CircleCI run step on an aggregate still passes - we should make these fail. Examples below:

https://app.circleci.com/pipelines/github/dbt-labs/dbt-external-tables/291/workflows/cd1bc6eb-8c67-4932-b96c-bb201a4fa189/jobs/1297

https://app.circleci.com/pipelines/github/dbt-labs/dbt-external-tables/291/workflows/cd1bc6eb-8c67-4932-b96c-bb201a4fa189/jobs/1298

And writeup from CircleCI:

https://support.circleci.com/hc/en-us/articles/115015733328-Step-Should-Fail-but-Job-Finished-Successfully

Checklist

jeremyyeo commented 1 year ago

Merging this while acknowledging current incompatibility with the latest dbt-redshift 1.5 rc - https://github.com/dbt-labs/dbt-external-tables/issues/200