ccao-data / data-architecture

Codebase for CCAO data infrastructure construction and management
https://ccao-data.github.io/data-architecture/
5 stars 3 forks source link

Factor out more dbt YAML selectors #446

Open jeancochrane opened 1 month ago

jeancochrane commented 1 month ago

In https://github.com/ccao-data/data-architecture/pull/373 we introduced a YAML selector in order to factor out a common node selection pattern that we use to run QC tests. We should factor out more of the node selection logic that we use in GitHub workflows so that everything is easier to reuse.

Note that since --selector overrides any behavior specified by other selection flags (issue context), we'll need to make use of YAML anchors and aliases in selectors.yml to make this code DRY.

Here are the patterns I think it would make sense to factor out:

dfsnow commented 1 month ago

@wagnerlmichael This one goes to you and I'll review/supervise.