dbt-labs / dbt-core

dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
https://getdbt.com
Apache License 2.0
9.99k stars 1.63k forks source link

[Feature] define groups in root-level yml file #10945

Open graciegoheen opened 3 weeks ago

graciegoheen commented 3 weeks ago

Describe the feature

Currently, you can define groups in any “content” yml file (models/, seeds/, etc), but not as a root-level file.

Since groups can be applied to most (soon more) resource types, it would make sense to be able to define them in a root yml file.

See slack thread

Acceptance Criteria

I can define my group in a root-level yml file:

# /groups.yml

 groups:
  - name: marketing
    owner:
      name: marketing_pod
      slack: ask-marketing-pod
dbeatty10 commented 2 weeks ago

Since a groups: can already be added to any YAML file in the models, etc. directories, it would surprise me if there were also a top-level groups.yml file where it could be defined.

There is precedent for additional top-level YAML files besides dbt_project.yml, but none of them have cross-over with YAML properties files:

Overall, it would seem most simple to not do this since it is already possible by just adding group definitions to our properties YAML files.

If we do choose to adopt a new top-level groups.yml, it would require more code to juggle the multiple potential locations plus more explanation in our product documentation. It's definitely doable, but seems unnecessary.

Current options

For example, the convention I prefer is to add them to a stand-alone _groups.yml at the root of the models directory.

But people can choose whichever convention they like best. For example, you could create a groups/groups.yml file and configure it with model-paths within dbt_project.yml like this:

model-paths: ["models", "groups"]