search

z3z1ma / dbt-osmosis

Provides automated YAML management, a dbt server, streamlit workbench, and git-integrated dbt model output diff tools
https://z3z1ma.github.io/dbt-osmosis/
Apache License 2.0
422 stars 45 forks source link

Remove if sources or models are empty #97

Closed syou6162 closed 9 months ago

syou6162 commented 9 months ago

Why

I use dbt-osmosis with the following settings written in dbt_project.yml. This means there is one yaml file for each model.

models:
  my_model:
    raw_stage:
      +dbt-osmosis: "{model}.yml"
...

sources are managed in a separate yaml. When running dbt-osmosis yaml refactor with this setting, an empty sources setting is output to the yaml file.

version: 2
models:
  - name: my_model
    columns:
      - name: my_comumn
        data_type: DATE
...
sources: []

The empty sources section is meaningless, so I use yq to delete the sources section every time (yq --inplace 'del(.sources)' models/my_model.yml is executed).

It's tedious to perform this file operation every time, so I modified it to delete them if sources or models are empty in the yaml generated by running dbt-osmosis yaml refactor.