z3z1ma / dbt-osmosis

Provides automated YAML management, a dbt server, streamlit workbench, and git-integrated dbt model output diff tools
https://z3z1ma.github.io/dbt-osmosis/
Apache License 2.0
422 stars 45 forks source link

[Feature Request] Ignore specific source columns #88

Closed AaronCoquet-Easypark closed 9 months ago

AaronCoquet-Easypark commented 9 months ago

I have a project that uses Fivetran (and some other tools) to sync data to Snowflake as sources for dbt. I don't want to include certain columns (_FIVETRAN_SYNCED, for example), since they're not terribly useful for our use case. I'd like to be able to configure dbt-osmosis to not inject them for every single source.

z3z1ma commented 9 months ago

Added in latest release 0.12.1

You can leverage this via adding a _blacklist key to your dbt-osmosis vars config as so:

# dbt_project.yml

vars:
  dbt-osmosis:
    _blacklist: ["columns", "to", "exclude", "[Rr]egex.*is_ok"] # <- this is the new reserved key
    <source_name>: <path>
    <source_name>:
      path: <path>
      schema: <schema>