marco-roy / DDO

A DBT package to perform DataOps & administrative CI/CD on your data warehouse.
Apache License 2.0
16 stars 5 forks source link

DBT 0.21.0 requires new configuration to override global DBT macros #1

Open mroy-seedbox opened 3 years ago

mroy-seedbox commented 3 years ago

See https://docs.getdbt.com/reference/dbt-jinja-functions/dispatch#overriding-global-macros

dispatch:
  - macro_namespace: dbt
    search_order: ['my_project/my_package', 'dbt']
AdrianCio commented 2 months ago

@mroy-seedbox did you get this to work? I'm struggling with this

mroy-seedbox commented 2 months ago

Yup, all you need is something like this in your dbt_project.yml:

name: my_project

# Allow my macros to override global DBT macros
dispatch:
  - macro_namespace: dbt
    search_order: ['my_project', 'dbt']
AdrianCio commented 2 months ago

This is strange, I did this but it gives me No materialization 'plain' was found for adapter postgres! (searched types 'default' and 'postgres')

mroy-seedbox commented 2 months ago

That is weird indeed! But I've never tested it with Postgres (only with Snowflake)... maybe there's something missing to make it work? 🤔

mroy-seedbox commented 2 months ago

Try with something like this:

# Allow my macros & DDO macros to override global DBT macros
dispatch:
  - macro_namespace: dbt
    search_order: ['my_project', 'dbt_packages/DDO/macros', 'dbt']
AdrianCio commented 2 months ago

Same issue :(

macro-paths: 
  - "macros"
  - dbt_modules/DDO/macros/materializations

# Allow my macros & DDO macros to override global DBT macros
dispatch:
  - macro_namespace: dbt
    search_order: ['my_project', 'dbt_modules/DDO/macros', 'dbt']
mroy-seedbox commented 2 months ago

Try with dbt_packages/ instead of dbt_modules/.

dbt_modules/ was for older versions of DBT.

AdrianCio commented 2 months ago

Same result with this config. It is a good point that I was using the old name, but unfortunately it didn't change the output. I also ran a quick test where I removed dbt_packages from clean-targets but again, same result as in my previous post.

macro-paths: 
  - "macros"
  - dbt_packages/DDO/macros/materializations

# Allow my macros & DDO macros to override global DBT macros
dispatch:
  - macro_namespace: dbt
    search_order: ['my_project', 'dbt_packages/DDO/macros', 'dbt']

target-path: "target"  # directory which will store compiled SQL files
clean-targets:         # directories to be removed by `dbt clean`
  - "target"
  - "dbt_packages"
mroy-seedbox commented 2 months ago

Have you tried creating a copy of the materialization in your own project?

If that doesn't work, then it must be something specific to do with the DBT Postgres implementation. 😕

mroy-seedbox commented 2 months ago

@AdrianCio: did it work? 😅

AdrianCio commented 2 months ago

Apologies, I was off for a week. Indeed I ended up copying the materialization in my project which works fine.