Closed mcowart123 closed 3 months ago
Never mind; I figured out the syntax.
# setting configuration
dbt.config(
materialized = "incremental",
unique_key = ['_md_source_group', 'period_key', 'market_key', 'product_key'],
incremental_strategy = "merge",
partition_by = ['_md_source_group', 'period_key'])
The documentation is confusing here though:
There's a limit to how complex you can get with the dbt.config() method. It accepts only literal values (strings, booleans, and numeric types) and dynamic configuration. Passing another function or a more complex data structure is not possible. The reason is that dbt statically analyzes the arguments to config() while parsing your model without executing your Python code. If you need to set a more complex configuration, we recommend you define it using the config property in a YAML file.
Describe the bug
I'm trying to use a python model with multiple partition_by columns and either it's not supported or I cannot figure out the syntax to pass the columns as. The documentation only shows examples of single columns.
Expected behavior
Multiple partition columns should be allowed.
Steps to reproduce
This works (single unquoted column name passed to partition_by):
Any variation of the following (no quotes, no brackets, etc) fail:
Output of dbt run
System information
The output of
dbt --version
:The operating system you're using: MacOS (jobs running on Databricks)
The output of
python --version
: 3.10.13