LewisDavies / upstream-prod

A dbt package for easily using production data in a development environment.
32 stars 4 forks source link

The modified "ref" macro breaks when using "dbt run-operation" #10

Closed franzoni315 closed 1 year ago

franzoni315 commented 1 year ago

In my team, we commonly use the macro generate_model_yaml from the package dbt-codegen in order to easily generate the yml documentation for a new model.

However, while running dbt run-operation generate_model_yaml --args '{"model_names": [<model_name>]}', we had the following error mentioning the altered macro ref()

12:57:59  Running with dbt=1.3.2
12:58:00  Encountered an error while running operation: Compilation Error in macro ref (macros/ref.sql)
  'this' is undefined

  > in macro generate_model_yaml (macros/generate_model_yaml.sql)
  > called by macro ref (macros/ref.sql)

The problem is that in dbt run-operation the variable this does not seem to exist.

The variable this is the default value for the variable current_model, but this seems only to be used in a jinja's log macro.

LewisDavies commented 1 year ago

Thanks for raising this, this is completely new to me as I don't use dbt-codegen. You're right about current_model, it was much more important in early versions of this package but is currently only used for logging purposes. I'll think about the best way to deprecate it since the variable isn't really needed anymore.

In the meantime, you should be able to fix this by using this ref macro in your project:

{% macro ref(
    parent_model, 
    current_model=this.name if this is defined, 
    prod_database=var("upstream_prod_database", None), 
    prod_schema=var("upstream_prod_schema", None),
    enabled=var("upstream_prod_enabled", True),
    fallback=var("upstream_prod_fallback", False),
    env_schemas=var("upstream_prod_env_schemas", False)
) %}

    {% if current_model %}
        {% do return(upstream_prod.ref(parent_model, current_model, prod_database, prod_schema, enabled, fallback, env_schemas)) %}
    {% else %}
        {% do return(builtins.ref(parent_model)) %}
    {% endif %}

{% endmacro %}
LewisDavies commented 1 year ago

Fixed in version 0.5.0