LewisDavies / upstream-prod

A dbt package for easily using production data in a development environment.
32 stars 4 forks source link

'dict object' has no attribute 'file_key_name' in macro ref (macros/ref.sql) #16

Closed ViniciusRaphael closed 1 year ago

ViniciusRaphael commented 1 year ago

Hi there, We've updated our upstream-prod to version 0.6.1 a few days ago, since then when we try to use dbt compile we face an error. The issue does not happen when upstream-prod is set as False.

ERROR:

20:12:23  Running with dbt=1.5.1
20:12:32  Found 449 models, 645 tests, 4 snapshots, 2 analyses, 1144 macros, 3 operations, 0 seed files, 139 sources, 35 exposures, 0 metrics, 0 groups
20:12:32  
20:12:32  Concurrency: 4 threads (target='dev')
20:12:32  
20:12:35  Encountered an error:
Runtime Error
  Compilation Error in model int_active_...  (folder/int_active_...sql)
    'dict object' has no attribute 'file_key_name'

    > in macro ref (macros/ref.sql)
    > called by macro ref (macros/ref.sql)
    > called by model int_active_... (folder/int_active_...sql)

P.S: I've hidden the model and folder because the error is showing different model names when we test with different analysts.

LewisDavies commented 1 year ago

Thanks for opening an issue! Does your project have any singular tests where the ref is hardcoded? From some quick reading it seems like they don't have a file_key_name on the graph, which would cause the error you're seeing.

LewisDavies commented 1 year ago

@ViniciusRaphael I've opened #17 that should fix this. Would you mind testing the fix with your project? You can do this by changing the packages.yml entry to:

packages:
  - git: https://github.com/LewisDavies/upstream-prod.git
    revision: fix-singular-tests
ViniciusRaphael commented 1 year ago

Hi Lewis, The revision version worked. Dbt compile ran fine.

About your first question: Thanks for opening an issue! Does your project have any [singular tests](https://docs.getdbt.com/docs/build/tests#singular-tests) where the ref is hardcoded? From some quick reading it seems like they don't have a file_key_name on the graph, which would cause the error you're seeing.

We are using this package and in order to run it we had to change our ref macro

  - package: EqualExperts/dbt_unit_testing
    version: 0.2.7
MACRO REF
{% macro ref(
    parent_model, 
    v=1,
    prod_database=var("upstream_prod_database", 'bp-datawarehouse'), 
    prod_schema=var("upstream_prod_schema", get_inferred_schema(parent_model)),
    enabled=var("upstream_prod_enabled", False),
    fallback=var("upstream_prod_fallback", False),
    env_schemas=var("upstream_prod_env_schemas", False)
) %}

  -- even though the variable "upstream_prod_enabled" was cast to bool in dbt_project.yml, there is a bug that var("upstream_prod_enabled") is returning a string.
  -- https://github.com/dbt-labs/dbt-core/issues/6382
  {% if enabled is string %}
    {% set enabled = (enabled == 'True') %}
  {% endif%}

  {% if 'unit-test' in config.get('tags') %}
    {{ return (dbt_unit_testing.ref_cte_name(parent_model)) }}
  {% else %}
    {% do return(upstream_prod.ref(parent_model, prod_database, prod_schema, enabled, fallback, env_schemas)) %}
  {% endif %}

{% endmacro %}

{% macro source(source, model_name) %}
   {{ return(dbt_unit_testing.source(source, model_name)) }}
{% endmacro %}
LewisDavies commented 1 year ago

Thanks for confirming, I'll merge and release a new version today 🎉

I've not used dbt-unit-testing but I see how that could have caused this error. Great to know that you've got it to play nicely with this package too.