LewisDavies / upstream-prod

A dbt package for easily using production data in a development environment.
32 stars 4 forks source link

`ref()` returns `None` when prod table doesn't exist even if `upstream_prod_fallack: True` #4

Closed garfieldthesam closed 1 year ago

garfieldthesam commented 1 year ago

This might be a misunderstanding of how the package works so please let me know if this is just an unsupported feature.

Using dbt-bigquery I have a config like this in my profiles.yml for upstream_prod:

vars:
  # upstream-prod config
  upstream_prod_schema: prod_schema_prefix
  upstream_prod_enabled: True
  upstream_prod_fallack: True

Where prod_schema_prefix matches the prefix applied to the many prod schemas in our data warehouse (eg. there's my personal dev schemas like dbt_myname_staging and prod schemas like prod_schema_prefix_staging).

I have created a new table tbl1 and updated tbl2 to query from tbl1 like from {{ ref("tbl1") }}. When this compiles with upstream_prod_enabled: True, it compiles as from None.

LewisDavies commented 1 year ago

Thanks for flagging this! No misunderstanding, there's a typo in the last variable: the B is missing, it should be upstream_prod_fallback. I've fixed this on main and will create a new release soon. As long as tbl1 has been built in your dev environment, the package should work exactly as you expect.

Just FYI, upstream_prod_enabled defaults to True and should only need to be set if you want to temporarily disable the package globally.