LewisDavies / upstream-prod

A dbt package for easily using production data in a development environment.
32 stars 4 forks source link

Support custom databases #38

Open davidwitk opened 3 months ago

davidwitk commented 3 months ago

Custom schemas are supported, but not custom databases (only partially with the prod_database_replace feature).

In our project, we write and read to and from different databases. This setup is currently not possible with upstream_prod (a model fails when it selects models from different databases).

We've added support for this by copying the solution for custom schemas.

  1. Adding the macro get_custom_database.sql:

    
    {% macro generate_database_name(custom_database_name=none, node=none, is_upstream_prod=False) -%}
    
    {%- set default_database = target.database -%}
    
    {%- if (target.name in ('prod') or is_upstream_prod == true) and custom_database_name is not none -%}
    
        {{ custom_database_name | trim }}
    
    {%- else -%}
    
        {{ default_database }}
    
    {%- endif -%}

{%- endmacro %}


2. Adding this to the `upstream_prod.ref` macro: 

{% set custom_database_name = parent_node.config.database %} {% set parent_database = generate_database_name(custom_database_name, parent_node, True) | trim %}



It would be cool if `upstream_prod` supports this out-of-the-box 🙂