DataRecce / recce

The dbt data-validation toolkit for teams that care about building better data
https://datarecce.io
Apache License 2.0
252 stars 6 forks source link

[DRC-591] [Feature] Specify different database connections (or dbt target) for base and current environments #400

Open aryaphan opened 2 months ago

aryaphan commented 2 months ago

Describe the feature

Hello,

Currently I think Recce interprets the base and current environments as same database but different schemas. When I run recce server, it only accepts one --target and connects to that one database. But I use different databases for development and production. This means if Recce can only connect to one database at a time, the advanced diffs, such as Value Diff, will always show no change in this case since it reads from the same db for my base and current environments.

Is it possible to add a feature where we can specify which dbt target used for base and which dbt target used for current environment?

Describe alternatives you've considered

To prepare for my base environment

dbt docs generate --target prod --target-path target-base

To prepare for my development environment

dbt run --target dev
dbt docs generate --target dev

When I use Recce

recce server --target dev

Note here that my dev and prod databases are 2 different hosts. At this point, it only connects to the dev db host. But my production and development db have same database and schema names. So when I do Value DIff, it basically runs the same query on the same database => shows no change.

Value Diff is only useful when I use a different schema on the same db as my current environment.

Who will this benefit?

Organizations that use different databases for development, staging, production environments.

Are you interested in contributing this feature?

No response

Anything else?

No response

DRC-591