I would like to request a functionality that will allow user to define project variables (https://docs.getdbt.com/docs/build/project-variables) to filter set of rows from the source table without impacting other rows
example
lets say the underlying source contains clients info, so each row will represent a particular client. now we have a dbt model that looks like this
select * from {{}} where client_id= '{{ var("client_id") }}'
We like to perform now dbt run --vars {{client_id='abc'}} so that this job can only capture client ABC records from the source table and not all others
SQL behind the scene --> select * from client where client_id='abc'
since streams are one-time process only, so we like to add this if-else logic to handle above use case
if dbt model has a project variable configured etc {{ var("client_id") }}' then
create a tmp view from the source (eg:create or replace view v_base_raw_client_abc as select * from client where client_id='abc)
create stream on top of it (eg: S_target_client_raw_client_abc)
create view on top of a stream (eg: V_raw_client_target_client_abc)
use this view to MERGE into target. (eg; V_target_client_abc)
drop the stream view S_raw_client_target_client_abc
I would like to request a functionality that will allow user to define project variables (https://docs.getdbt.com/docs/build/project-variables) to filter set of rows from the source table without impacting other rows
example
lets say the underlying source contains clients info, so each row will represent a particular client. now we have a dbt model that looks like this
select * from {{}} where client_id= '{{ var("client_id") }}'
We like to perform now
dbt run --vars {{client_id='abc'}}
so that this job can only capture client ABC records from the source table and not all othersSQL behind the scene -->
select * from client where client_id='abc'
since streams are one-time process only, so we like to add this
if-else
logic to handle above use case{{ var("client_id") }}'
thencreate or replace view v_base_raw_client_abc as select * from client where client_id='abc
)S_target_client_raw_client_abc
)V_raw_client_target_client_abc
)V_target_client_abc)
S_raw_client_target_client_abc