search

arnoN7 / dbt-incremental-stream

DBT Package reproducing dbt incremental materialization leveraging on Snowflake streams
MIT License
23 stars 4 forks source link

support quoted databases and schemas (relevant for Snowflake adapter) #12

Open MarthaScheffler opened 2 days ago

MarthaScheffler commented 2 days ago

I am using quoted databases on Snowflake, and I used to quote them myself using "\"quoted-database-name\"" all places. However, the dbt-snowflake adapter also support setting the quoting parameter to true instead (https://docs.getdbt.com/reference/project-configs/quoting).

When I use the quoting parameter, the run code for streams looks like this:

CREATE STREAM IF NOT EXISTS quoted-database-name.schema_name.S_s__accounts_r__accounts ON TABLE "quoted-database-name".schema_name.r__accounts;

So the database for the TABLE gets correctly quoted, while the database for the STREAM doesn't. which results in an error, because Snowflake can't handle the names (that's why I need to quote them in the first place).

I got around that issue, as mentioned above, by not using the quoting parameter, and instead quoting the database myself. However, that doesn't work with seeds: https://github.com/dbt-labs/dbt-core/issues/10611

Now I am in a situation, where I can either have seeds OR incremental streams (or would need to rename databases, which I can't).

Maybe it is somehow possible to get the quoting of databases and schemas correctly applied to the streams?