fivetran_platform__audit_table.unique_table_sync_key definition should include "schema_name"

[abine-michael-planting]

Is there an existing issue for this?

• [X] I have searched the existing issues

Describe the issue

The current definition for building fivetran_platform__audit_table.unique_table_sync_key is:

{{ dbt_utils.generate_surrogate_key(['connector_id', 'destination_id', 'table_name', 'write_to_table_start']) }}

It should include schema_name as well since the same table name can occur across different schemas is the same connector, destination, table and write to table start.

Relevant error log or model output

FAIL 20196 unique_fivetran_platform__audit_table_unique_table_sync_key

Expected behavior

If unique_table_sync_key definition is updated to include schema_name, no error will return.

dbt Project configurations

fivetran_platform_database: 'fivetran' # this is where the source fivetran logs, same for prod and dev fivetran_platform_schema: 'fivetran_log' # this is where the source fivetran logs, same for prod and dev fivetran_platform_using_transformations: false # this will disable all transformation + trigger_table logic fivetran_platform_using_triggers: false # this will disable only trigger_table logic fivetran_platform_using_account_membership: false # this will disable only the account membership logic fivetran_platform_using_destination_membership: false # this will disable only the destination membership logic fivetran_platform_using_user: false # this will disable only the user logic

Package versions

• package: fivetran/fivetran_log version: 1.1.0

What database are you using dbt with?

snowflake

dbt Version

1.3

Additional Context

No response

Are you willing to open a PR to help address this issue?

• [ ] Yes.
• [ ] Yes, but I will need assistance and will schedule time during our office hours for guidance
• [X] No.

[fivetran-joemarkiewicz]

Hi @abine-michael-planting thank you for raising this observation with our team. I agree that the schema name should be included in the unique key to verify true uniqueness of these records. This should be a relatively easy addition we can include in our dbt package.

We will plan to make this update in our upcoming sprint (starting mid next week). Be sure to follow along here for updates from our team regarding expected release of this update. Thanks again for raising this to our team!

[abine-michael-planting]

Thanks!

[fivetran-joemarkiewicz]

Hi all, this issue has since been resolved in the latest v1.2.0 release of the Fivetran Log package. This version of the package should be live on the dbt hub at the top of the hour. Please remember to run a --full-refresh following the upgrade so the incremental model will make retroactive changes to the surrogate key which was updated.

Please let us know if this issue still persists following your upgrade. Thanks again for raising this to our team and for @JustMaris for opening the PR!

[abine-michael-planting]

Thank you! Which model folder should I run a full-refresh on?

[fivetran-joemarkiewicz]

You can run a full refresh on the fivetran log folder

[abine-michael-planting]

Do you know how I can run a full refresh in DBT Cloud? The models created by the package live in the target/compiled folder, or the dbt_packages/... when i try to refresh either of the model folder within those directories, dbt returns a message stating that no models are found. I can only run dbt run.... on the project/models folder. Thanks for the help.

[abine-michael-planting]

I got it figured out, I ran: dbt run -s fivetran_log --full-refresh. Thanks again for getting this fixed.