fivetran / dbt_fivetran_log

Data models for Fivetran's internal log connector built using dbt.
https://fivetran.github.io/dbt_fivetran_log/
Apache License 2.0
30 stars 24 forks source link

[Feature] Update docs to make clarify sync times #128

Closed fivetran-reneeli closed 2 months ago

fivetran-reneeli commented 3 months ago

Is there an existing feature request for this?

Describe the Feature

The sync times in FIVETRAN_PLATFORMAUDIT_TABLE are not the most recent sync times that are found in the source table because we only take record when a table is being written to. See [this CTE](https://github.com/fivetran/dbt_fivetran_log/blob/5fef3647a47e682236713c274895905ec5a81643/models/fivetran_platformaudit_table.sql#L89-L93) in that model that has the event_subtype limiter.

In conclusion, if there is no corresponding event logged that a table is written to, the sync time doesn't get persisted (that’s why the most recent sync times are present in the staging log table but not in our audit model since we only show the timestamps for when write events occur).

We should make this more clear for the fields in the docs for FIVETRAN_PLATFORM__AUDIT_TABLE related to sync times.

Describe alternatives you've considered

No response

Are you interested in contributing this feature?

Anything else?

No response

fivetran-joemarkiewicz commented 2 months ago

This has been addressed in the latest v1.8.0 release!