fivetran / dbt_fivetran_log

Data models for Fivetran's internal log connector built using dbt.
https://fivetran.github.io/dbt_fivetran_log/
Apache License 2.0
30 stars 24 forks source link

Bug/postgres performance #126

Closed fivetran-catfritz closed 3 months ago

fivetran-catfritz commented 3 months ago

PR Overview

This PR will address the following Issue/Feature:

This PR will result in the following new package version:

Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:

Performance Improvements

  • Updated the sequence of JSON parsing for model fivetran_platform__audit_table to reduce runtime.

Bug Fixes

  • Updated model fivetran_platform__audit_user_activity to correct the JSON parsing used to determine column email.

Under the hood

  • Updated logic for macro fivetran_log_lookback to align with logic used in similar macros in other packages.

PR Checklist

Basic Validation

Please acknowledge that you have successfully performed the following commands locally:

Before marking this PR as "ready for review" the following have been applied:

Detailed Validation

Please share any and all of your validation steps:

where cast(created_at as date) < cast('2024-05-10' as date)
and cast(created_at as date) >= cast('2024-05-07' as date)

which produces these results:

v1.7.2 run: ~ 1hr

Screenshot 2024-05-11 at 2 17 09 AM

v1.7.3 run: ~ 10 mins

Screenshot 2024-05-11 at 1 00 29 AM

If you had to summarize this PR in an emoji, which would it be?

🌳