fivetran / dbt_fivetran_log

Data models for Fivetran's internal log connector built using dbt.
https://fivetran.github.io/dbt_fivetran_log/
Apache License 2.0
30 stars 25 forks source link

Update/rename package #81

Closed fivetran-jamie closed 1 year ago

fivetran-jamie commented 1 year ago

PR Overview

This PR will address the following Issue/Feature:

height task T-390568 https://github.com/fivetran/dbt_fivetran_log/issues/83

This PR will result in the following new package version:

considering we are renaming the package, v1.0.0?

Please detail what change(s) this PR introduces and any additional information that should be known during the review of this PR:

what this does NOT do:

PR Checklist

Basic Validation

Please acknowledge that you have successfully performed the following commands locally:

Before marking this PR as "ready for review" the following have been applied:

Detailed Validation

Please acknowledge that the following validation checks have been performed prior to marking this PR as "ready for review":

ran the two versions of the package simultaneously (since they have different project + model names). I looked at various models, but here I will focus on the audit table model, since i updated the the Bigquery + Spark incremental strategy for that model

  1. ran a full refresh and spot checked different tables + connectors, but here are the total aggregates compared

    image
  2. ran an incremental run

    image

everything checks out there -- should i validate any other way.. ?

Standard Updates

Please acknowledge that your PR contains the following standard updates:

dbt Docs

Please acknowledge that after the above were all completed the below were applied to your branch:

If you had to summarize this PR in an emoji, which would it be?

📛
fivetran-jamie commented 1 year ago

@fivetran-joemarkiewicz added your changes! also for validation, i added some extra lines to the seed file

So, running before adding new dates:

image

I then add these new rows to the log seed file to simulate a sync of the media_insights table the next day (2021-12-10)

image

i dbt seed --full-refresh

i perform an incremental run, resulting in the following expected output (a new record for media_insights)

image