Update/ad-reporting-v2

[fivetran-sheringuyen]

Pull Request Are you a current Fivetran customer? Fivetran employee

What change(s) does this PR introduce? Breaking Changes PR [] makes the below updates that may affect your workflow:

• modified_timestamp columns have been renamed to modified_at and is_most_recent_version has been renamed to is_most_recent_record to reflect more recent package coding standards for the below models:
  • stg_microsoft_ads__account_history
  • stg_microsoft_ads__ad_group_history
  • stg_microsoft_ads__ad_history
  • stg_microsoft_ads__ad_performance_daily_report
  • stg_microsoft_ads__campaign_history
• Deprecating *_version_id fields in *_history models.

Feature Enhancements We have added the below feature enhancements to this package in (PR #)[]:

• Add get_*_columns macros for previously included models and newly added models.
• Updated staging model standards on old models to conform with recent package development standards. Updates were made to the below models:
  • stg_microsoft_ads__account_history
  • stg_microsoft_ads__ad_group_history
  • stg_microsoft_ads__ad_history
  • stg_microsoft_ads__ad_performance_daily_report
  • stg_microsoft_ads__campaign_history
• New history and daily performance staging models including:
  • stg_microsoft_ads__account_daily_report
  • stg_microsoft_ads__campaign_daily_report
  • stg_microsoft_ads__ad_group_daily_report
  • stg_microsoft_ads__search_daily_report
  • stg_microsoft_ads__keyword_daily_report
  • stg_microsoft_ads__keyword_history
• README updates for easier navigation and use of the package.
• Addition of identifier variables for each of the source tables to allow for further flexibility in source table direction within the dbt project.
• More robust testing for better data integrity including:
  • Freshness tests
  • Model grain tests
• Passthrough metric functionality for more flexibility in reporting.
• Additional documentation for new models added.

Did you update the CHANGELOG?

• [x] Yes

Does this PR introduce a breaking change?

• [ ] Yes (please provide breaking change details below.)
• [x] No (please provide an explanation as to how the change is non-breaking below.)

Did you update the dbt_project.yml files with the version upgrade (please leverage standard semantic versioning)? (In both your main project and integration_tests)

• [ ] Yes

Is this PR in response to a previously created Bug or Feature Request

N/A

How did you test the PR changes?

• [x] CircleCi
• [x] Local (please provide additional testing details below)

Select which warehouse(s) were used to test the PR

• [x] BigQuery
• [x] Redshift
• [x] Snowflake
• [x] Postgres
• [x] Databricks
• [ ] Other (provide details below)

Provide an emoji that best describes your current mood

:dancer: **Feedback** We are so excited you decided to contribute to the Fivetran community dbt package! We continue to work to improve the packages and would greatly appreciate your [feedback](https://www.surveymonkey.com/r/DQ7K7WW) on our existing dbt packages or what you'd like to see next.

[fivetran-sheringuyen]

Hey @fivetran-joemarkiewicz, re:

I really like the depth you put into the staging model tests. I do wonder if this would break if individuals add more metrics via the passthrough columns? Is this something we should consider?

I think in my head I've only been thinking of customers using that variable to passthrough metrics - in which case, this shouldn't be a problem at all. If they are passing through non-metrics, this would be a concern we should consider -- however, it would also break some of the sum( {{ metric }} ) logic that we have incorporated. But that would open another can of worms one that I think can be avoided since we do name the variables *_metrics so there should be the assumption that passthrough columns should only be metrics.

Additionally, before adding changes for the macro + CTE suggestions, I'd like to discuss that as a team first and walk through the nuances.

[fivetran-joemarkiewicz]

Hey @fivetran-joemarkiewicz, re:

I really like the depth you put into the staging model tests. I do wonder if this would break if individuals add more metrics via the passthrough columns? Is this something we should consider?

I think in my head I've only been thinking of customers using that variable to passthrough metrics - in which case, this shouldn't be a problem at all. If they are passing through non-metrics, this would be a concern we should consider -- however, it would also break some of the sum( {{ metric }} ) logic that we have incorporated. But that would open another can of worms one that I think can be avoided since we do name the variables *_metrics so there should be the assumption that passthrough columns should only be metrics.

Additionally, before adding changes for the macro + CTE suggestions, I'd like to discuss that as a team first and walk through the nuances.

@fivetran-sheringuyen this makes sense to me! Maybe we can add a small clarifying statement in the README to only include metrics in this passthrough variable. This way there is nothing left up to interpretation (although it does seem very obvious to me haha).

I also agree that we should discuss the passthrough + macro behavior closer as a team. We can do so during standup this morning!

[fivetran-sheringuyen]

Hey @fivetran-joemarkiewicz! Just updated the repo for PR changes. Let me know if you have any follow up questions!