This PR will address the following Issue/Feature:v0.8.0
This PR will result in the following new package version: Issue #35 and Issue #33
This is not necessarily a breaking change. However, the removal of the dependency on dbt-expecations and the tests associated with it are enough that I would prefer we make these major updates so it doesn't cause any hiccups following release.
Please detail what change(s) this PR introduces and any additional information that should be known during the review of this PR:
🚨 Breaking Changes 🚨
In October 2023 the Intercom API upgraded from 2.9 to 2.10 which resulted in the the connector schema has seeing updates. These updates have downstream impacts on the data models within this package. The following changes are a result of the Intercom API and connector upgrades:
The _fivetran_deleted column within the *_history staging models have been removed as they were deprecated in the connector.
Feature Updates
The _fivetran_active, _fivetran_start, and _fivetran_end fields have been added to all *_history staging models to address the removal of the _fivetran_deleted columns and properly track historical records.
Test Updates
Removal of the "author is not null" test within the stg_intercom__conversation_history model.
Updated the source freshness test enablement/disablement to leverage the dbt-core config: enabled: architecture.
Dependency Updates
Removal of the dbt_expectations dependency.
PR Checklist
Basic Validation
Please acknowledge that you have successfully performed the following commands locally:
[X] dbt compile
[X] dbt run –full-refresh
[X] dbt run
[X] dbt test
[X] dbt run –vars (confirmed the source freshness tests worked as expected with the various variables disabled)
Before marking this PR as "ready for review" the following have been applied:
[X] The appropriate issue has been linked and tagged
[X] You are assigned to the corresponding issue and this PR
[x] BuildKite integration tests are passing
Detailed Validation
Please acknowledge that the following validation checks have been performed prior to marking this PR as "ready for review":
[X] You have validated these changes and assure this PR will address the respective Issue/Feature.
[X] You are reasonably confident these changes will not impact any other components of this package or any dependent packages.
[X] You have provided details below around the validation steps performed to gain confidence in these changes.
In order to validate these changes performed the following:
I confirmed that the dbt test command worked without error as the test has been removed and dependency on dbt expectations has been removed as well.
I confirmed the source freshness didn't encounter any strangeness when running with the sources disabled.
Please note that I realized this is not working as expected in the latest dbt-core v1.7.0. I have opened an issue on dbt core that raises this question.
I confirmed the API updates were appropriate by confirming with Ray that the removal of _fivetran_deleted and addition of _fivetran_active, _fivetran_start, and _fivetran_end were appropriate.
Additionally, I was able to validate the outputs of the modified staging models via the new connector with the API upgrades were working as expected and the all tests were still properly passing.
Extensive testing of the staging models was not performed as the changes were fairly straight forward with the removal of the _fivetran_deleted and the addition of the other fields. Additionally, I feel fairly comfortable and confident that these changes are appropriate and no further validation is required.
Additionally, please see the Hex notebook for downstream validations linked in the Height ticket.
Standard Updates
Please acknowledge that your PR contains the following standard updates:
Package versioning has been appropriately indexed in the following locations:
[X] indexed within dbt_project.yml
[X] indexed within integration_tests/dbt_project.yml
[X] CHANGELOG has individual entries for each respective change in this PR
[X] README updates have been applied (if applicable)
[n/a] DECISIONLOG updates have been updated (if applicable)
[n/a] Appropriate yml documentation has been added (if applicable)
dbt Docs
Please acknowledge that after the above were all completed the below were applied to your branch:
[x] docs were regenerated (unless this PR does not include any code or yml updates)
If you had to summarize this PR in an emoji, which would it be?
PR Overview
This PR will address the following Issue/Feature:
v0.8.0
This PR will result in the following new package version: Issue #35 and Issue #33
This is not necessarily a breaking change. However, the removal of the dependency on dbt-expecations and the tests associated with it are enough that I would prefer we make these major updates so it doesn't cause any hiccups following release.
Please detail what change(s) this PR introduces and any additional information that should be known during the review of this PR:
🚨 Breaking Changes 🚨
_fivetran_deleted
column within the*_history
staging models have been removed as they were deprecated in the connector.Feature Updates
_fivetran_active
,_fivetran_start
, and_fivetran_end
fields have been added to all*_history
staging models to address the removal of the_fivetran_deleted
columns and properly track historical records.Test Updates
stg_intercom__conversation_history
model.config: enabled:
architecture.Dependency Updates
PR Checklist
Basic Validation
Please acknowledge that you have successfully performed the following commands locally:
Before marking this PR as "ready for review" the following have been applied:
Detailed Validation
Please acknowledge that the following validation checks have been performed prior to marking this PR as "ready for review":
In order to validate these changes performed the following:
I confirmed that the
dbt test
command worked without error as the test has been removed and dependency on dbt expectations has been removed as well.I confirmed the source freshness didn't encounter any strangeness when running with the sources disabled.
I confirmed the API updates were appropriate by confirming with Ray that the removal of
_fivetran_deleted
and addition of_fivetran_active
,_fivetran_start
, and_fivetran_end
were appropriate._fivetran_deleted
and the addition of the other fields. Additionally, I feel fairly comfortable and confident that these changes are appropriate and no further validation is required.Standard Updates
Please acknowledge that your PR contains the following standard updates:
dbt Docs
Please acknowledge that after the above were all completed the below were applied to your branch:
If you had to summarize this PR in an emoji, which would it be?
ℹ️