Closed fivetran-joemarkiewicz closed 11 months ago
Thanks much! Will it be released soon?
I am very close to releasing this! Likely Monday morning at this point. I heard from our product team that these additional fields (the non property_hs_
fields) were due to a bug in the connector and we may want to ignore the non property_hs_
fields altogether. I am just waiting on confirmation from our product team if I should keep the coalesce or remove it entirely.
Thanks much! Will it be released soon?
I am very close to releasing this! Likely Monday morning at this point. I heard from our product team that these additional fields (the non
property_hs_
fields) were due to a bug in the connector and we may want to ignore the nonproperty_hs_
fields altogether. I am just waiting on confirmation from our product team if I should keep the coalesce or remove it entirely.
From looking at my own data, they seem to be all nulls, so I think removing or coalescing would have the same effect (though I guess slightly more efficient to avoid the coalesce)
PR Overview
This PR will address the following Issue/Feature: dbt_hubspot Issue 119
This PR will result in the following new package version:
v0.12.0
This will technically be a breaking change since it will remove (via a coalesce) existing impacted fields if duplicates are identified. As such, I would feel more comfortable with this being a breaking change so customers are aware of the upgrade being applied.
Please detail what change(s) this PR introduces and any additional information that should be known during the review of this PR:
🚨 Breaking Changes 🚨
property_hs_
prefix from the source columns in the staging models. If a column with the prefix removed matches the same name as an existing column (for exampleproperty_hs_meeting_outcome
andmeeting_outcome
are both fields in the source table), then the new macro will coalesce the fields giving preference to theproperty_hs_
field as this is likely the most relevant field per the latest HubSpot API upgrade.stg_hubspot__engagement_call
stg_hubspot__engagement_company
stg_hubspot__engagement_contact
stg_hubspot__engagement_deal
stg_hubspot__engagement_email
stg_hubspot__engagement_meeting
stg_hubspot__engagement_note
stg_hubspot__engagement_task
stg_hubspot__ticket
stg_hubspot__ticket_company
stg_hubspot__ticket_contact
stg_hubspot__ticket_deal
stg_hubspot__ticket_engagement
stg_hubspot__ticket_property_history
Feature Updates
remove_duplicate_and_prefix_from_columns
has been included which expands off thefivetran_utils.remove_prefix_columns
macro by removing any duplicate columns that result from the prefix removal.PR Checklist
Basic Validation
Please acknowledge that you have successfully performed the following commands locally:
Before marking this PR as "ready for review" the following have been applied:
Detailed Validation
Please acknowledge that the following validation checks have been performed prior to marking this PR as "ready for review":
These steps were validated by recreating the issue with the seed data for our integration tests and also via validation with the customer that the fix does in fact resolve the error they are seeing.
Standard Updates
Please acknowledge that your PR contains the following standard updates:
dbt Docs
Please acknowledge that after the above were all completed the below were applied to your branch:
If you had to summarize this PR in an emoji, which would it be?
2️⃣