Added the following staging models, along with documentation and tests:
stg_hubspot__property
stg_hubspot__property_option
Bug fixes
Updated macro remove_duplicate_and_prefix_from_columns to accommodate incoming custom column names containing characters such as - or $ that are not permitted. The resulting column name will have these characters removed or replaced in its stg_* model.
Removed extra comma from stg_hubspot__ticket, which was causing compilation issues when passing through all columns.
PR Checklist
Basic Validation
Please acknowledge that you have successfully performed the following commands locally:
[x] dbt run –full-refresh && dbt test
[ ] ~dbt run (if incremental models are present)~
Before marking this PR as "ready for review" the following have been applied:
[x] The appropriate issue has been linked, tagged, and properly assigned
[x] All necessary documentation and version upgrades have been applied
[x] docs were regenerated (unless this PR does not include any code or yml updates)
[x] BuildKite integration tests are passing
[x] Detailed validation steps have been provided below
Detailed Validation
Please share any and all of your validation steps:
See ticket for validation link.
Additional Comments
I feel confident that my solution works, however I do realize adding it to the source is open for discussion. I chose this path because these 4 target tables (companies, contacts, deals, tickets) all diverge in logic in the transform with no clear pattern, so I'd have to be very mindful of when to bring in these fields and wonder how maintenance would be in the long run.
While this does require joins in the source, I felt it would be acceptable since it would only be enabled if the user enables it, and this way it's easy to see where these labels are being brought in alongside the other passthrough columns.
Again open for discussion, but just wanted to give a little insight into what I was thinking.
PR Overview
This PR will address the following Issue/Feature:
121
This PR will result in the following new package version:
Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:
🚨 Breaking Change 🚨
property_hs_*
column, you now have the option to include the corresponding, human-readable label in the staging models.stg_hubspot__company
stg_hubspot__contact
stg_hubspot__deal
stg_hubspot__ticket
Features
stg_hubspot__property
stg_hubspot__property_option
Bug fixes
remove_duplicate_and_prefix_from_columns
to accommodate incoming custom column names containing characters such as-
or$
that are not permitted. The resulting column name will have these characters removed or replaced in itsstg_*
model.stg_hubspot__ticket
, which was causing compilation issues when passing through all columns.PR Checklist
Basic Validation
Please acknowledge that you have successfully performed the following commands locally:
Before marking this PR as "ready for review" the following have been applied:
Detailed Validation
Please share any and all of your validation steps:
Additional Comments