search

fivetran / dbt_hubspot

Data models for Hubspot built using dbt.
https://fivetran.github.io/dbt_hubspot/
Apache License 2.0
33 stars 38 forks source link

[Issue-109] Optimize int_hubspot__contact_merge_adjust for redshift users #110

Closed kcraig-ats closed 1 year ago

kcraig-ats commented 1 year ago

Please provide your name and company

Kris Craig, ActBlue

Link the issue/feature request which this PR is meant to address

Issue #109 - Poor performance on int_hubspot__contact_merge_adjust

Detail what changes this PR introduces and how this addresses the issue/feature request linked above.

Updated the unnest logic for redshift users in the merge_contacts macro. Replace cross join logic redshift's PartiQL syntax.

How did you validate the changes introduced within this PR?

Tested locally with a custom model.

Which warehouse did you use to develop these changes?

Redshift

Did you update the CHANGELOG?

Did you update the dbt_project.yml files with the version upgrade (please leverage standard semantic versioning)? (In both your main project and integration_tests)

Provide an emoji that best describes your current mood

🍙 **Feedback** We are so excited you decided to contribute to the Fivetran community dbt package! We continue to work to improve the packages and would greatly appreciate your [feedback](https://www.surveymonkey.com/r/DQ7K7WW) on our existing dbt packages or what you'd like to see next. **PR Template** - [Community Pull Request Template](?expand=1&template=pull_request_template.md) (default) - [Maintainer Pull Request Template](?expand=1&template=maintainer_pull_request_template.md) (to be used by maintainers)
fivetran-jamie commented 1 year ago

hey thanks so much for opening this! this works perfectly for me on our (limited) sample data. curious- how much did this cut your own runtime? @kcraig-ats

kcraig-ats commented 1 year ago

hey thanks so much for opening this! this works perfectly for me on our (limited) sample data. curious- how much did this cut your own runtime? @kcraig-ats

@fivetran-jamie runtime dropped from 15 minutes to ~1 minute.

fivetran-jamie commented 1 year ago

sick! i'll go ahead and merge into the release branch. thanks so much for contributing

kcraig-ats commented 1 year ago

@fivetran-jamie I saw the check failure. Did I do something wrong/ do I need to need to do anything more?

fivetran-jamie commented 1 year ago

ah no ignore that! it's just a reminder to us to regenerate docs before releasing

fivetran-jamie commented 1 year ago

this is in v0.10.0 of the package!