fivetran / dbt_hubspot_source

Data models for Hubspot built using dbt.
https://fivetran.github.io/dbt_hubspot_source/
Apache License 2.0
31 stars 30 forks source link

Feature/hubspotv3 api update #112

Closed fivetran-joemarkiewicz closed 1 year ago

fivetran-joemarkiewicz commented 1 year ago

PR Overview

This PR will address the following Issue/Feature: Issue #111

This PR will result in the following new package version: v0.11.0

This will be a breaking change as it drastically changes how staging models are structured to bring in respecitive fields for downstream analysis. Due to the API v3 changes (see our release notes here from the connector), many tables were impacted on how they sync data. As such, we should make this a breaking change and require customers to ensure they are on the latest version of the HubSpot connector in order to leverage this package.

I do also want to call out that I was able to make these changes compatible with the old version of the connector (pre v3 API update). That being said, when using the old version of the connector with this package, you will see some fields are not documented as they have been removed from the newer API version. Likewise, due to the backwards compatibility, there are some fields that will always be null for users of the new API. These fields have been identified in the yml documentation and notified for deprecation and removal once all connectors are upgraded (next few months).

Please detail what change(s) this PR introduces and any additional information that should be known during the review of this PR:

🚨 Breaking Changes 🚨

Documentation Updates

PR Checklist

Basic Validation

Please acknowledge that you have successfully performed the following commands locally:

I ran various versions of the package with variables enabled and disabled. In particular I tested to ensure the changed models worked when enabled and disabled.

Before marking this PR as "ready for review" the following have been applied:

Detailed Validation

Please acknowledge that the following validation checks have been performed prior to marking this PR as "ready for review":

This package update has been validated to ensure it works as expected on the new v3 API update while also working on the older api version. Please see the corresponding Height ticket for the schemas used to validate. Additionally, I was able to gain confidence in these changes due to the customer in the linked Issue claiming the fix resolved upgrade issues they were experiencing.

Please note that these changes had downstream impacts. Please see the respective dbt_hubspot PR for notes on how this was validated.

Standard Updates

Please acknowledge that your PR contains the following standard updates:

dbt Docs

Please acknowledge that after the above were all completed the below were applied to your branch:

If you had to summarize this PR in an emoji, which would it be?

🙃
fivetran-joemarkiewicz commented 1 year ago

@fivetran-catfritz thanks for the review! I just responded and made appropriate updates based on your above comments. Let me know if you have further questions. Before I move forward I wanted to get your eyes on my new approach to the macros (your initial comment here).

If that looks good to you then I will apply to the other models and regen the docs. If not, let me know!