search

fivetran / dbt_hubspot_source

Data models for Hubspot built using dbt.
https://fivetran.github.io/dbt_hubspot_source/
Apache License 2.0
31 stars 30 forks source link

[Feature] Provide Support for HubSpot API v3 Updates #111

Closed fivetran-joemarkiewicz closed 1 year ago

fivetran-joemarkiewicz commented 1 year ago

Is there an existing feature request for this?

Describe the Feature

In May 2023 there were changes to the HubSpot connector that resulted in a myriad of changes to tables in order to support the HubSpot API v3. See the full list of changes to the connector here.

This feature will be to offer support for this new version of the connector. However, we will also likely want to ensure the older versions of the package works with previous connector build. Will consult with internal teams on best approach going forward.

Describe alternatives you've considered

No response

Are you interested in contributing this feature?

Anything else?

No response

sbailliez commented 1 year ago

Impacted by this issue. Would love to see a quick resolution and also in the future maybe for closer collaboration between the connector team and the dbt package maintainers

fivetran-joemarkiewicz commented 1 year ago

Hi @sbailliez thanks for sharing that you are also experiencing this issue. I am currently working on a resolution and hope to share an update as soon as possible.

I also agree with your sentiment on our teams working in closer collaboration. We are working with our internal teams to identify a better solution going forward for my team to be made aware of these changes sooner so we can make the appropriate changes before downstream users are impacted.

sbailliez commented 1 year ago

Thank you @fivetran-joemarkiewicz! Much appreciated!

fivetran-joemarkiewicz commented 1 year ago

Hi @sbailliez I wanted to reach out and let you know that I have been able to explore updating the package to be compatible with the API updates that are already present within the connector.

I was able to test the following locally and found success on the new API upgraded tables. If you have a moment, would you be able to test the following version of the package. You can test it by swapping out the official dbt hub version in your packages.yml for the below:

packages:
  - git: https://github.com/fivetran/dbt_hubspot.git
    revision: feature/hubspotv3-api-update
    warn-unpinned: false

Be sure to run a dbt clean && dbt deps to grab these updates before running the WIP branch. Let me know how it goes!

sbailliez commented 1 year ago

@fivetran-joemarkiewicz This looks like it is resolved. Could not check everything to confirm but for the immediate problem this seems to do it. Thank you for the super quick turnaround on this.

fivetran-joemarkiewicz commented 1 year ago

@sbailliez thanks so much for testing and confirming this works to resolve your immediate issue.

I will continue forward with polishing up the branch and open a PR. I do want to mention that I was not able to find a way to make these changes fully compatible with the previous version of the HubSpot connector. So when we ship this, it will be a requirement to be on the latest HubSpot connector. I am going to sync with our connector team to understand the rollout of the updates to existing connectors.

That being said, you likely should be able to see a new release of the package with these changes by late next week or early the week after. I will be sure to share updates here as this progresses.

fivetran-joemarkiewicz commented 1 year ago

Hi All,

I have been able to fully flesh out the HubSpot API updates within the package and now have a fully functioning branch that is currently in review! If you wanted to test these changes with your implementation, that would be great. Please note the branch has slightly changes from the one I shared previously.

packages:
  - git: https://github.com/fivetran/dbt_hubspot.git
    revision: feature/hubspotv3-api-update-new
    warn-unpinned: false

These changes were made to be both compatible with the old version and the new api version of the connector. Let me know if you see any issues when testing it out!

tashlinr commented 1 year ago

Hey Joe, testing this out now. Missed this until recently, but first glance, looks like its working as intended. Thanks!

fivetran-joemarkiewicz commented 1 year ago

Hi all! I wanted to share that the HubSpot v3 updates have all been pushed to the dbt_hubspot_source and dbt_hubspot packages! They should be live on the dbt hub at the top of the hour.

Thank you for your patience and helping validate these changes as we developed them. As these updates are now live, I will close out this ticket. Thanks again!