fivetran / dbt_hubspot

Data models for Hubspot built using dbt.
https://fivetran.github.io/dbt_hubspot/
Apache License 2.0
33 stars 38 forks source link

[Bug] Integer overflow #139

Closed stdiopt closed 4 months ago

stdiopt commented 5 months ago

Is there an existing issue for this?

Describe the issue

DBT transformation fails with an integer overflow most likely related to the cast to int where it could be dbt.type_bigint()?

https://github.com/fivetran/dbt_hubspot/blob/main/models/marketing/intermediate/int_hubspot__contact_merge_adjust.sql#L22

Relevant error log or model output

11:43:06
Database Error in model int_hubspot__contact_merge_adjust (models/marketing/intermediate/int_hubspot__contact_merge_adjust.sql)
11:43:06
Overflow (Integer valid range -2147483648 to 2147483647)
11:43:06
compiled Code at target/run/hubspot/models/marketing/intermediate/int_hubspot__contact_merge_adjust.sql

Expected behavior

No error in sync

What database are you using dbt with?

redshift

dbt Version

1.5.1

Are you willing to open a PR to help address this issue?

fivetran-jamie commented 5 months ago

hey there @stdiopt thanks for taking the time to open this issue!

i think you're certainly right that we'll need to cast as a bigint. i've made the change in a testing branch if you'd like to test it out:

# packages.yml -- replace hubspot reference with the below
packages:
  - git: https://github.com/fivetran/dbt_hubspot.git
    revision: bug/cast-vid_to_merge-bigint
    warn-unpinned: false 

also i don't think this really changes things at all, but do you have the hubspot_contact_merge_audit_enabled variable set to true?

thanks!

stdiopt commented 5 months ago

Hey Jamie,

also i don't think this really changes things at all, but do you have the hubspot_contact_merge_audit_enabled variable set to true?

to be honest I'm not entirely sure since this is managed by fivetran and based on quickstart transformations, I just disabled those transformations now since I noticed that we aren't using it

fivetran-jamie commented 5 months ago

Ah ok, the hubspot_contact_merge_audit_enabled variable is false by default and I believe Quickstart wouldn't change it to true.

As for testing, since you're on Quickstart, we'll work to recreate the issue on our end, see if the cast as {{ dbt.type_bigint() }} does the trick, and prioritze the fix for an upcoming sprint 🤠

fivetran-reneeli commented 5 months ago

Hi @stdiopt , we have been able to recreate the error and find the fix on our end. We'll be including this fix in our upcoming sprint, so keep an eye out for a new release!

EPNickBosma commented 4 months ago

🙏 this is currently biting us -- any indication of when 0.17.1 might be released please?

fivetran-reneeli commented 4 months ago

Hi @EPNickBosma , thanks for the patience-- we've just merged the branch with fix, so expect the release within the next few hours!

cc @stdiopt