search

fivetran / dbt_hubspot_source

Data models for Hubspot built using dbt.
https://fivetran.github.io/dbt_hubspot_source/
Apache License 2.0
31 stars 30 forks source link

Soft delete staging models #95

Closed kylemcnair closed 1 year ago

kylemcnair commented 1 year ago

Are you a current Fivetran customer? Yes

Kyle McNair, Product Expert, Datafold

What change(s) does this PR introduce?

This PR persists soft deleted records, rather than hard deleting records where is_deleted or _fivetran_deleted is true.

As described in this comment, this change will need corresponding updates to the transform package.

I'm opening this PR as a draft until a corresponding PR is ready for the transform package.

Did you update the CHANGELOG?

Does this PR introduce a breaking change?

Did you update the dbt_project.yml files with the version upgrade (please leverage standard semantic versioning)? (In both your main project and integration_tests)

Is this PR in response to a previously created Bug or Feature Request

How did you test the PR changes?

Select which warehouse(s) were used to test the PR

Provide an emoji that best describes your current mood

:face_with_spiral_eyes: **Feedback** We are so excited you decided to contribute to the Fivetran community dbt package! We continue to work to improve the packages and would greatly appreciate your [feedback](https://www.surveymonkey.com/r/DQ7K7WW) on our existing dbt packages or what you'd like to see next.
fivetran-sheringuyen commented 1 year ago

Happy Thursday @kylemcnair, this is awesome, thank you so much for creating this PR! I have also started working on this issue this sprint and have managed to make quite a bit of headway on the PRs for it as well. I noticed your emoji (😵‍💫) and I totally understand, HubSpot happens to be quite a large package! I would, however, love to continue working with you on these PRs and it looks like I might have taken some minor steps in addition to Jamie's game plan.

If you have any time, I welcome you to take a peak at the PRs I've created for this issue and would love to hear any feedback you may have! Here are the PRs:

Additionally, it'd be great to get a double check that these PRs are producing the data you're expecting. You can try the package by adding the below to your dbt project's packages.yml:

  - git: https://github.com/fivetran/dbt_hubspot.git
    revision: feature/include-deleted-records
    warn-unpinned: false

Please feel free to let me know if you have any questions!

kylemcnair commented 1 year ago

Hey @fivetran-sheringuyen, glad you're already on this - apologies that I missed your existing work. I'd be super happy to help test out the new package, and close this PR. I'll hop over to those PRs and provide any feedback there.