search

fivetran / dbt_stripe_source

Fivetran's Stripe source dbt package
https://fivetran.github.io/dbt_stripe_source/
Apache License 2.0
8 stars 27 forks source link

renamed subscription to subscription_history #37

Closed nachimehta closed 2 years ago

nachimehta commented 2 years ago

Pull Request Are you a current Fivetran customer? No. I am a SI. Nachi Mehta, CEO, OxWorks

What change(s) does this PR introduce? Updates subscription table to match new ERD (subscription history) Did you update the CHANGELOG?

Does this PR introduce a breaking change?

Did you update the dbt_project.yml files with the version upgrade (please leverage standard semantic versioning)? (In both your main project and integration_tests)

Is this PR in response to a previously created Bug or Feature Request

How did you test the PR changes?

Select which warehouse(s) were used to test the PR

Provide an emoji that best describes your current mood

:bruh: **Feedback** We are so excited you decided to contribute to the Fivetran community dbt package! We continue to work to improve the packages and would greatly appreciate your [feedback](https://www.surveymonkey.com/r/DQ7K7WW) on our existing dbt packages or what you'd like to see next.
fivetran-joemarkiewicz commented 2 years ago

Hi @nachimehta thanks so much for opening this PR! 🙌

This was actually something I had on my radar as well. However, I was unable to test effectively with the updated connector that contains the history tables.

My only question with this PR is to understand the implications of the naming change. For other packages where we introduced history tables we actually had to account for the historical records. See how we did this within our Salesforce package where we actually filter out the inactive records within the _tmp models.

I will want to check with our product team to understand if these models introduce history mode. If they do, then we will want to filter out the inactive records via the _fivetran_active field. Otherwise, we can move forward with the PR as is! I hope to post back here shortly!

fivetran-joemarkiewicz commented 2 years ago

@nachimehta would you actually be able to confirm if the Stripe data you are working with in fact does have a _fivetran_active field within the subscription_history source table?

nachimehta commented 2 years ago

@fivetran-joemarkiewicz confirmed:

Screen Shot 2022-04-11 at 11 04 54 AM
fivetran-joemarkiewicz commented 2 years ago

@nachimehta thanks so much for confirming this!

A few things I want to adjust before merging this with our branch and cutting a new release. I am actually going to retain the naming of the models to be subscription vs subscription_history because I will plan to filter out the inactive records within the model. Thus the model will not have historical data by default. I will still reference the historical table, but I feel it would be more confusing to name the model historical if it is in fact not. Finally, I am going to mark this as a breaking change since this is a pretty significant update to the package.

With that, I am going to merge this PR with a working branch I will have on our end so we may do further integration testing. It would be great if you could continue to follow the next PR as there will be a possibility for you to test out the changes before we merge and release the next update.

Thanks again so much for your contributions 🏅. The same commentary here will apply to the accompanying PR you opened on our dbt_stripe package.

fivetran-joemarkiewicz commented 2 years ago

@nachimehta I have been able to update the branches to account for all of these changes. Would you be able to test the following branch in your packages.yml and let me know if you see the same success?

packages:
  - git: https://github.com/fivetran/dbt_stripe.git
    revision: bugfix/nachimehta-subscription-history
    warn-unpinned: false

FYI This includes your changes from both the source and transform version of the package.