fivetran / dbt_stripe

Data models for Stripe built using dbt.
https://fivetran.github.io/dbt_stripe/
Apache License 2.0
29 stars 33 forks source link

[Bug] Issue with stripe source on dbt run #31

Closed shaon-chowdhury closed 2 years ago

shaon-chowdhury commented 2 years ago

Is there an existing issue for this?

Describe the issue

When I try to dbt run for the first time, I get the following error because the stripe subscription table does not exist.

Database Error in model stg_stripe__subscription_tmp (models/tmp/stg_stripe__subscription_tmp.sql)
relation "stripe.subscription" does not exist
LINE 7: from "db"."stripe"."subscription"
             ^
 compiled SQL at target/run/stripe_source/models/tmp/stg_stripe__subscription_tmp.sql
Done. PASS=32 WARN=0 ERROR=1 SKIP=4 TOTAL=37

In my database, Fivetran does not export a table named subscription but instead the following tables.

subscription_history
subscription_discount
subscription_item
subscription_tax_rate

Is there an easy fix for this?

Relevant error log or model output

No response

Expected behavior

This should either look for a table with another name or build properly.

dbt Project configurations

Using the latest dbt version as of 28th March and have only installed the stripe package.

Package versions

packages:
  - package: fivetran/stripe
    version: [">=0.6.0", "<0.7.0"]

What database are you using dbt with?

postgres

dbt Version

dbt installed version: 1.0.4

Additional Context

No response

Are you willing to open a PR to help address this issue?

fivetran-joemarkiewicz commented 2 years ago

Hi @shaon-chowdhury thanks so much for opening this issue 👋

If you do not have the subscriptions source table, then you will still be able to leverage this dbt package by disabling the subscription models. To do so, you can add the following to your dbt_project.yml

vars:
    using_subscriptions:   False

Let me know if you have any other questions!

shaon-chowdhury commented 2 years ago

@fivetran-joemarkiewicz thanks for your help! Still confused as to why Fivetran does not export the subscription table but I'll reach out to their support

fivetran-joemarkiewicz commented 2 years ago

Of course, happy to help!

When I have seen tables like this not being synced in the past it is usually because there isn't any data available from the endpoint. However, if you have the other subscriptions tables then I would wonder why the base table isn't being synced either. Reaching out to support would be the best next step.

Thanks!

rhinck commented 2 years ago

Looking at the ERD here, it looks like the subscription table is now subscription_history if I'm not mistaken.

I've been running into the error as @shaon-chowdhury. Is there a way using this package that I can remap subscription to subscription_history?

I just started using dbt today so I apologize if I'm missing something obvious.

fivetran-joemarkiewicz commented 2 years ago

Hi @rhinck you can in fact remap the package to point at the subscription_history table instead. To do so you can add the following to your dbt_project.yml

##dbt_project.yml

vars:
   subscription: "database.schema.subscription_history"

Likewise, if you would like to create your own source file that includes the subscription_history table then you can leverage the dbt source macro, otherwise the above will work just fine.

Further, this is being discussed in more detail within this PR. I hope to have a more permanent solution in place once this PR is merged.