airbytehq / airbyte-dbt-models

Models to be used to leverage Airbyte output connector data.
MIT License
5 stars 15 forks source link

Create dbt converter for Airbyte Recurly source connector #24

Closed marcosmarxm closed 2 months ago

marcosmarxm commented 4 months ago

The goal of this task is to ensure the Airbyte Recurly connector output can be used to run Fivetran's dbt analytical model.

Definition of Done:

Resources:

btkcodedev commented 2 months ago

Hi @marcosmarxm I would like to take this issue

btkcodedev commented 2 months ago

Step 1 Analysis:

The following tables are to be analyzed with recurly in https://github.com/fivetran/dbt_recurly_source/tree/main/models

image

btkcodedev commented 2 months ago

Doubt 1 @marcosmarxm We in airbyte have only account_notes for fivetran's recurly_account_overview and recurly_account_daily_overview, Also several columns are missing here? What should we do further

https://fivetran.github.io/dbt_recurly/#!/model/model.recurly.recurly__account_overview#columns

Airbyte: image

Fivetran: image

btkcodedev commented 2 months ago

Update: transformation from accounts stream could be used for account_balance

btkcodedev commented 2 months ago

URL: https://fivetran.github.io/dbt_recurly/#!/model/model.recurly_source

Fivetran Airbyte
stg_recurly__account_balance_history.sql Trasformation from accounts
stg_recurly__account_history.sql accounts
stg_recurly__account_note_history.sql account_notes
stg_recurly__billing_info_history.sql billing_info
stg_recurly__coupon_discount.sql coupons
stg_recurly__coupon_redemption_history.sql ❌ it doesn't exist
stg_recurly__credit_payment_history.sql credit_payments
stg_recurly__invoice_coupon_redemption_history.sql ❌ it doesn't exist
stg_recurly__invoice_history.sql invoices
stg_recurly__invoice_subscription_history.sql ❌ it doesn't exist
stg_recurly__line_item_history.sql line_items
stg_recurly__plan_currency_history.sql ❌ it doesn't exist
stg_recurly__plan_history.sql plans
stg_recurly__subscription_add_on_history.sql ❌ it doesn't exist
stg_recurly__subscription_change_history.sql ❌ it doesn't exist
stg_recurly__subscription_history.sql subscriptions
stg_recurly__transaction.sql transactions
stg_recurly__transaction_subscription.sql ❌ it doesn't exist
marcosmarxm commented 2 months ago

@btkcodedev for the missing tables can you confirm objects/json exist in parent table?

stg_recurly__subscription_change_history.sql | subscriptions -> change field?