Create dbt converter for Airbyte Intercom source connector

[marcosmarxm]

The goal of this task is to ensure the Airbyte Intercom connector output can be used to run Fivetran's dbt analytical model.

Definition of Done:

• Create a pull request that adds new tables or views, enabling the execution of Fiventran's dbt with Airbyte output data.
• Update the documentation inside the connector folder to explain how to configure the user dbt package to run the analytical model.

Resources:

• Check example of how it was done for Zendesk Support connector

[nthapa000]

Please assign it to me @marcosmarxm ??

[marcosmarxm]

It is yours!

[nthapa000]

Hey @marcosmarxm I am done with step 1. LMK if any correction is required otherwise I am waiting for your go ahead signal for step 2!! 🙇

Common Streams

Fivetran Stream Name	Airbyte Stream Name
admin	Admins
company_history	Companies
conversation_history	Conversations
conversation_part_history	Conversation Parts
contact_history	Contacts
tag	Tags
team	Teams

Streams Available in Fivetran but Not in Airbyte

Fivetran Stream Name	Airbyte Stream Name
company_tag_history	❌ it doesn't exist
contact_company_history	❌ it doesn't exist
contact_tag_history	❌ it doesn't exist
conversation_contact_history	❌ it doesn't exist
conversation_tag_history	❌ it doesn't exist
team_admin	❌ it doesn't exist

Links : Airbyte Streams Fivetran DBT streams

[marcosmarxm]

Hello @nthapa000 can you check if inside the parent table exist a nested/json object? Example for companies if exist a tag field the same for contacts

[nthapa000]

Here is the updated table @marcosmarxm ,Please take a look 🙇 Am I good to go now?

Common Streams

Fivetran Stream Name	Airbyte Stream Name
admin	Admins
company_history	Companies
conversation_history	Conversations
conversation_part_history	Conversation Parts
contact_history	Contacts
tag	Tags
team	Teams
company_tag_history	tags (Nested object in companies stream)
contact_company_history	companies (Nested object in contacts stream)
contact_tag_history	tags (Nested object in contacts stream)
conversation_contact_history	contacts (Nested object in conversations stream)
conversation_tag_history	tags (Nested object in conversations stream)
team_admin	admin_ids (Array in teams stream)

[marcosmarxm]

Let's go to second step!

[nthapa000]

Hello @marcosmarxm I have done some analysis for the step 2 : Most of the tables dont have corresponding fields for _fivetran_active , _fivetran_start and _fivetran_end.

What do I do about these ? Or can I just ignore these and proceed ahead?

For example

Table 2

Fivetran: models/stg_intercom__company_history.sql Airbyte : models/source/companies.yaml

Fivetran Column Name	Corresponding Airbyte Column Name
id as company_id	id / company_id (verify)
name	name
website	website
industry	industry
created_at	created_at
updated_at	updated_at
user_count	user_count
session_count	session_count
monthly_spend	monthly_spend
plan_id	plan (Nested object)
plan_name	plan (Nested object)
_fivetran_active	❌ Not present
_fivetran_start	❌ Not present
_fivetran_end	❌ Not present

Links: Fivetran DBT and Airbyte DBT

[marcosmarxm]

You can ignore these fields as they aren't used in models.

[nthapa000]

Please check my comments in PR conversation @marcosmarxm !

[marcosmarxm]

I'm gonna take a look sorry the delay

[nthapa000]

Hello @marcosmarxm I might be wrong , but , The table stg_intercom__admin.sql in Fivetran maps to admins.yaml and not teams.yaml. Therefore I think the changes you made are wrong. Instead of {{source('source_intercom', 'teams')}} it should be {{ source('source_intercom', 'admins') }} as I had it before. Moreover you missed job_title !! LMK your views!