fivetran / dbt_intercom_source

Data source models for Fivetran's Intercom connector built using dbt.
https://fivetran.github.io/dbt_intercom_source/
Apache License 2.0
3 stars 7 forks source link

conversation_part_history assign_to_id failing dbt_expectations null test even though description states it can be null #20

Closed pearse-infogrid closed 1 year ago

pearse-infogrid commented 1 year ago

Is there an existing issue for this?

Describe the issue

The yaml from stg_intercom.yml in the intercom_source package is causing this error upon dbt build. I can remove it and build locally sometimes (can't really figure out why) but then when I commit my changes, it fails the slim CI dbt build --select modified+.

I would like a way to remove the test in the first screenshot without it returning upon a slim CI build

Screenshot 2022-11-07 at 11 29 10 Screenshot 2022-11-07 at 14 01 32

Relevant error log or model output

No response

Expected behavior

A test I have to deleted from my code not to appear in a my slim CI build

dbt Project configurations

`# Name your project! Project names should contain only lowercase characters

and underscores. A good package name should reflect your organization's

name or the intended use of these models

name: 'infogrid_bi' version: '1.0.0' config-version: 2

This setting configures which "profile" dbt uses for this project.

profile: 'default'

These configurations specify where dbt should look for different types of files.

The model-paths config, for example, states that models in this project can be

found in the "models/" directory. You probably won't need to change these!

model-paths: ["models"] analysis-paths: ["analysis"] test-paths: ["tests"] seed-paths: ["data"] macro-paths: ["macros"] snapshot-paths: ["snapshots"]

target-path: "target" # directory which will store compiled SQL files clean-targets: # directories to be removed by dbt clean

We need to grant usage on schemas to users that will need to select from objects in those schemas.

This is because although dbt creates schemas at the start of runs, it isn't possible to configure schemas as their own objects in dbt (see: https://docs.getdbt.com/blog/configuring-grants)

on-run-end:

Configuring models

Full documentation: https://docs.getdbt.com/docs/configuring-models. Note, grants currently have to be at the user (not group) level.

models: +grants: select: ['dbt_cloud', 'hex', 'looker']

Materialize all views as late-binding to reduce Looker downtime. See https://docs.getdbt.com/reference/resource-configs/redshift-configs#late-binding-views

+bind: false

Tables output as a result of package runs go into separate schemas.

These outputs are then transferred to the analytics schema as needed. This helps reduce model clutter.

hubspot: +schema: dbt_packages_output hubspot_source: +schema: dbt_packages_source xero: +schema: dbt_packages_output xero_source: +schema: dbt_packages_source lever: +schema: dbt_packages_output lever_source: +schema: dbt_packages_source intercom: +schema: dbt_packages_output intercom_source: +schema: dbt_packages_source

Tables that we've built for final analysis go in the analytics schema as tables

Staging models should only contain light transformation so these are materialized as views

Intermediate models often contain heavy transformations. These are materialized as tables for query performance reasons. They are stored in a separate schema to avoid clutter.

infogrid_bi: athena_sensor_events: 01_staging: schema: 01_staging materialized: view 02_intermediate: schema: 02_intermediate materialized: table 03_core: materialized: table

aws_cost_report:
  01_staging:
    schema: 01_staging
    materialized: view

reporting:
  materialized: table

hubspot:
  01_staging:
    schema: 01_staging
    materialized: view
  02_intermediate:
    schema: 02_intermediate
    materialized: table
  03_core:
    materialized: table

lever:
  01_staging:
    schema: 01_staging
    materialized: view
  02_intermediate:
    schema: 02_intermediate
    materialized: table
  03_core:
    materialized: table

hibob:
  01_staging:
    materialized: view
  02_intermediate:
    schema: 02_intermediate
    materialized: table
  03_core:
    materialized: table

xero:
  01_staging:
    schema: 01_staging
    materialized: view
  # 02_intermediate:
  #   schema: 02_intermediate
  #   materialized: table
  03_core:
    materialized: table

company_targets:
  01_staging:
    materialized: view
  # 02_intermediate:
  #   schema: 02_intermediate
  #   materialized: table
  03_core:
    materialized: table

webapp_rds:
  01_staging:
    schema: 01_staging
    materialized: table
  02_intermediate:
    schema: 02_intermediate
    materialized: table
  03_core:
    materialized: table

seeds: infogrid_bi: hibob_people_daily_audit_historic_seed: +column_types: employee_id: varchar manager_s_id: varchar

vars: segment: segment_page_views_table: "{{ source('segment', 'pages') }}"

Setting custom options for Fivetran Hubspot dbt package

hubspot__pass_through_all_columns: true hubspot_email_event_spam_report_enabled: false hubspot_email_event_print_enabled: false hubspot_email_event_forward_enabled: false hubspot_contact_merge_audit_enabled: true

Setting custom options for Fivetran Intercom dbt package

intercomusing_conversation_tags: false intercomusing_company_tags: false

Setting the source data location for Fivetran dbt packages

hubspot_database: bi_database hubspot_schema: hubspot_fivetran lever_database: bi_database lever_schema: lever_fivetran xero_database: bi_database xero_schema: xero_fivetran intercom_database: bi_database intercom_schema: intercom segment_database: bi_database segment_schema: webapp

`

Package versions

`packages:

What database are you using dbt with?

redshift

dbt Version

dbt v1.2.0-latest

Additional Context

No response

Are you willing to open a PR to help address this issue?

pearse-infogrid commented 1 year ago

completed through dbt slack channel