fivetran / dbt_zendesk_source

Fivetran's Zendesk Support source dbt package
https://fivetran.github.io/dbt_zendesk_source/#!/overview
Apache License 2.0
13 stars 18 forks source link

BUG - Freshness checks fail for sources whose tables are excluded from sync #18

Closed wuservices closed 3 years ago

wuservices commented 3 years ago

Are you a current Fivetran customer? Jon Wu, Architect, Paper Culture

Describe the bug

When ticket form history isn't being used, freshness is still checked and subsequently causes an error. It should be ignored.

Steps to reproduce

  1. In dbt_project.yml disable form history with using_ticket_form_history: False. In addition, don't select this table for sync in Fivetran so that it doesn't exist.
  2. Run dbt source snapshot-freshness
  3. You'll see an error like ERROR freshness of zendesk.ticket_form_history followed by info saying that the table doesn't exist. For example, something like this for BigQuery:
    Runtime Error in source ticket_form_history (models/src_zendesk.yml)
    404 GET https://bigquery.googleapis.com/bigquery/v2/projects/[project]/queries/[query-id]?maxResults=0&location=US&prettyPrint=false: Not found: Table [project]:zendesk.ticket_form_history was not found in location US

Expected behavior

When you set using_ticket_form_history: False, freshness shouldn't be checked. Ideally the source isn't even defined so that it doesn't appear in the docs.

Project variables configuration

name: [project-name]
version: 1.0.0
config-version: 2

profile: default
source-paths: [models]
analysis-paths: [analysis]
test-paths: [tests]
data-paths: [data]
macro-paths: [macros]
snapshot-paths: [snapshots]
target-path: target
clean-targets:
  - target
  - dbt_modules

vars:
  zendesk_source:
    zendesk_database: [project]
  # Disable models for Zendesk features we aren't using or syncing
  using_ticket_form_history: False
  using_schedules: False

Package Version

packages:
  - package: fishtown-analytics/dbt_utils
    version: 0.6.6
  - package: fivetran/ad_reporting
    version: 0.2.2
  - package: fivetran/facebook_ads
    version: 0.2.1
  - package: fivetran/google_ads
    version: 0.2.1
  - package: fivetran/marketo
    version: 0.4.0
  - package: fivetran/microsoft_ads
    version: 0.2.1
  - package: fivetran/pinterest
    version: 0.3.2
  - package: fivetran/zendesk
    version: 0.5.1

Warehouse

- [x] BigQuery - [ ] Redshift - [ ] Snowflake - [ ] Postgres - [ ] Databricks - [ ] Other (provide details below) **Additional context** Similar to https://github.com/fivetran/dbt_hubspot/issues/41 **Screenshots**

Please indicate the level of urgency

Testing for the first time, but given that this was fixed in another package, it seems like a fix is known.

Are you interested in contributing to this package?

fivetran-jamie commented 3 years ago

included in v0.4.1!