search

fivetran / dbt_zendesk

Fivetran's Zendesk Support dbt package
https://fivetran.github.io/dbt_zendesk/#!/overview
Apache License 2.0
26 stars 30 forks source link

[Feature] Support dbt-dry-run #91

Open tim-summerton-brier opened 1 year ago

tim-summerton-brier commented 1 year ago

Is there an existing feature request for this?

Describe the Feature

We use dbt-dry-run when deploying our dbt code to check for any errors. On a dbt project using this package, dbt-dry-run will fail if you try to run it before the models have been run. Once the models have been run once and generated some data in BigQuery, the dry run will succeed.

Please can you support dbt-dry-run without having to run the models prior to the dry run?

Describe alternatives you've considered

No response

Are you interested in contributing this feature?

Anything else?

One error from the dry runner is this case statement:

https://github.com/fivetran/dbt_zendesk/blob/3af26a7d19e6cb14c0c8a7aa7b230c4cec0d46fb/models/ticket_history/int_zendesk__field_calendar_spine.sql#L24

It complains the two return values are of different types. No matching signature for operator CASE; all THEN/ELSE arguments must be coercible to a common type but found: STRING, TIMESTAMP; actual argument types (WHEN THEN) ELSE: (BOOL TIMESTAMP) STRING at [162:14]

I believe the fix for this issue is to wrap updated_at with a cast to timestamp, but I've not tested this.

Another set of errors it raises is it can't find any of the fields from ticket_field_history_columns in the schema of upstream models when trying to dry dun model.zendesk.zendesk__ticket_backlog. I imagine the root cause of this is that these fields are not physical columns in the source data. This will require further investigation from someone more familiar with this package.

fivetran-joemarkiewicz commented 1 year ago

Hi @tim-summerton-brier thanks for opening this feature request!

We typically only provide support for our package within the framework of native dbt Core functionality. I have never used dbt-dry-run myself, but curious to see what components are not compatible with this package. I actually wonder, are you able to run dbt-dry-run with other modeling type packages? The error it is showing you currently with the case statement seems like it is an issue with reading the dbt macro and not understanding it is a timestamp 🤔.

I wonder if there may be some cases where dbt-dry-run was not intended to work natively with dbt packages.

tim-summerton-brier commented 1 year ago

We currently have dbt-dry-runner working on a dbt project using the fivetran_log package, but we introduced the dry runner later on in that project, so data had already been generated. I'm not sure if the dry runner would have worked prior to generating data with fivetran_log.

The workaround we've employed for our current project is to disable the dry runner, deploy and run our dbt project to generate the data, then re-enable the dry runner.