[Bug] Zendesk Package Prevents Docs from Building in Slim CI in dbt Cloud

[erika-e]

Is there an existing issue for this?

• [X] I have searched the existing issues

Describe the issue

An error in the zendesk package prevents the docs from building if the models in the package aren't run by the PR.

@fivetran-joemarkiewicz suggested a fix in the dbt slack: https://getdbt.slack.com/archives/C01D1R2JLLA/p1659536144595479?thread_ts=1659463574.617639&cid=C01D1R2JLLA

It looks like this fix was implemented: https://github.com/fivetran/dbt_zendesk/commit/0933f9e19941cd47d07b16719bc91e620e5dcedd

and released: https://github.com/fivetran/dbt_zendesk/releases/tag/v0.9.1

I've upgraded to the new package version but I'm still seeing the error.

Relevant error log or model output

16:35:58  Running with dbt=1.2.3
16:35:59  Found 297 models, 323 tests, 8 snapshots, 1 analysis, 914 macros, 1 operation, 11 seed files, 221 sources, 0 exposures, 2 metrics
16:35:59  
16:36:01  Concurrency: 4 threads (target='default')
16:36:01  
Runtime Error
  Database Error in model int_zendesk__calendar_spine (models/utils/int_zendesk__calendar_spine.sql)
    002003 (42S02): SQL compilation error:
    Object 'ANALYTICS.DBT_CLOUD_PR_44565_386_STG.STG_ZENDESK__TICKET' does not exist or not authorized.

Expected behavior

Docs should build on CI.

dbt Project configurations

 zendesk:
    +schema: 
    +materialized: table
    agent_work_time:
      +schema: stg
    intermediate:
      +schema: stg
    reply_times:
      +schema: stg
    resolution_times:
      +schema: stg 
    sla_policy:
      +schema: stg
    ticket_history:
      +schema: stg
  zendesk_source:
    +schema: stg

Package versions

• package: fivetran/zendesk version: 0.9.1

What database are you using dbt with?

snowflake

dbt Version

1.2.0

Additional Context

No response

Are you willing to open a PR to help address this issue?

• [X] Yes.
• [X] Yes, but I will need assistance and will schedule time during our office hours for guidance
• [ ] No.

[fivetran-joemarkiewicz]

Hi @erika-e thanks for creating this issue and sorry to see you are running into an error when trying to build in Slim CI.

That is interesting that you are running into this particular error as our change here should ensure the package is referencing the source as opposed to the staging model (that is probably not yet created in your case).

A few things I am curious about and would like to confirm on your end:

• What command are you running when you run into this error?
• Would you be able to show the dbt deps results to confirm 0.9.1 is successfully installing.
• Do you have a source.yml defined somewhere that is named Zendesk and possibly references the staging model as the ticket table?
• In your dbt_packages folder, do you see the same code for the models/utils/int_zendesk__calendar_spine.sql location as is shown within the latest code on the repo?

Hopefully based on the above, we should be able to identify the core of the issue!

[erika-e]

I am also baffled. 😁 I was excited to see that y'all had pushed a fix and surprised it had not worked.

The commands on our CI job are:

dbt seed --full-refresh
dbt build --select state:modified+ --exclude config.materialized:snapshot

dbt deps result from a CI job:

15:23:38  Installing fivetran/zendesk
15:23:38    Installed from version 0.9.1
15:23:38    Up to date!
15:23:38  Installing fivetran/zendesk_source
15:23:39    Installed from version 0.7.0
15:23:39    Up to date!

Do you have a source.yml defined somewhere that is named Zendesk and possibly references the staging model as the ticket table? We do not.

In your dbt_packages folder, do you see the same code for the models/utils/int_zendesk__calendar_spine.sql location as is shown within the latest code on the repo? yep!

Happy to jump on a call sometime if this isn't enough to sleuth it out, you can reach me in the dbt slack.

[erika-e]

As a workaround I'm just building the problematic model on every CI run. When I do, the docs build.

[fivetran-joemarkiewicz]

Thanks for sharing this additional info @erika-e! I was on PTO last week and will start looking back into this now with the additional details you shared above. If I am unable to uncover the core issue, we can look into setting up a call to try and uncover the problem.

I hope to get back to you this week!

[fivetran-sheringuyen]

Hey @erika-e! I've been catching up on this thread, the slack thread as well as trying to replicate the error in my own environment with your respective dbt versions. I have a couple of follow up questions for you - but have also sent you a slack message to get some availabilities for an office hour some time early next week (hopefully!).

You can feel free to answer these here, or save them for the OH, either way, I figured I'd let you in on where my thinking is going:

1. Could you speak more to your workflow? For example, I'd love to get a step by step on where the models are run, where does CI fit in. On the slack it sounds like models aren't actually being run in the CI but based on your commands, it does look like you are seeding data then doing a dbt build within your CI?

2. Can you confirm that you are indeed getting the same error with v0.9.1 as v0.9.0? It seems odd that the error in int_zendesk__calendar_spine would still reference STG_ZENDESK__TICKET if we are now switching to the source reference in v0.9.1.

I have a couple of ideas that we could try on the call and looking forward to hearing back from you!

[erika-e]

:wave: @fivetran-sheringuyen

1. We run a "standard" slim CI with a deferral to the previous production run in dbt cloud. Models run to PR schemas and only the modified models and their downstream dependencies build. The box to generate docs is checked. In the past though, I haven't been successful with fixing docs bugs like this by adding selectors to the docs generate command.
2. According to our deps we are already on 0.9.1, do you need me to downgrade to 0.9.0 to test?

[fivetran-sheringuyen]

Thanks for that insight @erika-e ! It'll be interesting to see your dbt cloud set up on our call tomorrow.

As for which version, 0.9.1 is perfect! I mostly want to see what the error output is with the new 0.9.1 release and whether it is still outputting the below. It would be a very peculiar situation and may denote a deeper issue since the model below is no longer being referenced in the calendar_spine model:

Runtime Error
  Database Error in model int_zendesk__calendar_spine (models/utils/int_zendesk__calendar_spine.sql)
    002003 (42S02): SQL compilation error:
    Object 'ANALYTICS.DBT_CLOUD_PR_44565_386_STG.STG_ZENDESK__TICKET' does not exist or not authorized.

[fivetran-sheringuyen]

After some discussion, it looks like this is now a non-issue and currently cannot be reproduced on the customer's side so we will be closing out this issue for now.