[Bug] manifest generation error

[rafaelmarinm]

Is there an existing issue for this?

• [X] I have searched the existing issues

Describe the issue

I am trying to install the ad reporting model package in fivetran and I have not been able to get it to run. It runs fine in my local environment, but once I try to run it in fivetran I get the same error.

Details:

• dbt configuration passed all checks

• package: fivetran/ad_reporting version: 1.7.0

dbt version: 1.6.6

Error: The following dbt command:

dbt compile
failed during the Manifest Generation process.

The failure details:

Exit code: 2

Output:

07:46:41 Running with dbt=1.6.6
07:46:42 Registered adapter: snowflake=1.6.4
07:46:42 Unable to do partial parsing because saved manifest not found. Starting full parse.
07:47:06 Encountered an error:
Parsing Error
Invalid metrics config given in FilePath(searched_path='models', relative_path='ad_reporting_metrics.yml', modification_time=1674507072.0, project_root='/tmp/dbt/project/dbt_packages/ad_reporting')

The following internal error occurred during the Manifest Generation process:

The 'manifest.json' artifact file not found after executing the command generating the manifest.

Relevant error log or model output

No response

Expected behavior

Deploy ad reporting model

dbt Project configurations

name: 'ad_model'
version: '1.0.0'
config-version: 2

# This setting configures which "profile" dbt uses for this project.
profile: 'ad_model'

# These configurations specify where dbt should look for different types of files.
# The `model-paths` config, for example, states that models in this project can be
# found in the "models/" directory. You probably won't need to change these!
model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

clean-targets:         # directories to be removed by `dbt clean`
  - "target"
  - "dbt_packages"

# Configuring models
# Full documentation: https://docs.getdbt.com/docs/configuring-models

# In this example config, we tell dbt to build all models in the example/
# directory as views. These settings can be overridden in the individual model
# files using the `{{ config(...) }}` macro.
models:
  ad_model:
    # Config indicated by + and applies to all files under models/example/
    example:
      +materialized: view
vars:
    facebook_ads_schema: FACEBOOK_ADS
    facebook_ads_database: FIVETRAN_DATABASE

    google_ads_schema: GOOGLE_ADS
    google_ads_database: FIVETRAN_DATABASE 

    microsoft_ads_schema: BINGADS
    microsoft_ads_database: FIVETRAN_DATABASE

    linkedin_ads_schema: LINKEDIN_ADS 
    linkedin_ads_database: FIVETRAN_DATABASE  

    twitter_ads_schema: TWITTER_ADS
    twitter_ads_database: FIVETRAN_DATABASE 

vars:
  ad_reporting__amazon_ads_enabled: False # by default this is assumed to be True
  ad_reporting__apple_search_ads_enabled: False # by default this is assumed to be True
  ad_reporting__pinterest_ads_enabled: False # by default this is assumed to be True
  ad_reporting__reddit_ads_enabled: False # by default this is assumed to be True
  ad_reporting__snapchat_ads_enabled: False # by default this is assumed to be True
  ad_reporting__tiktok_ads_enabled: False # by default this is assumed to be True

Package versions

package: fivetran/ad_reporting version: 1.7.0

What database are you using dbt with?

snowflake

dbt Version

1.6.6

Additional Context

No response

Are you willing to open a PR to help address this issue?

• [ ] Yes.
• [ ] Yes, but I will need assistance and will schedule time during our office hours for guidance
• [X] No.

[fivetran-joemarkiewicz]

@rafaelmarinm thanks for opening this issue. We have seen similar issues of customers running into the same manifest error when deploying to either Fivetran or dbt Cloud. Since this runs fine locally, there must be something going on when trying to orchestrate the models and the manifest somehow gets in the way.

@Jstein77 do you have any thoughts as to what this manifest error is? It's interesting that this runs normally in a local environment but not when deploying for orchestration?

[Jstein77]

Hey @rafaelmarinm sorry for the delay! I was on vacation for the last 2 weeks. I tried to repro this issue using the same dbt version in a clean venv and wasn't able to - the fivetran package installed successfully and dbt compile ran as well. Are you still blocked here?

[rafaelmarinm]

Hi Jordan! Yes still blocked. If you can we can have a Zoom meeting to go over all the details.

On Wed, Nov 15, 2023 at 1:36 PM Jordan @.***> wrote:

Hey @rafaelmarinm https://github.com/rafaelmarinm sorry for the delay! I was on vacation for the last 2 weeks. I tried to repro this issue using the same dbt version in a clean venv and wasn't able to - the fivetran package installed successfully and dbt compile ran as well. Are you still blocked here?

— Reply to this email directly, view it on GitHub https://github.com/fivetran/dbt_ad_reporting/issues/106#issuecomment-1813147063, or unsubscribe https://github.com/notifications/unsubscribe-auth/A7NVIMGQQUQXOZXZQRAALFTYEUKVNAVCNFSM6AAAAAA6TJXPDWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJTGE2DOMBWGM . You are receiving this because you were mentioned.Message ID: @.***>

[fivetran-joemarkiewicz]

Hi @rafaelmarinm one quick thought, do you have a package-lock.yml in your project? If so, can you try and delete that file, then rerun dbt clean && dbt deps && dbt run to see if that resolves the issue.

I mainly ask, because @fivetran-catfritz on my team noticed the ad_reporting_metrics file that is causing the issue has since been deprecated following one of the latest releases. I think the old version of ad reporting may still be in use due to the package-lock.yml and we just need to reset it.

Let me know!

[Jstein77]

@rafaelmarinm Feel free to book some time to go over this issue! https://calendly.com/jordan-stein-1

[fivetran-joemarkiewicz]

Hi @rafaelmarinm I am curious if my above suggestion worked for you? Otherwise, were you able to meet with @Jstein77 to investigate further?

[fivetran-joemarkiewicz]

I will continue to leave this ticket open in case others have more to share in the thread. However, I will mark the ticket as wontfix since it seems there may not be any updates that need to be applied to the Ad Reporting package.

Please let me know if anyone believes there should be an update to the package. Thanks!

[trueclassic02]

Hi @fivetran-joemarkiewicz, I ran into a similar issue. I'm using dbt version 1.7.3 and tried to run the ad_reporting project on Fivetran. I'm seeing the following error

The node semantic_model.ad_reporting.ad_report not found

It works without any issue if ran locally. Would you be able to help to identify the issue?

[fivetran-joemarkiewicz]

As identified within Issue #108, the root cause of this issue seems to be caused by Fivetran transformations not taking into account the semantic_manifest.json. This is not something that will result in an update within the dbt package itself. Instead, if any others come across this issue, please open a Fivetran Support Ticket.

Thank you!