search

dbt-labs / dbt-core

dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
https://getdbt.com
Apache License 2.0
9.72k stars 1.61k forks source link

[Feature] Clarify error when file(s) contain duplicate macros #10387

Open gwenwindflower opened 3 months ago

gwenwindflower commented 3 months ago

Is this a new bug in dbt-core?

Current Behavior

Reported from a user in Community Slack.

These two listed files contain duplicate macros, but confuse users because we list the file names, not the macros. Not only that, but in my repro I discovered that we always say "two" instead of how many duplicates actually exist and only report the first two files that conflict. That last bit may not be fixable given that we error out when it happens, but I propose that ideally we should say something like:

dbt found [n] macros named "cents_to_dollars" in the project "jaffle shop".
To fix this error, rename or remove duplicates, they can be found in these files:
  - macros/hi_doug.sql
  - macros/hi_jer.sql
  - macros/cents_to_dollars.sql

My repro error from straight Core:

Screenshot 2024-07-01 at 9 56 31 AM

User from Slack's error (looks like dbt Power User): image

Expected Behavior

I would expect dbt to give me a clearer error re which files contain all the duplicate macros

Steps To Reproduce

  1. In dbt Core
  2. Create two or three macros in different files with the same name
  3. Run a dbt compile

Relevant log output

dbt found two macros named "some_custom macro_with_sql" in the project "uft_dbt". To fix this error, rename or remove one of the following macros: - macros get_incremental_merge_insert_sq.sql - macros get _incremental_merge_update_sql.sql

Environment

- OS: MacOS 14
- Python: 3.11.8
- dbt: 1.8.0-b3

Which database adapter are you using with dbt?

snowflake

Additional Context

No response

dbeatty10 commented 3 months ago

Thanks for raising this @gwenwindflower !

TLDR

We could definitely take a look at rephrasing this error message so that it is easier to understand and take action upon. The relevant code is here and here.

We've been reserving the bug label for when dbt is behaving in an obviously incorrect way and then using enhancement when dbt has room to improve somehow (like a more helpful error message). So I'm going to re-label this as a help wanted enhancement / feature request.

Proposed error message

What do you think about one of these variants as the new error message?

When the macro definitions are in different files:

dbt found multiple macros named "cents_to_dollars" in the project "jaffle shop".
To fix this error, rename or remove duplicates. They can be found in these files:
  - macros/hi_doug.sql
  - macros/hi_jer.sql

When the macro definitions are in the same file:

dbt found multiple macros named "cents_to_dollars" in the project "jaffle shop".
To fix this error, rename or remove duplicates. They can be found in this file:
  - macros/cents_to_dollars.sql

Note: It looks to me like dbt will immediately stop and raise an error message upon the first duplicate (rather than collecting all of them first). That's why I replaced "[n] macros" with "multiple macros" for practical purposes.

gwenwindflower commented 3 months ago

thank you @dbeatty10 yea i couldn't decided between bug and enhancement — good to know for the future! i can take a crack at this i love a great error message!