[CT-2808] [Feature] Support hiding all resources from a package (not just models)

[jeremyyeo]

Is this your first time submitting a feature request?

• [X] I have read the expectations for open source contributors
• [X] I have searched the existing issues, and I could not find an existing issue for this feature
• [X] I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature

This is an extension of https://github.com/dbt-labs/dbt-core/issues/1671

Currently, we can hide docs of models in packages by doing:

# dbt_project.yml
name: my_dbt_project

models:
  local_package:
    +docs:
      show: false
  my_dbt_project:
....

However, this causes package macros to still show up in the docs site. Since macros is not a top level key in the dbt_project.yml file (https://docs.getdbt.com/reference/dbt_project.yml), you can't hide docs of a packages macros easily. And of course, most dbt packages include macros :)

We can try this out quite simply:

# dbt_project.yml
name: my_dbt_project
profile: snowflake
config-version: 2
version: 1.0

models:
  my_dbt_project:
    +materialized: table

# packages.yml
packages:
  - local: ../some_package
  - package: brooklyn-data/dbt_artifacts
    version: 2.3.0

# ../some_package/dbt_project.yml
name: "some_package"
version: "0.1.0"
config-version: 2

-- ../some_package/models/bar.sql
select 1 id

Now let's hide those docs!

# dbt_project.yml
name: my_dbt_project
profile: snowflake
config-version: 2
version: 1.0

models:
  some_package:
    +docs:
      show: false
  dbt_artifacts:
    +docs:
      show: false
  my_dbt_project:
    +materialized: table

^ There is no longer a folder for some_package since it only contains models but still one for dbt_artifacts - cause of the macros.

Describe alternatives you've considered

Not too sure how one go about hiding entire package macros from docs today.

Who will this benefit?

Anyone who wants to remove some clutter from their dbt docs site.

Are you interested in contributing this feature?

No response

Anything else?

Trying to put a top level macros key like so:

# dbt_project.yml
name: my_dbt_project
profile: snowflake
config-version: 2
version: 1.0

models:
  some_package:
    +docs:
      show: false
  dbt_artifacts:
    +docs:
      show: false
  my_dbt_project:
    +materialized: table

macros:
  dbt_artifacts:
    +docs:
      show: false

Results in:

$ dbt docs generate

01:02:26  Running with dbt=1.5.2
01:02:27  Encountered an error:
Runtime Error
  at path []: Additional properties are not allowed ('macros' was unexpected)

Error encountered in /Users/jeremy/src/dbt-sandcastles/dbt/snowflake/dbt_project.yml

[dbeatty10]

Thanks for such a nice write-up @jeremyyeo 🤩

It seems reasonable to enable configs for macros within dbt_project.yml as you described:

# dbt_project.yml

...

macros:
  dbt_artifacts:
    +docs:
      show: false

This could include the docs config shown in the screenshot below as well as any other current or future configs for macros (like meta).

---

@jtcohen6 Is there any reason why we wouldn't want to add macros as a top level key in dbt_project.yml?

[jtcohen6]

@jtcohen6 Is there any reason why we wouldn't want to add macros as a top level key in dbt_project.yml?

Just that macros don't yet / haven't historically supported any other configs that could be set in dbt_project.yml. Sounds like this might want to be the first!

[logankopas]

Is there a reason the docs config isn't implemented for sources? I have a use-case where I want to hide most of my sources (since they are mostly duplicate schemas and they clutter up the documentation site).

[dbeatty10]

@logankopas I don't know of a specific reason why the docs config wasn't implemented for sources.

Looking at the resource types listed for the docs config, some are listed as being supported within dbt_project.yml, and some are not:

• ✅ Models
• ❌ Sources
• ✅ Seeds
• ✅ Snapshots
• ❌ Analyses
• ❌ Macros

macros is the only one not currently possible to add to dbt_project.yml.

sources and analyses are possible to add to dbt_project.yml, but setting the docs config does not have any effect (to the best that I can tell).

From inspecting manifest.json, it looks like the docs property for analyses is inherited from models 🤯

Acceptance criteria

• any resource that can be displayed in a generated documentation website can have show: true | false via the docs within dbt_project.yml
• after dbt parse, the configuration of docs within dbt_project.yml is reflected within target/manifest.json
• after dbt docs generate && dbt docs serve, when show: false, then all applicable resources will be hidden from the directory tree within the generated website
• docs for one resource type doesn't affect any other resource type

Here's an example of "macros" and "analyses" displaying within a documentation website:

[logankopas]

@dbeatty10 Thanks for the info.

For sources specifically the behaviour is:

• If the docs property is set in dbt_project.yml the property is largely ignored. It is added to manifest.json under the unrendered_config key, but is unused.
• If the docs property is set directly on the source config, a parsing error is raised:

  Parsing Error
  Invalid sources config given in models/sources.yml @ sources: {'name': 'test', 'schema': 'test', 'docs': {'show': False}, 'tables': [{'name': 'test'}]} - at path []: Additional properties are not allowed ('docs' was unexpected)

[brunodeprez]

Any news on this issue. I also want to hide macros from my doc

[hvignolo87]

Any news on this issue. I also want to hide macros from my doc

Check this out

[dbeatty10]

@brunodeprez

This feature is not a priority for us right now. Implementing this feature correctly requires fairly extensive knowledge of our codebase, so we're not encouraging pull requests from the community on this one either.

As a result, using a workaround like @hvignolo87 suggested is your best bet: https://github.com/dbt-labs/dbt-core/issues/8061#issuecomment-2138408151