[Feature] `docs generate` - retrieve column descriptions from DB when available - for Sources

[eilone]

Is this your first time submitting a feature request?

• [X] I have read the expectations for open source contributors
• [X] I have searched the existing issues, and I could not find an existing issue for this feature
• [X] I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature

I am requesting an enhancement to the dbt docs generate command to automatically retrieve column descriptions for sources from the database (e.g., BigQuery) when they are not provided in the YAML files. This functionality is already partially in place as dbt docs generate accesses the database to retrieve metadata for building the catalog.json. Extending this capability to fetch column descriptions would streamline the documentation process, especially for sources managed by external teams.

Describe alternatives you've considered

1.  Manually maintaining column descriptions in YAML files for all sources.
2.  Developing custom scripts to extract and merge database column descriptions into the dbt project.

Both alternatives add significant manual overhead and are prone to errors and inconsistencies.

Who will this benefit?

This feature will benefit dbt users who manage projects with external data sources, particularly in environments where:

•   Sources are managed by other teams.
•   Column descriptions are maintained within the database itself.
•   Consistent and up-to-date documentation is critical.

Example use case: A data engineering team using BigQuery with tables owned by different departments. Automatically retrieving column descriptions will ensure accurate and current documentation without duplicating efforts across teams.

Are you interested in contributing this feature?

Not really, but I'd love to help if needed

Anything else?

• dbt docs generate documentation

This feature will leverage the existing infrastructure in dbt docs generate to enhance the documentation capabilities, providing a more seamless and integrated experience for users.

[fcavallaro-reply]

that would be super useful!

[eilone]

Any update?