z3z1ma / dbt-osmosis

Provides automated YAML management, a dbt server, streamlit workbench, and git-integrated dbt model output diff tools
https://z3z1ma.github.io/dbt-osmosis/
Apache License 2.0
422 stars 46 forks source link

Allow docs block descriptions to propagate un rendered. #128

Closed VDFaller closed 3 months ago

VDFaller commented 4 months ago

Problem

When I do dbt-osmosis yaml document whatever if it has a docs block upstream, it ends up putting the compiled markdown. I'd love it if it could do just keep it the docs block. I looked for closed issues around docs blocks so I hope I'm not duplicating something already done.

dbt-databricks==1.6.6 dbt-core==1.6.9 dbt-osmisis==0.12.4

If you think this is doable, I'm happy to help with the MR.

Edit Looking further it looks like it reads from the manifest. Which is fully compiled so this isn't a layup. But maybe we could pull in the node.original_file_path then manually load the columns descriptions? Not sure if it's antithetical to your goals.

node = {"database": "my_db",
    "schema": "my_schema",
    "name": "my_source",
    "resource_type": "source",
    "package_name": "my_dbt",
    "path": "models\\staging\\my_schema\\_my_schema__sources.yml",
    "original_file_path": "models\\staging\\my_schema\\_my_schema__sources.yml", 
    "source_name": "my_schema"
}

with Path(node['original_file_path']).open() as f:
    data = yaml.safe_load(f)

source_dict = next((item for item in data['sources'] if item['name'] == node['source_name']), None)
model_dict = next((item for item in source_dict['tables'] if item['name'] == node['name']), None)
z3z1ma commented 4 months ago

Yep your on the money. We would need to read the YAML and store the raw form. It might be something I can work on but also open to PR

VDFaller commented 4 months ago

I'll try to make some time to work on it this week. Anything I need to do special to contribute?

I'm thinking probably here is the right place to plug in?

z3z1ma commented 3 months ago

Commented on the PR