z3z1ma
commented
1 year ago Valid statement. However I think overwriting descriptions should only be done explicitly. So what we really want here is a --force-inheritance flag that can be used in conjunction with some fine grains selection criteria.
I think we can piggyback of dbt's API or use a subprocess with dbt ls to enable graph selectors in dbt-osmosis. Which in combination with a force inheritance flag, would be a decent solution.
cc @ProkhorovAlexander
Hi,
I apologize in advance if my statement is not valid or if this issue has already been raised before. I wasn't able to find it myself.
I understand that column descriptions don't overwrite the descriptions that already exist downstream. This behavior is suitable for the first run and for runs involving newly created columns.
However, if a column description was previously propagated and later changed, the downstream descriptions remain unchanged.
I believe there should be an option to compare previously created artifacts and detect changes in a similar way to how DBT does it with the --defer selector. In this case, if one of the parent models has a changed description and all of its children previously had the same description as the parent, it should be safe to overwrite the children's descriptions as well.
I hope my suggestion is clear and it would be great to have this functionality implemented.
Thank you for your attention.