A common feature request has been to see all upstream nodes from an injected project. For some time, this has not been available for a simple technical reason: dbt-core's access checks do not fire if you are not injecting dbt Labs' undocumented dependency structure at runtime. The consequence of this is that all access checks would break, allowing for private models to be ref-ed outside of groups if the referrer is in a different project. Unfortunately, dbt-core is designed in a purposefully obfuscated way to prevent mere mortals for leveraging this depenedency argument, so extra measures were required.
To that end, I've implemented some patches that allow us to wrap the existing reference access checks with injections that allow for dependency checks to occur as expected!
With this out of the way, we now have the following features:
Cross-project DAGs include non-executing nodes (ModelArgNodes) for all upstream Models.
Cross-project refs that reference non-public models fail with a Parsing Error as expected.
I was able to remove some extra logic that was filtering down the DAG, so there will be a small performance boost.
Description and Motivation
A common feature request has been to see all upstream nodes from an injected project. For some time, this has not been available for a simple technical reason: dbt-core's access checks do not fire if you are not injecting dbt Labs' undocumented
dependencystructure at runtime. The consequence of this is that all access checks would break, allowing for private models to be ref-ed outside of groups if the referrer is in a different project. Unfortunately, dbt-core is designed in a purposefully obfuscated way to prevent mere mortals for leveraging thisdepenedencyargument, so extra measures were required.To that end, I've implemented some patches that allow us to wrap the existing reference access checks with injections that allow for dependency checks to occur as expected!
With this out of the way, we now have the following features:
ModelArgNodes) for all upstreamModels.refsthat reference non-public models fail with a Parsing Error as expected.Resolves: #40