Closed joellabes closed 1 year ago
@joellabes Love it when we have the exact same idea!
We weren't planning for this to be in the initial scope of v1.6, but the idea would very much be to consolidate both package
and project
dependencies into a single file named dependencies.yml
, and have it look like:
# dependencies.yml
packages:
- package: dbt-labs/dbt_utils
version: [">=1.0.0", "<2.0.0"]
projects: # or something else?
- project: jaffle_shop
In the event that a project is included in both spots — my thinking is, that would enable model access enforcement while resolving ref
s across projects. I wrote a bit about this here: https://docs.getdbt.com/docs/collaborate/govern/model-access#what-about-referencing-models-from-a-package
Outcomes:
projects
(new thing) in packages.yml
(old file name)packages
(old thing) in dependencies.yml
(new file name)packages
key in both packages.yml
and dependencies.yml
— we raise an errordependencies.yml
Relates to #7736
This was resolved by https://github.com/dbt-labs/dbt-core/issues/7372. However, in local testing I found that Jinja-in-yaml conditional rendering is supported in the new dependencies.yml
file. I'll open a new issue to track removing / disabling that capability.
@jtcohen6 I've just created https://github.com/dbt-labs/hubcap/issues/275 to ensure that all the dependencies still get captured by the Hub. idk if resolving this is a core eng job or something that a friendly neighbourhood Doug can knock out, but I know it's not something I can sort out myself 🙀
@joellabes Good thinking!
Is this your first time submitting a feature request?
Describe the feature
I just watched Sung's speedrun of cross-project refs, and was introduced to
dependencies.yml
for the first time. I get how we got there, but I think ofdependencies
as a package-specific thing (cf npm's dependencies and devDependencies in package.json), and I wonder if people coming in cold will be confused about the difference betweendependencies.yml
andpackages.yml
.Describe alternatives you've considered
Easy and covers most of the problem: rename
dependencies.yml
Every other part of this process' artifacts talks about publication, so maybe something along the lines of
publication_imports.yml
orpublication_discovery.yml
.Probably better, but a lot more lifting: unify
packages.yml
anddependencies.yml
Fundamentally, these are just two different ways of extending your project:
Ideally if we did this, the file wouldn't be called
packages.yml
anymore - at this point it probably does deserve thedependencies.yml
name! I don't think the rename is mandatory, but would be tidier for the long term - we'd just have to do a bit more checking that there's not both versions of the file in play for one project. `Who will this benefit?
People coming to multi-project or packages for the first time, with assumptions about what content goes in what files
Are you interested in contributing this feature?
Nah, but I will applaud loudly from the sidelines
Anything else?
No response