buf.yaml v2 nested path modules don't work

[ahmednfwela]

GitHub Repository

https://github.com/ahmednfwela/buf-nested-modules-issue

Commands

buf lint

Output

nested-dep\m1\sample\v1\service.proto:3:1:Files with package "sample.v1" must be within a directory "sample\v1" relative to root but were in directory "m1\sample\v1".
nested-dep\m1\sample\v1\service.proto:3:1:Multiple directories "m1/sample/v1,m2/sample/v1" contain files with package "sample.v1".
nested-dep\m2\sample\v1\service.proto:3:1:Files with package "sample.v1" must be within a directory "sample\v1" relative to root but were in directory "m2\sample\v1".
nested-dep\m2\sample\v1\service.proto:3:1:Multiple directories "m1/sample/v1,m2/sample/v1" contain files with package "sample.v1".

Expected Output

should work

Anything else?

buf version: 1.34.0

[doriable]

So the lint rules being triggered here are:

• PACKAGE_SAME_DIRECTORY: This rule checks that all files with a given package are in the same directory.
• PACKAGE_DIRECTORY_MATCH: This rule checks that all files are in a directory that matches their package name.

These lint rules are being correctly triggered when your buf.yaml defines nested-dep as one of the modules, because the module root is at the root of the nested-dep directory, and your files for package sample.v1 are at the paths m1/sample/v1 and m2/sample/v1. So these file paths do not match the package name, sample.v1, and the same package name is used across different paths.

When you changed your buf.yaml to define two modules, one with the module root m1 and one with the module root m2, then your paths for your package sample.v1 will be at sample/v1 for each module, and not dispersed across multiple paths.

Could you clarify what you mean by "should work"?

[ahmednfwela]

the lints are a side effect of another problem, which is that buf cli considers a single root buf.yaml only, if a module references another buf.yaml file, the scope is not changed for the files in that inner module

[bufdev]

You can't have nested modules - that's not a feature of Buf. You can't have a buf.yaml point to a buf.yaml. This is working as intended. Can you clarify what you're trying to do here?

[bufdev]

Closing for now as there is no bug - the commented out "works" section in your example is the correct way to do what you want. There is no capability to have a buf.yaml point at another buf.yaml (and there never has been). Let us know if you have other questions!

[ahmednfwela]

@bufdev my main use-case is depending on another buf module that's located in a git repository, but not in the BSR.

but since the deps section only accepts BSR urls, I thought with the v2 buf.yaml, it's now a supported case to clone the git repo as a submodule, and then add it to the new modules section.

[bufdev]

Can you point us to the docs that led you to this conclusion? We would appreciate it.

[ahmednfwela]

The modules section, specifically:

Any dependencies that the files have on each other are automatically taken into account when building and shouldn't be declared in the deps section.

And in the following path section:

which must be defined relative to the workspace root (the directory that contains the buf.yaml file)

This lead me to believe that there is a possibility to nest buf.yaml files, and each would be considered a "root" in its own folder