Open AlexTereshenkov opened 9 months ago
If we do proceed with a global option, I've done a bit of research, where I think we shall make changes:
Hmmmmm, TBH I think we should be moving away from 1:1:1 recommendations and more towards getting rid of targets altogether as much as possible. The thing that would enable this is composition of target metadata up the filesystem. So there is no longer any concept of a target owning a source file. Instead the source file is the main concept, and zero, one or more targets just optionally add metadata to it (I'm talking about source targets like python_sources
, binary targets like pex_binary
are a different thing).
In this world you can attach metadata however you want - local globs, higher-level recursive globs, whatever makes sense in your repo, for the specific metadata in question, and 1:1:1 is no longer a useful recommendation. And of course in this world a source file can be referenced by zero source targets if there is no interesting metadata to add to it.
So I'm slightly leery of doing things that move us away from this world.
Oh okay, this is not something I have explored yet. Feel free to close the ticket!
For anyone who is not ready for this world and/or operates in the repo with plenty of targets, the suggestion would be to use some tooling capable of finding recursive globs patterns, either in automated or semi-automated manner.
Is your feature request related to a problem? Please describe. Pants supports providing sources either as individual files or as patterns; one can use
*
for globs over just the current working directory,**
for recursive globs over everything below (at any level the current working directory), with prefix!
for ignores.The best practice, however, is to follow the 1:1:1 principle: metadata about your code should live near the code itself. For this reason, I think it may be helpful to enforce this via a global option so that users will be able to forbid using recursive globs in
sources
fields (or elsewhere where the files ownership is declared) similarly to how we handle globs in BUILD files that do not match files on disk (ignore/warn/fail).Describe the solution you'd like Having this directory with source code, e.g.
with
running
yields all the files in the directory tree, recursively. With the global option to control whether using recursive globs is permitted, one should be able to raise an error or warn about it.
Describe alternatives you've considered Currently, users who would like to control whether recursive globs are used would need to use some kind of semantic grep / AST tools to identify recursive globs usage which may be a lot of work.
Additional context I can see a use case where there's a directory with lots of inner directories containing some non-code resources and one wouldn't want to create individual BUILD files inside them, particularly, if the directories are added/removed all the time. See https://github.com/pantsbuild/pants/issues/18217 for illustration. So one may want to allow using a recursive pattern on some targets, but not all of them (e.g. allow on
resources
, but forbid onpython_sources
). A global option won't work in this case, but adding a new field to a base target that deals with file ownership may be unjustified?