pantsbuild / pants

The Pants Build System
https://www.pantsbuild.org
Apache License 2.0
3.19k stars 613 forks source link

Automatically infer dependency owner from directory tree, if eligible owner occurs exactly once in parent directory tree #21117

Closed jasondamour closed 4 days ago

jasondamour commented 5 days ago

Is your feature request related to a problem? Please describe. My repository has 20+ python module directories, each with a pyproject.toml. Therefore, lots of 3rd party dependencies throw the error:

Pants cannot safely infer a dependency because more than one target owns this module

It doesn't seem to make sense to create separate resolves for each python directory, because versions are shared across all submodules to guarantee compatibility.

Describe the solution you'd like Each of my python_source/python_test targets have only 1 poetry_requirements in the parent directory tree. It seems reasonable that if a dependency is declared only once in the parent directory tree, it could be assumed to be the owner.

For example:

.
├── pants.toml
├── module_foo/
│   ├── pyproject.toml  # declares `pytest` dependency
│   └── src/
│       └── foo_test.py  # should infer `pytest` dependency is owned by `module_foo:poetry#pytest`
└── module_bar/
    ├── pyproject.toml  # declares `pytest` dependency (same version as module_foo)
    └── src/
        └── bar_test.py  # should infer `pytest` dependency is owned by `module_bar:poetry#pytest`

Describe alternatives you've considered Otherwise, I'd have to explicitly add dependencies to 900+ targets in my repo:

➜  pants list :: --filter-target-type=python_sources,python_tests | wc -l   
     903

Additional context This could/should be optional behavior in either dependency declaration (ie poetry_requirements(recursive=true)) or in the dependency consumer (ie python_sources(modules_from_parent_dirs=true), or could be configured globally in pants.toml

benjyw commented 5 days ago

Have you tried this option? https://www.pantsbuild.org/2.20/reference/subsystems/python-infer#ambiguity_resolution

It is intended to do exactly what you're suggesting.

We should probably switch the default though...

jasondamour commented 4 days ago

Oh wow, this seems exactly like what I'm looking for! Didn't think to check this separate backend.

benjyw commented 4 days ago

Glad that worked for you! To clarify terms, python-infer is a "subsystem" (a namespace of options), within the python backend.