Using pip-compile with --build-deps-for or --all-build-deps gives error when using multiple source files.

[Walter-Gates-Bose]

Using pip-compile --all-build-deps requirments.in pyproject.toml gives the error --build-deps-for and --all-build-deps can be used only with the setup.py, setup.cfg and pyproject.toml specs..

According to the help, using multiple source files is valid, so any options that apply to at least one source file should be allowed as long as that option does not cause a conflict.

Environment Versions

1. pip-tools version: 7.4.0+

Steps to replicate

1. pip-compile --all-build-deps requirments.in pyproject.toml

Expected result

No error.

Actual result

--build-deps-for and --all-build-deps can be used only with the setup.py, setup.cfg and pyproject.toml specs.

[webknjaz]

Yeah, I don't remember what our stance on that was... However, combining multiple different envs tends to over-complicate things so I'm an advocate of doing the opposite.

Still, it was designed to lock one's package's build deps, not build deps of every single package in the tree. It'd be ambiguous which project you want the build deps to be retrieved from, if there's many. Plus, it's not uncommon that several projects in the tree have conflicting build dep restrictions (and each of those projects is built in isolation).

[Walter-Gates-Bose]

@webknjaz

Is your concern for a command like pip-compile --all-build-deps foo.in pyproject.toml or one like pip-compile --all-build-deps projectA\pyproject.toml projectB\pyproject.toml?

My use case is that I don't want to keep extra dependencies in extras_require or [project.optional-dependencies], I want to keep them in .in files so I can't use pip-compile --all-build-deps --extra foo pyproject.toml. Currently, I am forced to keep an extra build_deps.in file and use pip-compile build_deps.in foo.in pyproject.toml, which duplicates my build dependencies in both build_deps.in and pyproject.toml.

As far as I can tell (Edit: see added test), a command of the form pip-compile --all-build-deps projectA\pyproject.toml projectB\pyproject.toml is perfectly valid right now.

Would allowing only one setup file in the sources list satisfy your concern?

[webknjaz]

I once wanted to combine the deps into the same lockfile but later I changed my mind because it causes more problems than it solves: https://github.com/jazzband/pip-tools/issues/1326#issuecomment-1834517252.

That test case does not cover the situation when the build dependencies of projectA and projectB conflict with each other. Try adding a test case where there's setuptools < 40 in one and setuptools > 50 in the other.