Add option to add files that need to be considered for the cache key in pyproject.toml

[albertferras-vrf]

Consider the following pyproject.toml file, which uses dynamic metadata to reference requirements.in files.

[project]
name = "myproject"
version = "0.1.0"
requires-python = ">=3.9"
dynamic = ["dependencies"]

[tool.setuptools.dynamic]
dependencies = {file = ["requirements.in"]}

When invoking the command uv pip compile -o requirements.txt pyproject.toml then setuptools is used to build the project, which is the tool that understands the dynamic and tool.setuptools.dynamic configuration. Once the build is finished, uv reads the package's requires.txt and generates requirements.txt. The problem is that after (only) modifying requirements.in, the command uv pip compile does not update the requirements.txtfile. This is because the content pyproject.toml did not change and uv decide to not reinstall the package (call setuptools) again. The current workaround is to add --reinstall or --reinstall-package=myproject to the command or configuration and force setuptools to run.

This was discussed in discord, and the user @ThiefMaster suggested that we could add a new configuration under [tool.uv] which allows specifying which files have to be considered as part of 'pyproject.toml's cache key. One example would be:

[tool.uv]
cache_key_files = ["requirements.in", "requirements.dev.in"]

Worth mentioning also (not part of this ticket) that it would be good if uv supported popular dynamic implementations natively and not need to invoke setuptools for it.

[charliermarsh]

I want to work on this but I don't yet have a good solution for setuptools_scm or other things that depend on the Git state.

[charliermarsh]

I could special-case .git in the array though it's not a great API.

[charliermarsh]

@konstin @zanieb -- any ideas? If we're gonna support this, I think we need an answer for setuptools_scm too since it's so popular.

[zanieb]

Well... something like cache_key_paths = ["path.txt", { file = "other-path.txt }, { git = "." }) is an option? e.g. allow dictionaries so we can define types and default to paths if a simple value is provided? (git = "." being a Git repository rooted in the project directory, we could also do like git = true but I'm not sure it's better)

[charliermarsh]

That seems reasonable to me.

[zanieb]

How complicated does support for Git state need to be? Like... the commit hash changed? There are unstaged changes? The tree is dirty? Number of commits since a tag? etc.

[charliermarsh]

I think "commit hash changed" is fine.

[charliermarsh]

I guess setuptools-scm will also reflect uncommitted changes, but that may not be necessary.

[charliermarsh]

This exists as of v0.4.8. You can do things like:

[tool.uv]
cache-keys = [{ file = "pyproject.toml" }, { file = "requirements.txt" }, { git = true }]