Support for PEP 735 dependency groups

[cthoyt]

What's the problem this feature will solve?

PEP 735 introduced dependency groups, which are complementary to optional dependencies in that dependency groups might not correspond to features in the package, but rather be something like development or release dependencies.

You can install something with dependency groups like this:

$ pip install --dependency-groups=test,typing

ReadTheDocs currently supports specifying optional dependencies (c.f., https://docs.readthedocs.io/en/stable/config-file/v2.html#packages) with configuration like the following, where docs probably has sphinx, sphinx-rtd-theme, and other sphinx plugins.

version: 2

python:
  install:
    - method: pip
      path: .
      extra_requirements:
        - docs

Describe the solution you'd like

I'd like an additional configuration in the install dictionary that works similar to extra_requirements that would correspond to dependency groups. Let's say I have used the PEP 735 definition of my docs, like in this abbreviated pyproject.toml:

[dependency-groups]
tests = [
    "pytest",
    "coverage",
]
docs = [
    "sphinx>=8",
    "sphinx-rtd-theme>=3.0",
    "sphinx_automodapi",
]

then I would want to define it with an alternate key, like dependency_groups (just a suggestion for the name):

version: 2

python:
  install:
    - method: pip
      path: .
      dependency_groups:
        - docs

Alternative solutions

This approach isn't required to get the intended results, which is of course to install the right extra dependencies to get my docs to build. However, the main goal is to better organize my metadata, and not to expose sphinx as an "extra" on PyPI, which doesn't exactly fit the spirit of extras/optional dependencies

Additional context

This is motivated by recent improvements in pip, uv, and tox! I have an (almost) working demo in my cookiecutter project which shows how to update configuration properly to support this https://github.com/cthoyt/cookiecutter-snekpack/pull/32

I am not familiar at all with the RTD stack, but if there's a way I can contribute to coding this up, please let me know :)

Nitty-gritty

Here are a few places which probably would be part of a theoretical implementation:

https://github.com/readthedocs/readthedocs.org/blob/404d82a448295c81a271c3143d0fc9c10a924555/readthedocs/config/models.py#L77-L81

I'd simply add a new slot dependency_groups here

https://github.com/readthedocs/readthedocs.org/blob/404d82a448295c81a271c3143d0fc9c10a924555/readthedocs/doc_builder/python_environments.py#L66-L67

here's how I'd update this:

# Added these next lines
dependency_group_args = []
if install.dependency_groups:
    # not clear if the equals is necessary or if
    # this can be broken into two parts
    dependency_group_args.append("--dependency-groups={}".format(",".join(install.dependency_groups))

extra_req_param = ""
if install.extra_requirements:
    extra_req_param = "[{}]".format(",".join(install.extra_requirements))
self.build_env.run(
    self.venv_bin(filename="python"),
    "-m",
    "pip",
    "install",
    "--upgrade",
    "--upgrade-strategy",
    "only-if-needed",
    "--no-cache-dir",
    "{path}{extra_requirements}".format(
        path=local_path,
        extra_requirements=extra_req_param,
    ),
    *dependency_group_args,
    cwd=self.checkout_path,
    bin_path=self.venv_bin(),
)

As a minor note, I would also do a bit of refactoring to store all of the args into the list and then splat all of them into run()

[humitos]

Hi @cthoyt, thanks for opening this issue.

I don't think we will support this soon since it requires a lot of extra work from our side and we are focusing ourselves on different features currently.

However, this use case will be a really good fit for the work we are doing on https://github.com/readthedocs/readthedocs.org/pull/11710. Once that PR gets merged, you will be able to write something like:

version: 2

build:
  jobs:
    install:
      - pip install --dependency-groups=test,typing 

[cthoyt]

@humitos super, I hope this brings a lot of simplification to your work!

I will be happy to test run that this is working properly when https://github.com/readthedocs/readthedocs.org/pull/11710 is ready.

Feel free to close or leave this issue open in case anyone else comes looking for it.