jazzband / pip-tools

A set of tools to keep your pinned Python dependencies fresh.
https://pip-tools.rtfd.io
BSD 3-Clause "New" or "Revised" License
7.6k stars 607 forks source link

Recursive extra dependencies compiles wrong requirements.txt #2110

Open maxime1907 opened 2 weeks ago

maxime1907 commented 2 weeks ago

When pyproject.toml references its own local package to recursively include extra dependencies, it outputs a requirements.txt file which references the local package with the absolute path which is a problem as its not portable and should instead just list the dependencies

It was previously possible with setup.py with this configuration:

from setuptools import find_packages, setup

extra_require_tools = ["build"]
extra_require_test = ["pyyaml"]

setup(
    name='package',
    version='1.0.0',
    packages=find_packages(),
    extras_require={
        "dev": extra_require_tools + extra_require_test,
        "tools": extra_require_tools,
        "test": extra_require_test,
    },
)

Environment Versions

  1. OS Type: Ubuntu 22.04.4 LTS
  2. Python version: Python 3.11.6
  3. pip version: pip 23.2.1
  4. pip-tools version: pip-compile, version 7.4.1

Steps to replicate

  1. Create a pyproject.toml with this content:
    
    [project]
    name = "package"
    version = "1.0.0"

[build-system] requires = ["setuptools>=69.1", ] build-backend = "setuptools.build_meta"

[project.optional-dependencies] dev = ["package[test,tools]"] tools = ["build"] test = ["pyyaml"]

2. Generate the requirements-dev.txt
```bash
pip-compile --extra=dev --no-emit-index-url --output-file=requirements-dev.txt pyproject.toml

Expected result

#
# This file is autogenerated by pip-compile with Python 3.11
# by the following command:
#
#    pip-compile --extra=dev --no-emit-index-url --output-file=requirements-dev.txt pyproject.toml
#
build==1.2.1
    # via package
packaging==24.1
    # via build
pyproject-hooks==1.1.0
    # via build
pyyaml==6.0.1
    # via package

Actual result

#
# This file is autogenerated by pip-compile with Python 3.11
# by the following command:
#
#    pip-compile --extra=dev --no-emit-index-url --output-file=requirements-dev.txt pyproject.toml
#
build==1.2.1
    # via package
package[test,tools] @ file:///tmp/example
    # via package (pyproject.toml)
packaging==24.1
    # via build
pyproject-hooks==1.1.0
    # via build
pyyaml==6.0.1
    # via package

Linked issues

chrysle commented 2 weeks ago

This is an edge case; we probably need to add some logic to drop the origin requirements if they are recursive extras, as they are redundant if the dependencies they imply are already part of the requirements file. That should happen directly after the resolution process. It's rather low priority, I presume, we have a lot of other issues to tackle right now - you're welcome to try and draft a fix, of course.