jazzband / pip-tools

A set of tools to keep your pinned Python dependencies fresh.
https://pip-tools.rtfd.io
BSD 3-Clause "New" or "Revised" License
7.77k stars 611 forks source link

Add an argument for ignore_installed #2123

Open antony-frolov opened 3 months ago

antony-frolov commented 3 months ago

What's the problem this feature will solve?

I'm trying to lock my dependencies for a docker image build. I've got a package already installed in my base image which is not present in any package index, so I need the dependency resolver to consider installed packages.

My setup seems to be similar to https://github.com/jazzband/pip-tools/issues/1596

Describe the solution you'd like

I've found that BacktrackingResolver has an option ignore_installed to consider (or ignore) installed packages and that seems to be the solution, pip-compile works fine for me if I set ignore_installed to False here.

That would really help managing dependencies when some packages in a docker build are inherited from a base image.

antony-frolov commented 3 months ago

Here's an example of what I'm trying to do

docker run --rm nvcr.io/nvidia/pytorch:24.02-py3 bash -c '
    git clone https://github.com/jazzband/pip-tools.git \
    && pip install ./pip-tools \
    && echo torch==2.3.0a0+ebedce2 > constraints.txt \
    && echo timm > requirements.in \
    && pip-compile -c constraints.txt requirements.in
'

this runs a container with torch==2.3.0a0+ebedce2 pre-installed and I'm trying to add timm on top of an existing environment. This fails because there's no torch==2.3.0a0+ebedce2 in any index.

If ingore_installed is set to False in resolve.py it seems to work fine

docker run --rm nvcr.io/nvidia/pytorch:24.02-py3 bash -c '                        ─╯
    git clone https://github.com/jazzband/pip-tools.git \
    && sed -i 's/ignore_installed=True/ignore_installed=False/g' pip-tools/piptools/resolver.py \
    && pip install ./pip-tools \
    && echo torch==2.3.0a0+ebedce2 > constraints.txt \
    && echo timm > requirements.in \
    && pip-compile -c constraints.txt requirements.in
'