Issue running a library that internally uses `python -m pip ...`

[EdgyEdgemond]

I understand the issue behind this, uv is replacing pip so there is no pip in the virtualenv, I am wondering if there is a way to ensure there is a pip in the venv using uv correctly.

MRE:

pyproject.toml

[project]
name = "uv-test"
version = "0.0.0"
requires-python = ">=3.10,<3.11"

dependencies = [
    "numpy == 1.26.3",
    "great_expectations == 0.18.1",
    "spacy[transformers] == 3.6.1",
]

lock/sync and then run:

>> uv run spacy download en_core_web_md-3.6.0 --direct
/home/edgy/code/nrwl/scripts/uv-test/.venv/bin/python: No module named pip

Under the hood spacy download is firing off python -m pip *args which understandably blows up. In our docker images and pipelines I can uv pip compile to get a requirements.txt and use the system pip to install things and spacy is happy.

But locally it picks up the .venv and triggers the error above.

[kbernhagen]

Maybe use uv add --dev pip

[zanieb]

Or uv run --with pip spacy download ... to add it temporarily.

[zanieb]

You could also do something more controversial and alias pip to uv pip which would work fine if they're not depending on less common pip behaviors.

[EdgyEdgemond]

Was coming to the realisation pip isn't a special python tool, it's just a module I can use as a dependency. Thanks for the confirmation on the solution I should have realised yesterday. I'll confirm it works shortly.

[EdgyEdgemond]

Adding pip as a specified dependency did work.

Surprisingly --with pip didn't work, it did install pip before running.

>> uv run --with pip spacy download en_core_web_md-3.6.0 --direct 
Installed 1 package in 8ms
/uv-test/.venv/bin/python: No module named pip

[EdgyEdgemond]

Feel free to close this if --with pip is somehow expected behaviour, as I have a solution that works for me.

[charliermarsh]

What did you end up doing to solve it? uv tool run --with pip spacy download en_core_web_md-3.6.0 --direct works for me, for what it's worth.

[charliermarsh]

I think the uv run version fails due to how our environment layering works... They use [sys.executable, "-m", "pip", "install"], and I guess sys.executable refers to the base interpreter despite the fact that we layer the --with requirements on top as a separate environment.

[EdgyEdgemond]

I did not try uv tool run, i ended up adding pip to the projects requirements, as spacy does indeed require it :)

Ill keep uv tool run in mind next time i encounter an odd situation.

At this point we have successfully migrated all projects at work to uv, so happy with the current state of things. Thanks for all your work on this tool. Looking forward to if/when build tooling is in house.

[charliermarsh]

Awesome, thank you @EdgyEdgemond. (We do have uv build but maybe you're referring to something different :))

[charliermarsh]

I think I'd consider this a bug, though it's hard to say, arguably SpaCy should be importing / calling pip some other way than via subprocess... We can only fix this by redesigning the overlay system.

[EdgyEdgemond]

I think this is an edge case of a library performing under assumptions, that can be handled with existing functionality. It should have pip as a dependency in an ideal world etc. It would be nice to work around using --with but not a deal breaker.

@charliermarsh I was thinking of publish architecture not build, twine etc :)

[urvisism]

@EdgyEdgemond @charliermarsh I installed a spacy model using the given command and it worked.

uv tool run --with pip spacy download de_core_news_lg-3.7.0 --direct

But, when I tried to use it in the given code. It gives me an error. Code:

import spacy
nlp = spacy.load("de_core_news_lg")

Error:

OSError: [E050] Can't find model 'de_core_news_lg'. It doesn't seem to be a Python package or a valid path to a data directory.

Need some help.

[EdgyEdgemond]

I solved this by adding pip as a requirement of my project (as it is a requirement of spacy), and ran spacy in the venv directly.

Im not sure if uv tool puts pip somewhere else and as a result spacy download ends up in a different location. That would be my guess without understanding the internals like @charliermarsh

[urvisism]

@EdgyEdgemond Could you please guide me how to add pip as a requirement in project ?

[EdgyEdgemond]

Simple as adding pip to your dependencies. pip is just another python package.

dependencies = [
    ...,
    "pip"
]