uv refuses to resolve deps with dynamic SCM version, missing tags and cylic dependencies `A:dev -> B`, `B -> A`, hinting at name shadowing

[pawamoy]

It happens in several of my projects now. Proper issue as a follow-up of #7329.

Example:

• CI job on tag 0.26.2: https://github.com/mkdocstrings/mkdocstrings/actions/runs/11307283146/job/31448849897 ("Install dependencies" step OK
• CI job on same commit (triggered via regular push): https://github.com/mkdocstrings/mkdocstrings/actions/runs/11307282883/job/31448849515 -> "Install dependencies" step fails

Using CPython 3.10.15 interpreter at: /opt/hostedtoolcache/Python/3.10.15/x64/bin/python
Creating virtual environment at: .venv
  × No solution found when resolving dependencies:
  ╰─▶ Because only the following versions of mkdocstrings-python-legacy are
      available:
          mkdocstrings-python-legacy<=0.2.1
          mkdocstrings-python-legacy==0.2.2
          mkdocstrings-python-legacy==0.2.3
          mkdocstrings-python-legacy==0.2.4
      and mkdocstrings-python-legacy>=0.2.1,<=0.2.2 depends on your project,
      we can conclude that mkdocstrings-python-legacy>=0.2.1,<0.2.3 depends on
      your project.
      And because mkdocstrings-python-legacy>=0.2.3 depends on your project,
      we can conclude that mkdocstrings-python-legacy>=0.2.1 depends on your
      project.
      And because mkdocstrings[python-legacy] depends on
      mkdocstrings-python-legacy>=0.2.1 and your project requires
      mkdocstrings[python-legacy], we can conclude that your projects's
      requirements are unsatisfiable.

      hint: The package `mkdocstrings-python-legacy` depends on the package
      `mkdocstrings` but the name is shadowed by your project. Consider
      changing the name of the project.

Here uv claims that since mkdocstrings-python-legacy depends on mkdocstrings, and my project is named mkdocstrings, it cannot resolve deps. My project is mkdocstrings, it's not shadowing it :sweat_smile:

In this case, mkdocstrings depends on mkdocstrings-python-legacy as an extra (optional dependency).

[project.optional-dependencies]
python-legacy = ["mkdocstrings-python-legacy>=0.2.1"]

But I get the same error when project A depends on B as a development dependency.

I'm not sure if the non-editable mode is the issue here, as I cannot reproduce this behavior locally.

[pawamoy]

Here are more verbose logs: https://github.com/mkdocstrings/griffe/actions/runs/11307459194/job/31449208076.

[pawamoy]

Ah, maybe it's because my version is dynamic, and since the project is shallow (tags not fetched), my build backend (pdm-backend) ends up computing the version as 0.1, which is clearly inferior to what the dev/extra-dependency requires.

[pawamoy]

Yeah that's probably it. So, possible improvement for uv's error message: show what's the project computed version, and show that it's what is actually causing the resolution failure. Or maybe uv could ignore the fact that the built project has a version incompatible with what a dependency is requiring? Maybe a "responsible adult" move, maybe a footgun.

As for the immediate solution, I guess I'll just always fetch tags in CI! Feel free to close :slightly_smiling_face:

[pawamoy]

Wait, no, I actually fetch all tags in this job :weary: Lemme try again with different options on the checkout action.

[pawamoy]

OK so my previous technique for fetching tags was probably not working well:

    - name: Fetch all tags
      run: git fetch --depth=1 --tags

I replaced it with:

    - name: Checkout
      uses: actions/checkout@v4
      with:
        fetch-depth: 0
        fetch-tags: true

...and now it works :hot_face:

[charliermarsh]

Thank you for following up 🙏

Is there a bug in the error message hint then? Or is this resolved?

[pawamoy]

I wouldn't call it a bug, I would say it is confusing. Seeing this error message didn't help me understand what was going on, and the "name shadowing" hint is... incorrect? As mentioned above, I'm not shadowing a dependency name, the project and the dependency are the same project.

So my recommendation, if this is actually what's happening, and if it's possible, is to update the error message by showing that the computed project A version (current project, built for a non-editable install), and the version of project A required by a (dev-)dependency are not compatible. Bonus point if uv detects that the version is dynamic and the Git repository shallow, suggesting to fetch all commits and tags to build the project with the correct version.

   × No solution found when resolving dependencies:
  ╰─▶ Because only the following versions of X are
      available:
          X<=1.2.3
          X==1.2.3
          etc.
      and X>=1.2.3,<=1.2.4 depends on your project,
      we can conclude that X>=1.2.3,<=1.2.4 depends on
      your project.
      And because your project version is 0.1, and X depends on your project>=4.5.6,
      we can conclude that your projects' requirements are unsatisfiable.

      hint: Your project version is computed dynamically, and your Git repository is shallow
      -> you might need to fetch all commits and tags to compute the up-to-date version.

(well, something like this)

My alternative suggestion above is allow incompatibility between the current project version and the version required by dependencies, and install the current project anyway, always, maybe just showing a warning:

Your project depends on X>=1.2.3, which depends on your project>=7.8.9,
but your project is currently in v4.5.6, which is incompatible.
uv will install your project anyway, but things might break.

hint: Your project version is computed dynamically, and your Git repository is shallow
-> you might need to fetch all commits and tags to compute the up-to-date version.

Let me know if anything is unclear :relaxed:

[pawamoy]

One argument in favor of allowing the current project version to be imcompatible with the version required by a dependency is that when forking a repository on GitHub, tags are not copied over. Then when triggering CI on your fork, the checkout action can't fetch tags since they don't exist, and uv fails to resolve. To fix this, authors of the fork have to explicitly fetch tags from upstream and push them back into their fork.

Or that's an argument in favor of me dropping dynamic versioning and using a static, auto-bumped version for each release :smile:

[pawamoy]

I have updated the title, and built a reproduction.

git clone git@github.com:pawamoy/repro-uv-8148-a
cd repro-uv-8148-a
uv sync  # works fine

# Now we clone a fork of the same project,
# meaning we don't have the Git tags.
cd ..
git clone git@github.com:pawamoy-forks/repro-uv-8148-a repro-uv-8148-a-fork
cd repro-uv-8148-a-fork
uv sync  # fails with the following output

% uv sync
Using CPython 3.12.7 interpreter at: /home/pawamoy/.basher-packages/pyenv/pyenv/versions/3.12.7/bin/python
Creating virtual environment at: .venv
  × No solution found when resolving dependencies:
  ╰─▶ Because only repro-uv-8148-b==0.1.1 is available and repro-uv-8148-b==0.1.1 depends on your project, we can conclude that all versions of repro-uv-8148-b depend on
      your project.
      And because repro-uv-8148-a:dev depends on repro-uv-8148-b and your project depends on repro-uv-8148-a:dev, we can conclude that your project's requirements are
      unsatisfiable.

      hint: The package `repro-uv-8148-b` depends on the package `repro-uv-8148-a` but the name is shadowed by your project. Consider changing the name of the project.

I'd love to have a flag (CLI flag or config option in pyproject.toml or both) to tell uv that I'm not shadowing the dependency's name, that they're the same thing, and that I want to install the current project anyway, even if its built version isn't compatible with what the dependency requires.

[pawamoy]

I thought about and tried using dependency overrides, which solves the resolution issue, but makes uv actually install the current project as a dependency instead of as an editable install.

[tool.uv]
override-dependencies = ["mkdocstrings>=0"]

I also tried configuring a source for mkdocstrings (which, again, is the current project) like so:

[tool.uv.sources]
mkdocstrings = { workspace = true }

...and:

[tool.uv.sources]
mkdocstrings = { path = ".", editable = true }

...but it is still not installed as editable. The dependency override seems to take precedence.

[pawamoy]

Haha, I then tried this:

[tool.uv]
override-dependencies = ["mkdocstrings @ ."]

...which failed with this message:

warning: Failed to parse `pyproject.toml` during settings discovery:
  TOML parse error at line 88, column 25
     |
  88 | override-dependencies = ["mkdocstrings @ ./"]
     |                         ^^^^^^^^^^^^^^^^^^^^^
  relative path without a working directory: ./
  mkdocstrings @ ./
                 ^^

error: Failed to parse: `pyproject.toml`
  Caused by: TOML parse error at line 88, column 25
   |
88 | override-dependencies = ["mkdocstrings @ ./"]
   |                         ^^^^^^^^^^^^^^^^^^^^^
relative path without a working directory: ./
mkdocstrings @ ./

And then tried this:

[tool.uv]
override-dependencies = ["mkdocstrings @ ${PWD}"]

...which worked :tada: but looks so weird haha

Overriding self with self :melting_face: It makes sense in a way: "yes, use self project as is, don't pay attention to what others say about it".

So, in the end, I think you can close! A paragraph in the docs could be nice (but it feels super niche too, so maybe not worth it).

[pawamoy]

One issue with using ${PWD} is that uv commands will only work in the root of the repo. Not an issue in my case though, I usually only run commands in the root. Oh, it might not work on Windows, too :thinking: Could uv provide a PROJECT_ROOT env var that I could use instead of PWD :thinking:?

[pawamoy]

Hmm, uv already supports this env var, although it's undocumented (or at least searching for it in the docs doesn't yield anything).

What I'm not sure to understand is that it seems to be equivalent to PWD: if I enter a subdirectory and run uv lock from there, it says "can't build because subdirectory is not a Python project". Passing --project .. or --project /abs/path/to/project doesn't change anything. Is this because I'm not using workspaces/monorepos and therefore PROJECT_ROOT defaults to the CWD somehow?

[zanieb]

Hmm, uv already supports this env var, although it's undocumented (or at least searching for it in the docs doesn't yield anything).

I think we need to support that to build projects that use Hatch (but otherwise it's not really a first-class thing).

[zanieb]

I guess what you're looking for here is for us to respect sources on top of overrides? I'm sort of surprised we don't, honestly.

[pawamoy]

override-dependencies works for me, I'm just looking for something more robust than ${PWD} when specifying the path :slightly_smiling_face: ${PROJECT_ROOT} is already great, even though I'm a bit confused by its behavior :smile:

If you think there's some room for improvement in the way uv handles overrides and sources, I'm happy to help any way I can! Just to be clear, I'm only using overrides now, not declaring any source.

[pawamoy]

I prefer declaring this:

[tool.uv]
override-dependencies = ["mkdocstrings @ ${PROJECT_ROOT}"]

Than this:

[tool.uv]
override-dependencies = ["mkdocstrings>=0"]

[tool.uv.sources]
mkdocstrings = { workspace = true }
# or mkdocstrings = { path = "${PROJECT_ROOT}", editable = true }

...because even if the result is the same, it feels like the latter will completely disregard any constraints incompatibilities in dependencies regarding mkdocstrings (>=0), while the former will only "force" installation of the current project, while still checking that dependencies have compatible constraints about mkdocstrings. I don't think it's the case though, and both options completely ignore constraints regarding mkdocstrings. Even then, the first option is more appealing, as it requires less things to declare.

(I hope it's fine that I commented so much on this issue! Not trying to get your attention, just documenting my journey and thought process :smile:)

[pawamoy]

Hmm wait, need to confirm. Ah, one more issue, with such an override ("mkdocstrings @ ${PROJECT_ROOT}"), the --no-editable flag in uv sync has no effect anymore :confused:

Yeah so it's worse than that, the override actually prevents the project to be installed in editable mode. Damn I'm going in circles :rofl:

This might be an argument favor in of the second option above, if its changed to support editable installs again, thanks to sources.

UPDATE: I've updated https://github.com/pawamoy/repro-uv-8148-a to document all this in the readme.

[pawamoy]

In the end you guessed right @zanieb: I think the correct thing to declare on my end is this:

[tool.uv]
override-dependencies = ["mkdocstrings @ ${PROJECT_ROOT}"]

[tool.uv.sources]
mkdocstrings = { path = "${PROJECT_ROOT}", editable = true }

(or some simplification of this, if such a simplification is already possible, or deemed useful enough to become supported in the future)

uv then just needs to respect the sources on top of overrides, like you said :slightly_smiling_face:

[zanieb]

@charliermarsh, wdyt about changing this interaction of sources and overrides?

[charliermarsh]

Yeah that makes sense... It seems like sources should apply to overrides and constraints too?