Open bulletmark opened 1 week ago
This is surprising to me as well :) will investigate
For reference I will link to this one: https://github.com/astral-sh/uv/issues/7604 There seems some oddity in choosing and running given python verisons 🤔
@zanieb this problem is 100% reproducible for maintainers and others isn't it?
Yes. Actually looking closer at this though and I'm a bit confused at what surprised me initially. We prefer managed interpreters by default so once you install one it should be used over system interpreters. I think this is working as-intended.
So this bug should be closed then because it is working as designed?
But surely you can see the problem here? A user installs a very old version just for compatibility testing for a specific venv etc, then from then on it becomes his default python for all new venvs!
Then I'd recommend changing your python-preference
from managed
to system
— which can be persisted to a configuration file. I think it's reasonable for us to use Python versions installed and managed by uv over arbitrary Python versions. I agree it's a bit of a rough edge if you mix system and managed versions, but I don't see a great alternative. I'd just uv python install 3.9 3.10 3.11 3.12
.
(I still think we can improve the documentation here, at least)
I'd like a python-preference
option managed-if-higher-version
(or system-if-higher-version
, etc). This may also be a better (i.e. least surprising) default?
I've run into a similar uv venv
problem, just with Pyenv-managed Pythons.
The uv venv
command works reasonably in a fresh shell, but funky things happen if uv
is invoked within this venv: uv venv
reproducibly flip-flops between two Python version, never re-creating the venv with the Python version that's already in use by the previous venv.
My initial worry was that this may stem from undefined directory iteration order, since uv just takes the first acceptable result, and doesn't seem to sort directory entries anywhere. However, that can't be it, as uv consistently flip-flops between two choices, whereas directory entry order should remain stable unless entries are added/deleted.
Then I had hoped that recent changes to the relevant searching code would have resolved this:
But alas, the observed behaviour is unchanged.
I suspect that uv is poisoning its caches with unusable results earlier in the $PATH
, thus rejecting a valid solution. Perhaps the cache dereferences symlinks, but the decision whether or not to include a particular executable is based on the non-dereferenced path?
I load the project's venv into my $PATH
, and also have a couple of Python installations via pyenv
:
$ echo $PATH
$PROJECT/.venv/bin:$HOME/.pyenv/shims:...
When starting the experiment, the venv's python
is symlinked to a python3.9
managed by pyenv:
$ ls -l .venv/bin | grep python
lrwxrwxrwx ... python -> $HOME/.pyenv/versions/3.9.19/bin/python3.9
lrwxrwxrwx ... python3 -> python
lrwxrwxrwx ... python3.9 -> python
My pyenv installation contains 3.8, 3.9, 3.11, and 3.12.2, but per pyenv global
the system
installation should have priority (which is a Python 3.12.3).
$ ls ~/.pyenv/shims/python* | grep -v -
$HOME/.pyenv/shims/python
$HOME/.pyenv/shims/python3
$HOME/.pyenv/shims/python3.11
$HOME/.pyenv/shims/python3.12
$HOME/.pyenv/shims/python3.8
$HOME/.pyenv/shims/python3.9
I tested with two uv
versions, 0.4.17
and the current main branch at https://github.com/astral-sh/uv/commit/9312a08009e1f5f77687ab9d1b0a09f09e2e029d. They produced equivalent behaviour.
$ uv@2024-09-28 --version
uv 0.4.17 (9312a0800 2024-09-28)
$ uv --version
uv 0.4.17
When running uv venv
in our python3.9
-venv, it will correctly refuse using the .venv/bin/python*
executables. But when it comes to the ~/.pyenv/shims/python*
candidates, uv immediately picks python3.12
, thus changing the venv's Python version.
If we run uv venv
again, the venv's python3.12
is rejected, whereas the pyenv's 3.9
is picked again:
Of course things work perfectly if I force a particular Python version. It seems that uv venv --python 3.12
consistently resolves to my system Python, regardless of the initial state of the .venv
directory.
If I exit the venv, then uv venv
also consistently picks one Python version, namely the 3.12 from pyenv.
You described it really well. This is mostly what I had experience in https://github.com/astral-sh/uv/issues/7604. One detail that played role in my case I had another uv init
one level above, hence it was picking up the env from level up, needless to say, the whole uv
did show some oddities that match to what you described above. Thanks for documenting this, which also makes me verify my sanity :)
PS. Also working here on the pyenv
Thanks for all the details. I'll need to dig into this more during the week, but pyenv is generally quite problematic for our caching since their shim can return any interpreter.
I wondered why my venv was using a very old Python version and then did the following test:
That is surprising to users, particularly when
uv
is choosing such an old version 3.8 for default use compared to 3.12.