charliermarsh
commented
3 months ago Thanks, will take a look.
Closed taranlu-houzz closed 1 month ago
charliermarsh
commented
3 months ago Thanks, will take a look.
charliermarsh
commented
3 months ago What's the result of running pip install llvmlite==0.35.0 --use-pep517 --no-cache --force-reinstall?
taranlu-houzz
commented
3 months ago ↓2 ~/Documents/uv_llvmlite_bug via 🐍 v3.11.7 (.venv)
❯ pip install llvmlite==0.35.0 --use-pep517 --no-cache --force-reinstall
Collecting llvmlite==0.35.0
Downloading llvmlite-0.35.0.tar.gz (121 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: llvmlite
Building wheel for llvmlite (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for llvmlite (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [29 lines of output]
running bdist_wheel
/Users/taran/Documents/uv_llvmlite_bug/.venv/bin/python3.11 /private/var/folders/_s/t531j8b96sg3gzbpsc40s_5c0000gn/T/pip-install-zfgo_lkj/llvmlite_8f49a3f59eef4b7
195a32f3df02acf1f/ffi/build.py
LLVM version... Traceback (most recent call last):
File "/private/var/folders/_s/t531j8b96sg3gzbpsc40s_5c0000gn/T/pip-install-zfgo_lkj/llvmlite_8f49a3f59eef4b7195a32f3df02acf1f/ffi/build.py", line 105, in main_p
osix
out = subprocess.check_output([llvm_config, '--version'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/taran/.local/share/mise/installs/python/3.11.7/lib/python3.11/subprocess.py", line 466, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/taran/.local/share/mise/installs/python/3.11.7/lib/python3.11/subprocess.py", line 548, in run
with Popen(*popenargs, **kwargs) as process:
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/taran/.local/share/mise/installs/python/3.11.7/lib/python3.11/subprocess.py", line 1026, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/Users/taran/.local/share/mise/installs/python/3.11.7/lib/python3.11/subprocess.py", line 1950, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'llvm-config'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/private/var/folders/_s/t531j8b96sg3gzbpsc40s_5c0000gn/T/pip-install-zfgo_lkj/llvmlite_8f49a3f59eef4b7195a32f3df02acf1f/ffi/build.py", line 191, in <modul
e>
main()
File "/private/var/folders/_s/t531j8b96sg3gzbpsc40s_5c0000gn/T/pip-install-zfgo_lkj/llvmlite_8f49a3f59eef4b7195a32f3df02acf1f/ffi/build.py", line 185, in main
main_posix('osx', '.dylib')
File "/private/var/folders/_s/t531j8b96sg3gzbpsc40s_5c0000gn/T/pip-install-zfgo_lkj/llvmlite_8f49a3f59eef4b7195a32f3df02acf1f/ffi/build.py", line 107, in main_p
osix
raise RuntimeError("%s failed executing, please point LLVM_CONFIG "
RuntimeError: llvm-config failed executing, please point LLVM_CONFIG to the path for llvm-config
error: command '/Users/taran/Documents/uv_llvmlite_bug/.venv/bin/python3.11' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for llvmlite
Failed to build llvmlite
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (llvmlite)It's very likely a problem with the package itself then. I'm having trouble reproducing this because that llvmlite version requires a very old LLVM, and newer versions work fine:
LLVM_CONFIG=/opt/homebrew/opt/llvm@14/bin/llvm-config uv pip install https://files.pythonhosted.org/packages/9f/3d/f513755f285db51ab363a53e898b85562e950f79a2e6767a364530c2f645/llvmlite-0.43.0.tar.gz --no-cache --force-reinstallCan you try setting LLVM_CONFIG explicitly?
taranlu-houzz
commented
3 months ago Hmm, maybe I am doing something wrong? This is the full output of the actual command:
uv tool install --python 3.11 --python-preference only-managed --extra-index-url '<internal repo>' --prerelease allow --refresh --verbose my-package
For some reason, uv is choosing that very old version of llvmlite as a dep for numba, but when I install via pipx it chooses a much more recent version: 0.43.0.
joennlae
commented
3 months ago I am not sure if this is an uv issue. I have this issue with v2.3.7 and v3.0.
I was able to fix it by pinning the numba version to numba>=0.60.0
Garfield100
commented
3 months ago I have a similar problem to you @joennlae with the llvm version. I would prefer not installing this old llvm version and it would be impossible to get my whole student team to do it :sweat_smile: The llvmlite docs say to use binary wheels to install it: https://llvmlite.readthedocs.io/en/latest/admin-guide/install.html# How does one specify this to uv?
edit: I believe uv add --no-build-package llvmlite llvmlite worked
taranlu-houzz
commented
3 months ago @joennlae @charliermarsh Unless I am using uv tool install incorrectly, I think that it is incorrectly resolving to an old version, since I am able to correctly install via pipx.
pipx command that works: pipx install my-package --pip-args='--index-url=http://<internal repo>:<port> --trusted-host=<internal repo> --pre' --python $(which python3.11)
uv command that has an issue: uv tool install --python 3.11 --python-preference only-managed --extra-index-url '<internal repo>' --prerelease allow --refresh --verbose my-package
I guess one other difference is that I am using the uv managed version of Python vs. one that was installed via mise.
charliermarsh
commented
3 months ago Does llvmlite exist on your internal index?
taranlu-houzz
commented
3 months ago No, llvmlite is not on the internal index.
charliermarsh
commented
3 months ago @taranlu-houzz -- Just to clarify, in your previous message, you used my-package -- is that llvmlite, or this is a separate issue?
taranlu-houzz
commented
3 months ago I replaced the name of the actual package with my-package and it has a dependency (actually, dep of a dep) that needs llvmlite. Sorry for the confusion.
thisismygitrepo
commented
3 months ago Same on Windows.
Resolved 174 packages in 125ms
error: Failed to prepare distributions
Caused by: Failed to fetch wheel: llvmlite==0.34.0
Caused by: Build backend failed to build wheel through `build_wheel()` with exit code: 1
--- stdout:
running bdist_wheel
C:\Users\alex\AppData\Local\uv\cache\builds-v0\.tmptz0R7c\Scripts\python.exe C:\Users\alex\AppData\Local\uv\cache\built-wheels-v3\pypi\llvmlite\0.34.0\aL2IMNnikQ99dItFJEjv2\llvmlite-0.34.0.tar.gz\ffi\build.py
Trying generator 'Visual Studio 15 2017 Win64'
--- stderr:
Traceback (most recent call last):
File "C:\Users\alex\AppData\Local\uv\cache\built-wheels-v3\pypi\llvmlite\0.34.0\aL2IMNnikQ99dItFJEjv2\llvmlite-0.34.0.tar.gz\ffi\build.py", line 191, in <module>
main()
File "C:\Users\alex\AppData\Local\uv\cache\built-wheels-v3\pypi\llvmlite\0.34.0\aL2IMNnikQ99dItFJEjv2\llvmlite-0.34.0.tar.gz\ffi\build.py", line 179, in main
main_win32()
File "C:\Users\alex\AppData\Local\uv\cache\built-wheels-v3\pypi\llvmlite\0.34.0\aL2IMNnikQ99dItFJEjv2\llvmlite-0.34.0.tar.gz\ffi\build.py", line 88, in main_win32
generator = find_win32_generator()
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\alex\AppData\Local\uv\cache\built-wheels-v3\pypi\llvmlite\0.34.0\aL2IMNnikQ99dItFJEjv2\llvmlite-0.34.0.tar.gz\ffi\build.py", line 76, in find_win32_generator
try_cmake(cmake_dir, build_dir, generator)
File "C:\Users\alex\AppData\Local\uv\cache\built-wheels-v3\pypi\llvmlite\0.34.0\aL2IMNnikQ99dItFJEjv2\llvmlite-0.34.0.tar.gz\ffi\build.py", line 28, in try_cmake
subprocess.check_call(['cmake', '-G', generator, cmake_dir])
File "C:\Users\alex\AppData\Local\Programs\Python\Python311\Lib\subprocess.py", line 408, in check_call
retcode = call(*popenargs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\alex\AppData\Local\Programs\Python\Python311\Lib\subprocess.py", line 389, in call
with Popen(*popenargs, **kwargs) as p:
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\alex\AppData\Local\Programs\Python\Python311\Lib\subprocess.py", line 1026, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Users\alex\AppData\Local\Programs\Python\Python311\Lib\subprocess.py", line 1538, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [WinError 2] The system cannot find the file specified
error: command 'C:\\Users\\alex\\AppData\\Local\\uv\\cache\\builds-v0\\.tmptz0R7c\\Scripts\\python.exe' failed with exit code 1
This is my first use of uv and the error gives me impression I'm doing something wrong because of that weird python.exe path provided. I thought I'm using my correct vevnv which has different path.
derek-pyne
commented
3 months ago Had this same behaviour where uv was trying to install an old version of llvmlite which was a dependency of one of my dependencies (in my case: umap-learn). This dependency doesn't need the old version of llvmlite.
These dependencies were resolvable with poetry (I'm migrating).
Tried to recreate with a smaller project without success
Solution was to explicitly add llvmlite as a dependency (ie. uv add llvmlite). This makes sure that it uses a recent version of llvmlite (>=0.43.0)
zanieb
commented
3 months ago 👍 @derek-pyne you can also use
[tool.uv]
constraint-dependencies = ["llvm>0.43.0"]to add a constraint without adding it as a dependency of your project.
derek-pyne
commented
3 months ago ahhh, even better!
taranlu-houzz
commented
3 months ago @zanieb That is a cool feature, but I think this may still be a bug. Is there a way to apply your suggestion when using uv install? I'd rather not have to add a workaround on the package side of things since pipx is still able to correctly install the package with the correct (newer) version of llvmlite).
charliermarsh
commented
3 months ago If you run uv tool install numba or similar, and we get an older version of llvmlite, that is not necessarily a bug. In fact it's probably not? My guess is that we just ended up with a different (but valid) resolution that pip. Most likely, some other package in the resolution was installed at a higher version than the corresponding package in pip.
taranlu-houzz
commented
3 months ago @charliermarsh So, this is the dep tree (via pipdeptree) that is installed in the pipx created virtualenv:
├── largestinteriorrectangle [required: >=0.2.1, installed: 0.2.1]
│ └── numba [required: Any, installed: 0.60.0]
│ ├── llvmlite [required: >=0.43.0dev0,<0.44, installed: 0.43.0]
│ └── numpy [required: >=1.22,<2.1, installed: 2.0.1]I would think that the version requirements of numba for llvmlite would make it invalid to try to install 0.35.0, right?
taranlu-houzz
commented
3 months ago Is it possible that uv is choosing to install an older version of numba since the dep requirement is Any, and perhaps that version relies on the old llvmlite?
charliermarsh
commented
3 months ago Can you run uv with --verbose, and we can look at the decisions it's making and why?
taranlu-houzz
commented
3 months ago Yes, that is what the output in this post is from: https://github.com/astral-sh/uv/issues/6281#issuecomment-2299964691
Hmm, maybe I am doing something wrong? This is the full output of the actual command:
uv tool install --python 3.11 --python-preference only-managed --extra-index-url '<internal repo>' --prerelease allow --refresh --verbose my-packageOutput For some reason,
uvis choosing that very old version ofllvmliteas a dep fornumba, but when I install viapipxit chooses a much more recent version:0.43.0.
At the time, I think I was using either uv version 0.3.0 or 0.3.1. Should I update and try again?
charliermarsh
commented
3 months ago Thanks! It's roughly what I was describing above... With that set of dependencies, we end up solving for numpy==2.1.0, while pip ends up with numpy==2.0.1. If you trace the log closely, you'll see that we pick numpy==2.1.0, then try various numba versions until we hit numba==0.52.0rc2, which depends on numpy>=1.15. (All the later numba versions have an upper-bound on NumPy). Once we pick that, we have to pick an llvmlite version that meets llvmlite>=0.35.0.dev0, <0.36.
In the absence of more constraints from the user, both resolutions are valid. We can't really "know" that you would prefer a more recent llvmlite and numba, but a less recent numpy, unless you give us more information.
I might suggest adding numpy>=2, <2.1 to your requirements? Or adding that as a constraint file.
joennlae
commented
3 months ago My story:
So my python version is 3.11.
I fixed numpy<2.0. Only the numba version above numba>=0.57 can be installed in python 3.11 (https://github.com/numba/numba/blob/288a38bbd5a15418a211bf067878dfdf3c139509/setup.py#L23) except for numba==0.51 (which did not set an upper limit yet).
Numba for me is installed because of librosa there they fix numba>=0.51 (https://github.com/librosa/librosa/blob/b2cd8fb803345685152a2b41e752ad09f33d261b/setup.cfg#L93)
So it jumped down to numba=0.51. I can not reproduce it anymore. So, for me, it is fixed now.
taranlu-houzz
commented
3 months ago @charliermarsh Ah, okay, I think that makes sense. Can the constraints file be used directly with the uv tool install command for testing?
charliermarsh
commented
2 months ago @taranlu-houzz -- You can pass the file like uv tool install --with-requirements constraints.txt <tool>.
taranlu-houzz
commented
2 months ago @charliermarsh Sort of a follow-up to this: Is it possible to layer on addition deps to an already installed tool using the method you mentioned above? For example, I have installed coverage using uv tool install coverage --with pytest --with pytest-spec --with pytest-sugar so that I can run uvx coverage run -m pytest on any project.
For one of my projects, I have an additional dep on matplotlib in order to run the tests. I tried using uv tool run --with-requirements constraints.txt coverage run -m pytest, but it seems that it cannot find pytest in the generated venv.
charliermarsh
commented
2 months ago I think uv tool run --with-requirements constraints.txt coverage run -m pytest should work. Could you share a sequence of commands I can use to reproduce?
taranlu-houzz
commented
2 months ago @charliermarsh I'm guessing that it throws out the venv and creates a new one with the specified requirements and runs the tool using that (but that disregards the additional packages from the initial tool installation):
↓2 ~
❯ uv tool uninstall coverage
Uninstalled 2 executables: coverage, coverage3
↓2 ~
❯ uv tool run coverage run -m pytest --version
No module named 'pytest'
↓2 ~
❌ 1 ❯ uv tool install coverage --with pytest --with pytest-spec --with pytest-sugar
Resolved 8 packages in 4ms
Installed 8 packages in 17ms
+ coverage==7.6.1
+ iniconfig==2.0.0
+ packaging==24.1
+ pluggy==1.5.0
+ pytest==8.3.3
+ pytest-spec==4.0.0
+ pytest-sugar==1.0.0
+ termcolor==2.4.0
Installed 2 executables: coverage, coverage3
↓2 ~
❯ uv tool run coverage run -m pytest --version
pytest 8.3.3
/Users/taran/.local/share/uv/tools/coverage/lib/python3.12/site-packages/coverage/control.py:894: CoverageWarning: No data was collected. (no-data-collected)
self._warn("No data was collected.", slug="no-data-collected")
↓2 ~
❯ echo "matplotlib>=3.9.1" > constraints.txt
↓2 ~
❯ touch test_uv_tool_run_with_requirements.py
↓2 ~ via 🐍 v3.12.1
❯ uv tool run --with-requirements constraints.txt coverage run -m pytest ./test_uv_tool_run_with_requirements.py
No module named 'pytest'
↓2 ~ via 🐍 v3.12.1
❌ 1 ❯ uv tool run coverage run -m pytest ./test_uv_tool_run_with_requirements.py
Test session starts (platform: darwin, Python 3.12.5, pytest 8.3.3, pytest-sugar 1.0.0)
rootdir: /Users/taran
plugins: sugar-1.0.0, spec-4.0.0
collecting ...
―――――――――――――――――――――――――――――――――――――――――――――――――――――――― ERROR collecting test_uv_tool_run_with_requirements.py ――――――――――――――――――――――――――――――――――――――――――――――――――――――――
ImportError while importing test module '/Users/taran/test_uv_tool_run_with_requirements.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.local/share/uv/python/cpython-3.12.5-macos-aarch64-none/lib/python3.12/importlib/__init__.py:90: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
test_uv_tool_run_with_requirements.py:1: in <module>
import matplotlib.pyplot as plt
E ModuleNotFoundError: No module named 'matplotlib'
collected 0 items / 1 error
======================================================================= short test summary info ========================================================================
FAILED test_uv_tool_run_with_requirements.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Results (0.11s):
↓2 ~ via 🐍 v3.12.1
❌ 2 ❯ cat test_uv_tool_run_with_requirements.py
import matplotlib.pyplot as plt
def test_basic_plot():
# Create a simple plot
fig, ax = plt.subplots()
ax.plot([1, 2, 3], [1, 4, 9])
# Check if the figure and axes are created
assert fig is not None
assert ax is not None
↓2 ~ via 🐍 v3.12.1
❯ uv --version
uv 0.4.9 (77d278f68 2024-09-10)Sorry, yes, I think that's right. We don't reuse an installed tool if that tool environment doesn't satisfy the request (i.e., uv tool run does not modify existing tools). It's possible that we could layer the --with requirements on top of the existing tools, though it would lead to some confusing interactions (i.e., any --with requirements from the already-installed tool would also be present).
taranlu-houzz
commented
2 months ago Hmm, I don't know if it would really be better to allow for that added complexity... Maybe a different workflow would be better on my end. I was mainly just investigating some options for pulling more project dev deps into a single, unified location.
I ran into this while trying to install an internal package from a private repo that has a dep (or rather, dep of a dep, etc.) on
llvmlite. I am able to successfully install the package when usingpipx, butuvfails to install it.It seems like the issue occurs when trying to install the
llvmlitepackage, which it tries to build (the error below is a little different than the initial error).Platform:
macOS 14.6 (23G80)uv version:uv 0.3.0 (dd1934c9c 2024-08-20)