Open humanzz opened 8 months ago
Interesting thanks for the thorough report!
Related:
This looks like an issue with our uv-extract
crate. At the very least we should provide more information about what's gone wrong.
Yeah. It's very tough to know what's going wrong here with such short message.
Another point on pip install
and its output: verbose
is too chatty, but omitting verbose
is also too silent. pip
on the other hand does print out some useful things like what's being downloaded, what's already satisfied, what index was used, etc. which are useful bits of info, that are much shorter than the chatty verbose output. It'd be nice if uv
does a similar thing.
Yep agree see https://github.com/astral-sh/uv/issues/1569
Awesome, I'll follow there for progress on that!
I've started seeing a similar error on Mac (Apple Silicon), this time when using pip compile
and uv==0.1.11
e.g.
UV_CACHE_DIR=<a cache dir> UV_INDEX_URL=<a custom index> uv pip compile pyproject.toml --all-extras --upgrade -o requirements.lock
error: Failed to download: ipython==8.22.1
Caused by: The wheel ipython-8.22.1-py3-none-any.whl is not a valid zip file
Caused by: an upstream reader returned an error: unexpected BufError
Caused by: unexpected BufError
It looks like maybe this comes from flate2
via async-compression
via rs-async-zip
which I think we call at https://github.com/astral-sh/uv/blob/8d721830db8ad75b8b7ef38edc0e346696c52e3d/crates/uv-extract/src/stream.rs#L22
For decompression this means that more input is needed to continue or the output buffer isn’t large enough to contain the result. The function can be called again after fixing both.
It seems like either:
So, a bit more information - though likely a bit vague.
The setup I have, essentially runs a local proxy on my machine. I checked the logs of that proxy, to check entries when using pip
and when using uv pip
.
When using pip
, logs look fine, no exceptions/errors.
When using uv pip
, I did see some errors along the lines of
ERROR org.glassfish.jersey.server.ServerRuntime$Responder - An I/O error has occurred while writing a response message entity to the container output stream. - org.glassfish.jersey.server.internal.process.MappableException: java.io.IOException: Buffer overflow
...
...
Caused by: java.io.IOException: Buffer overflow.
at org.glassfish.jersey.netty.connector.internal.JerseyChunkedInput.write(JerseyChunkedInput.java:211) ~[jersey-netty-connector-2.31.jar:?]
at org.glassfish.jersey.netty.connector.internal.JerseyChunkedInput.write(JerseyChunkedInput.java:191) ~[jersey-netty-connector-2.31.jar:?]
at org.glassfish.jersey.netty.connector.internal.JerseyChunkedInput.write(JerseyChunkedInput.java:182) ~[jersey-netty-connector-2.31.jar:?]
at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:185) ~[jersey-common-2.35.jar:?]
I feel like I've read somewhere, on some of the issues, that uv
, as an optimization, can stream files to read specific parts it might need. I wonder if maybe there's some weird interaction here, uv
is reading the part and closing the connection, and that proxy is not properly handling that.
I'm not sure, really, but trying to hypothesize what might be wrong, given I don't have access to that proxy code, and I'm Rust-ignorant :(
It's reasonable that we should just fall back to non-range requests when we see a BufError
.
I would recommend reporting that buffer overflow as a bug to the proxy you're using (if you can), it seems like the proxy is failing to buffer the streamed data correctly sometimes.
In case it'd help someone else, I got this BufError
error when the wheel file itself was corrupted.
In our case, the cache was serving a corrupted file (2.5MB), whereas the same file on public pypi is 9.6 MB, and uv was correctly failing when trying to use it.
It was just a bit difficult to understand what was the nature of the failure based on the error BufError
.
Hello,
My team develops locally mostly using MacOS, but we use a CI/CD (Amazon Linux 2) system as part of our development process.
We have a python package, with
pyproject.toml
and we use https://github.com/pyinvoke/invoke to manage our build scripts.The main task in question now in
tasks.py
, usesuv pip install
and looks as followsWe use
uv==0.1.5
.We're trying to move to using
uv
for its speed gains. On MacOS, everything works as expected. On the CI/CD host, we sometimes - more often than not - run into an error that originates from theuv pip install
call in thedevelop
task.The error consistently looks as follows. The error sometimes changes the dependency it's failing to fetch.
I'm attaching the
--verbose
output for a failed run and its successful retry. Note that I've slightly modified this to drop index urls, and package name, but the full list of dependencies should be there at the end of 2nd successful attempt.uv_attempt1_fail.log uv_attempt2_success.log