oras-project / rust-oci-client

A Rust crate to interact with OCI registries
Apache License 2.0
86 stars 45 forks source link

Partial Requests and some other pull refactorings #145

Closed SuperTux88 closed 1 week ago

SuperTux88 commented 3 weeks ago

The main reason for this Pull-Request is to allow doing partial requests. This is a very useful feature if you have very slow/unstable internet (like in Germany) and the pull often fails for various reasons (timeout or other connection errors). So it doesn't help if you can download the first few MB of a blob over and over again before the network connection craps out after some minutes and you can start over again. If your upstream registry supports partial requests, you at least can pull the next few MB every time, and eventually get the whole blob.

I only added the streaming version of this function, as I didn't know how to handle it when giving a writer as parameter, as if the upstream registry doesn't support it, it will respond with a full response and there is no good way to signal that (except maybe just return an error if the upstream registry responded with 200 OK, but I didn't really like this idea). But let me know if you want me to add something. For my use-case the streaming version is enough and I can signal if it's a full or partial response using an enum.

I also did some refactoring to re-use some mostly identical code instead of duplicating it.

And then I also changed the return type of the streaming variants to also include the Content-Length header of the response (if available), so you can use that information without reading the whole response (for example to also set that header when proxying). This is a breaking change for the pull_blob_stream(...) function, as it changes the return type, but I hope that's fine as in #124 was already a breaking change for that function, which isn't released yet.

thomastaylor312 commented 1 week ago

@SuperTux88 Sorry for the delay here! Got pulled away to work on a few other things. I'll get to this tomorrow!