frewsxcv / rust-crates-index

Rust library for retrieving and interacting with the crates.io index
https://docs.rs/crates-index/
Apache License 2.0
72 stars 37 forks source link

Add HTTP support to SparseIndex #119

Closed Jake-Shadle closed 1 year ago

Jake-Shadle commented 1 year ago

This adds support for creating HTTP requests and parsing HTTP responses for an HTTP sparse index via the http crate. This method follows the sans-io approach to allow the calling crate to decide what HTTP client it wants to use to send HTTP requests so that this crate is not tied to any particular HTTP client implementation, nor does it have to care if the user wants async or not I/O or whatever else.

This is behind the sparse-http feature gate but that can be removed if the feature is deemed acceptable by default since it is fairly lightweight, especially when compared to git.

This implementation leaves it up to the user to decompress gzip encoded server responses, but gzip decompression would be easy to add either behind an additional feature flag, or as part of the sparse-http feature.

Resolves: #87

Jake-Shadle commented 1 year ago

Note this removes the separator parameter from crates_prefix in favor of unconditionally using / it is entirely pointless for how this crate is used as Windows/libgit has no issues understanding / or \ as equivalent path separators.

kornelski commented 1 year ago

Have you tested this on Windows?

I previously got bug reports about Windows actually caring about path separator char: https://github.com/frewsxcv/rust-crates-index/issues/93

Jake-Shadle commented 1 year ago

Ehh I guess I'll revert that, that's annoying.

kornelski commented 1 year ago

Thank you