yoshidan / google-cloud-rust

Google Cloud Client Libraries for Rust.
MIT License
233 stars 85 forks source link

GCS: simplify error type #122

Closed danburkert closed 1 year ago

danburkert commented 1 year ago

While using google-cloud-storage to develop an application with strict retry & error handling requirements, I found it difficult to work with the google_cloud_storage::http::Error type. It exposes many error variants, and it's not clear how they should be handled, and many of the variants have a lot of overlap.

This PR simplifies the error type by reducing it to three variants:

  1. GCS errors, returned by the GCS server
  2. HTTP errors (wrapping reqwest::Error)
  3. Token provider errors

Everything else, like the various decoding variants, has been removed. As an application author, I find this much easier to work with, since I don't have to consider how to handle as many types of errors. Reqwest's error type already has variants for failure to decode response bodies, so such errors only need to be handled in one way now.

To implement this, I had to tweak a few areas of the implementation to return a reqwest 'body' or 'decode' error instead of serde or base64 error. Primarily this is done via more uses of Response::json, which in effect pushes error creation into reqwest, who then wraps it appropriately.

Finally, I simplified and fixed some bugs in the resumable upload API. Previously the ChunkSize::new constructor did validation. This validation is unnecessary, since the GCS server will do the validation, and they are ultimately the authority. Indeed, the validation was overly strict and didn't allow empty chunks to be uploaded (which is useful for finishing a streaming upload if you don't know you are done till you have no more bytes to write).

danburkert commented 1 year ago

Looks like I broke signing, I’ll take a look.

danburkert commented 1 year ago

Signing issue should be fixed now.

yoshidan commented 1 year ago

Thanks!