Azure / azure-sdk-for-rust

This repository is for the active development of the Azure SDK for Rust. For consumers of the SDK we recommend visiting Docs.rs and looking up the docs for any of libraries in the SDK.
MIT License
708 stars 246 forks source link

Blob storage fails with invalid content range with 0 byte blob #1093

Open magicmaaaaan opened 2 years ago

magicmaaaaan commented 2 years ago

Version: f9ec73563f615553ca097b0457bed8347ad2227d

I'm trying to download a zero byte blob from blob storage using BlobClient::get and am running into errors. Note that this only happens when the blob is empty (0 bytes).

Here's a simple program that throws an error (similar to stream_blob_01.rs).

use azure_storage_blobs::prelude::*;
use futures::stream::StreamExt;

#[tokio::main]
async fn main() -> azure_core::Result<()> {
    let storage_client = ClientBuilder::emulator();
    let blob_client = storage_client.blob_client("test", "empty.txt");
    let mut stream = blob_client.get().into_stream();
    while let Some(res) = stream.next().await {
        println!("{:?}", res.unwrap());
    }
    Ok(())
}
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { context: Full(Custom { kind: DataConversion, error: Error { context: Custom(Custom { kind: DataConversion, error: ParseIntError { kind: Empty } }) } }, "unable to parse header 'HeaderName(\"content-range\"): HeaderValue(\"bytes 0--1/0\")' into azure_core::request_options::content_range::ContentRange") }', sdk/storage_blobs/examples/stream_blob_02.rs:10:30

bytes 0--1/0 cannot be parsed as a ContentRange because the end range is -1. (I'm not sure if this is valid in the first place and may actually be an issue with Azurite.)

But I also tried with the actual blob storage service.

use azure_storage::StorageCredentials;
use azure_storage_blobs::prelude::*;
use futures::stream::StreamExt;

#[tokio::main]
async fn main() -> azure_core::Result<()> {
    let storage_credentials = StorageCredentials::Key("...".into(), "...".into());
    let storage_client = ClientBuilder::new("...", storage_credentials);
    let blob_client = storage_client.blob_client("test", "empty.txt");
    let mut stream = blob_client.get().into_stream();
    while let Some(res) = stream.next().await {
        println!("{:?}", res.unwrap());
    }
    Ok(())
}
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { context: Full(Custom { kind: HttpResponse { status: RequestedRangeNotSatisfiable, error_code: Some("InvalidRange") }, error: HttpError { status: RequestedRangeNotSatisfiable, details: ErrorDetails { code: Some("InvalidRange"), message: None }, headers: {"x-ms-version": "2019-12-12", "content-type": "application/xml", "content-length": "249", "x-ms-request-id": "REDACTED", "server": "Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0", "content-range": "bytes */0", "x-ms-error-code": "InvalidRange", "date": "REDACTED"}, body: b"\xef\xbb\xbf<?xml version=\"1.0\" encoding=\"utf-8\"?><Error><Code>InvalidRange</Code><Message>The range specified is invalid for the current size of the resource.\nRequestId:REDACTED\nTime:REDACTED</Message></Error>" } }, "server returned error status which will not be retried: 416") }', sdk/storage_blobs/examples/stream_blob_03.rs:16:30

In this case, it seems like the Rust SDK sent an invalid request.

bmc-msft commented 2 years ago

I can confirm this fails on normal storage accounts, not just azurite.

bmc-msft commented 2 years ago

Note, we're sending the range with this format:

x-ms-range: bytes=0-16777215

Azure Storage accepts this for files with a length greater than zero, but not for zero-length files.

The specifications for x-ms-range specify the format as bytes=startByte- or bytes=startByte-endByte.

rylev commented 2 years ago

@bmc-msft #1094 fixed this for azurite but you noted this is also an issue when hitting the production services. What needs to be done to fix that issue?