list_objects response fails to decode

[kmathew]

Error: HttpClient(reqwest::Error { kind: Decode, source: Error("missing fieldcrc32c", line: 23, column: 5) })

    let list_result = client
        .list_objects(&ListObjectsRequest {
            bucket: "bucketname".to_string(),
            ..Default::default()
        })
        .await?;

The code above fails to properly decode the response. It's requiring fields that are available only when getting a specific object.

Typical JSON Response:

{
  "items": [
    {
      "eventBasedHold": false, 
      "kind": "storage#object", 
      "contentType": "text/plain", 
      "name": "thumbnails/", 
      "etag": "CP+Qir3Onf4CEAE=", 
      "generation": "1681071265712255", 
      "temporaryHold": false, 
      "bucket": "bucketname", 
      "updated": "2023-04-09T20:14:25.713Z", 
      "metageneration": "1", 
      "mediaLink": "https://storage.googleapis.com/download/storage/v1/b/bucketname/o/thumbnails%2F?generation=1681071265712255&alt=media", 
      "kmsKeyName": "kmskey/path/name", 
      "timeStorageClassUpdated": "2023-04-09T20:14:25.713Z", 
      "size": "0", 
      "timeCreated": "2023-04-09T20:14:25.713Z", 
      "id": "bucketname/thumbnails//1681071265712255", 
      "selfLink": "https://www.googleapis.com/storage/v1/b/bucketname/o/thumbnails%2F", 
      "storageClass": "STANDARD"
    }, 
    {
      "kind": "storage#object", 
      "contentType": "image/png", 
      "name": "thumbnails/Screenshot 2023-01-27 at 11.54.48 AM.png", 
      "etag": "CJzcx/nSnf4CEAE=", 
      "generation": "1681072466292252", 
      "bucket": "bucketname", 
      "updated": "2023-04-09T20:34:26.293Z", 
      "metageneration": "1", 
      "mediaLink": "https://storage.googleapis.com/download/storage/v1/b/bucketname/o/thumbnails%2FScreenshot%202023-01-27%20at%2011.54.48%20AM.png?generation=1681072466292252&alt=media", 
      "kmsKeyName": "kmskey/path/name", 
      "timeStorageClassUpdated": "2023-04-09T20:34:26.293Z", 
      "size": "207122", 
      "timeCreated": "2023-04-09T20:34:26.293Z", 
      "id": "bucketname/thumbnails/Screenshot 2023-01-27 at 11.54.48 AM.png/1681072466292252", 
      "selfLink": "https://www.googleapis.com/storage/v1/b/bucketname/o/thumbnails%2FScreenshot%202023-01-27%20at%2011.54.48%20AM.png", 
      "storageClass": "STANDARD"
    }
  ], 
  "kind": "storage#objects"
}

Need to define a struct to handle for list object's case or make some of the existing fields in Object optional.

[kmathew]

Looked further into this and we can update this to be:

    #[serde(skip_serializing_if = "Option::is_none")]
    pub crc32c: Option<String>,

There may be additional fields that need the same treatment. Made a PR

[danburkert]

I've never observed the crc32c being absent in a list objects response - the md5Hash field is documented to be optional, but not the crc32c. Is there something special about these objects?

[DoumanAsh]

@danburkert

In the JSON API, the objects resource md5Hash and crc32c properties contain base64-encoded MD5 and CRC32C hashes, respectively. Providing either metadata property is optional. Supplying either property as part of a resumable upload or JSON API multipart upload triggers server-side validation for the new object. If Cloud Storage calculates a value for either property that does not match a supplied value, the object is not created. If the properties are not provided in an upload, Cloud Storage calculates the values and writes them to the object's metadata.

https://cloud.google.com/storage/docs/hashes-etags#json-api

Because either of these can be omitted when writing object, crc32c in fact can be optional. At least that's how I read it

[kmathew]

crc32c is an optional field and was the only field with the issue. The PR to treat it as optional was merged and this issue is resolved.