seung-lab / cloud-files

Threaded Python and CLI client library for AWS S3, Google Cloud Storage (GCS), in-memory, and the local filesystem.
BSD 3-Clause "New" or "Revised" License
36 stars 8 forks source link

Verify for S3 Multi-Part #71

Open william-silversmith opened 2 years ago

william-silversmith commented 2 years ago

We've currently disabled verification for multi-part, but it seems there's a hidden feature to get the correct information.

 curl --head "https://s3.amazonaws.com/bossdb-open-data/iarpa_microns/minnie/minnie65/em/8_8_40/03b8a.shard?partNumber=2"
HTTP/1.1 206 Partial Content
x-amz-id-2: +YQe6D4SiKQGldTUf4/mjPxYkTuxrwQ5zVyHcEpL2sd52DvUkzp7xIIOhZflShCVLPPOowBr57o=
x-amz-request-id: QM4Y96HJHVFGJEP0
Date: Tue, 07 Dec 2021 20:41:16 GMT
Last-Modified: Thu, 01 Jul 2021 16:03:34 GMT
ETag: "dc51d2d05a7682bde56128b4b88d7e1c-2"
x-amz-mp-parts-count: 2
x-amz-storage-class: INTELLIGENT_TIERING
Accept-Ranges: bytes
Content-Range: bytes 4999341932-5105184407/5105184408
Content-Type: binary/octet-stream
Server: AmazonS3
Content-Length: 105842476