search

storj / gateway-st

Single-tenant, S3-compatible server to interact with the Storj network
Apache License 2.0
71 stars 19 forks source link

ListObjects(V2): no max_keys specified defaults to 4500 in response #54

Closed amwolff closed 2 years ago

amwolff commented 2 years ago

https://github.com/storj/minio/commit/a982baff27410d3bfc77a13e74987498e9f82277 changed ListObjects(V2)'s behavior to default to 4500 keys per listing limit. We cap the limit to 1000 in the object layer, but MinIO still responds with 4500 because the S3 specification requires the server to respond with the limit that the client has specified or the default limit if no limit was specified.

We need this to pass s3tests.functional.test_s3.test_bucket_list_maxkeys_none from https://github.com/storj/splunk-s3-tests.

wthorp commented 2 years ago

It's worth clarifying that S3 still lists the limit as 1000, so this does appear to be a Minio-specific "fix". https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html

Can we just set maxObjectList back to 1000?

storjBuildBot commented 2 years ago

Change https://review.dev.storj.io/c/storj/minio/+/6895 mentions this issue.

storjBuildBot commented 2 years ago

Change https://review.dev.storj.io/c/storj/gateway/+/6906 mentions this issue.

storjBuildBot commented 2 years ago

Change https://review.dev.storj.io/c/storj/gateway/+/6907 mentions this issue.