search

bouncestorage / swiftproxy

Access other storage backends via the Swift API
Apache License 2.0
31 stars 2 forks source link

Listing contents may report a bucket twice and results out of order from AWS S3 #59

Open timuralp opened 7 years ago

timuralp commented 7 years ago

I'm observing this with the following container names: s3-api-test-bucket, s3-apit-test-bucket-frankfurt, ss-bench-test-oregon, and swiftstack-bounce.

Specifically, the results for the LIST request look as follows:

{"name":"s3-api-test-bucket","count":0,"bytes":0},
{"name":"ss-api-test-bucket-frankfurt","count":0,"bytes":0},
{"name":"o.swiftstack.org-sync","count":0,"bytes":0},
{"name":"swiftstack-bounce","count":0,"bytes":0},
{"name":"ss-bench-test-oregon","count":0,"bytes":0}

The expected order would be: o.swiftstack.org-sync, s3-api-test-bucket, ss-api-test-bucket-frankfurt, ss-bench-test-oregon, swiftstack-bounce.

The client then submits one more request with the marker set to ss-bench-test-oregon, which returns one more result swiftstack-bounce. Setting the marker to that container returns no new results, which is correct.

I wonder if this is related to the AWS LIST API changes? Will try to investigate.

gaul commented 7 years ago

AccountResource.getAccount generates this response. It sorts entries for only the transient blobstore, curious since BlobStore.list() should always return sorted output. AWS has not changed any of its APIs; it has only added a second list keys RPC.