BUG: s3_list_keys/s3_list_objects

[captchanjack]

Currently using versions: AWS = "1.24.0" AWSS3 = "0.8.3"

Bug: s3_list_keys/s3_list_objects does not list all objects (recursively) as it should.

I've narrowed it down to the 'delimiter' kwarg.

Looks like either the delimiter query kwarg is not being passed correctly to the lower-level function, or that the bug is occuring in the lower-level function :

r = S3.list_objects(bucket, q; aws_config=aws, kwargs...)

To reproduce bug, leave the delimiter kwarg as the default value '/':

using AWS
using AWSS3

AWS_ACCESS_KEY_ID = "fake"
AWS_SECRET_ACCESS_KEY = "key"
AWS_REGION = "ap-southeast-2"
creds = AWSCredentials(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
AWSConfig = global_aws_config(; region=AWS_REGION, creds=creds)

data = s3_list_keys(AWSConfig, "my-bucket")
result = []

for res in result
    push!(result, res)
end

This works:

data = s3_list_keys(AWSConfig, "my-bucket"; delimiter="")

[mattBrzezinski]

I'm not too familiar with Delimiter/Prefix usages, but I just walked through this guide on them using both AWSS3.jl and AWS.jl and can replicate the same results as the tutorial provides.

[mattBrzezinski]

Could you provide an example where you create a new bucket, some objects in it and show how it's not providing them back? It's a bit hard to follow along without a full example.