peak / s5cmd

Parallel S3 and local filesystem execution tool.
MIT License
2.71k stars 241 forks source link

rm doesn't delete 0-byte folder placeholders #707

Open iainelder opened 9 months ago

iainelder commented 9 months ago

I expect these commands to empty and then delete bucket1.

s5cmd rm --all-versions "s3://bucket1/*"
s5cmd rb "s3://bucket1"

Instead the rb operation failed because the bucket is still not empty.

ERROR "rb s3://bucket1": BucketNotEmpty: The bucket you tried to delete is not empty status code: 409, request id: ..., host id: ...

I check what objects remain.

aws s3api list-objects-v2 --bucket hg-mediadata-inventory | jq -c '.Contents[] | {Key, Size}'

The bucket still contains 0-byte objects that act as "folder" placeholders.

{"Key":"bucket2/inventory1/data/","Size":0}
{"Key":"bucket3/inventory1/data/","Size":0}
{"Key":"bucket4/inventory1/data/","Size":0}
...

The S3 inventory service created the keys. bucket1 stores inventory snapshots for the other buckets in the account.

Whatever created them, s5cmd's rm command should delete them when I use the command to empty the bucket.