aws / aws-cli

Universal Command Line Interface for Amazon Web Services
Other
15.33k stars 4.08k forks source link

aws s3 sync fails if there is a directory in the target location with the same name as a file in the source location #3218

Open pako-pl opened 6 years ago

pako-pl commented 6 years ago
$ aws s3 sync --delete s3://my-test-bucket/ /backup/my-test-bucket/
download failed: s3://my-test-bucket/item.208 to ../backup/my-test-bucket/item.208 [Errno 21] Is a directory

item.208 is a file in the source S3 bucket and a directory in the target local directory. The situation occured after we moved around some files in S3, including replacing a directory with a file with the same name. It caused our backups to stop running due to "[Errno 21] Is a directory". We would expect "s3 sync" with the "--delete" flag to be able to cope with such a situation.

$ aws --version
aws-cli/1.14.63 Python/2.7.13 Linux/4.9.0-6-amd64 botocore/1.9.16
kyleknap commented 6 years ago

@pako-pl Can you clarify what the expected behavior would be for this scenario? Would it be to?

I think it would be the latter, but it may not be possible to do because the commands are trying to map a key/value store to a file system which has caused other issues like this in the past.

pako-pl commented 6 years ago

The expected behavior would be to delete the item.208 directory from the target and replace it with the item.208 object from S3, as I am running a "sync" with the "--delete" flag.

arochettesteinberg commented 5 years ago

I think I have the same issue here. Any progression? Workaround?

linture commented 4 years ago

same issue here

stani-o commented 2 years ago

This issue happens also when you try to sync a s3 bucket that has equally named files and directories under the same hierarchy level. I assume this happens because the s3 sync download's first the equally named folder locally and then fails when it download's the equally named file.

Can be easily reproduced:

  1. Create a file named "test"
  2. Create a folder "test"
  3. Create a file test2 in the folder test/
  4. aws s3 sync s3://bucket/path .