search

peak / s5cmd

Parallel S3 and local filesystem execution tool.
MIT License
2.72k stars 241 forks source link

Add --skip-existing #648

Closed markkdev closed 1 year ago

markkdev commented 1 year ago

Hi,

I was wondering if there were any plans in the future to include a --skip-existing flag similar to the one provided by s3cmd.

P.S. wanted to thank you for this wonderful tool, our asset upload time dropped from 2-5mins to 8-12s after migrating from s3cmd!

k0ste commented 1 year ago

@markkdev --if-size-differ and --if-source-newer is not the same?

igungor commented 1 year ago

@markkdev we have such a feature under the flag --no-clobber. You may use it like below:

$ s5cmd --log=debug cp file1 s3://bucket/ibrahim/             
cp file1 s3://bucket/ibrahim/file1  # initial upload

$ s5cmd --log=debug cp file1 s3://bucket/ibrahim/
cp file1 s3://bucket/ibrahim/file1  # override destination object

$ s5cmd --log=debug cp --no-clobber file1 s3://bucket/ibrahim/
DEBUG "cp file1 s3://bucket/ibrahim/file1": object already exists