s3tools / s3cmd

Official s3cmd repo -- Command line tool for managing S3 compatible storage services (including Amazon S3 and CloudFront).
https://s3tools.org/s3cmd
GNU General Public License v2.0
4.56k stars 905 forks source link

expire command should allow setting multiple rules with different prefixes #863

Open kdvolder opened 7 years ago

kdvolder commented 7 years ago

For example:

I attempted to set expiration times for several folders like this:

s3cmd expire s3://my.bucket \
    --expiry-prefix weekly --expiry-days 7 \
    --expiry-prefix monthly --expiry-days 30

But it seems the effect is only to apply the last provided rule.

I've also tried adding the rules one by one (by separate commands) but the effect is the same. The last command overwrites the previous one instead of adding an additional rule.

kdvolder commented 7 years ago

Note: s3 does allow multiple rules for a bucket (upto 100 according to this: https://aws.amazon.com/blogs/aws/amazon-s3-object-expiration/)

Dhimantjadeja commented 6 years ago

Does this command remove the object from S3 bucket after the specified number of days in --expiry-days ?

pdrocaldeira commented 6 years ago

Almost a year and nothing changed I think. Is there a workaround?

kdvolder commented 6 years ago

Is there a workaround?

Define the rules using something other than s3cmd? Other than that I couldn't come up with anything.

pdrocaldeira commented 6 years ago

In stack overflow they suggest aws-cli but it have the same problem :(

https://stackoverflow.com/questions/49615977/multiple-lifecycles-s3cmd

tgmedia-nz commented 5 years ago

Might be old but you CAN use awscli, like so

s3bucket-lifecycle-policy.json:

{
    "Rules": [
        {
            "Expiration": {
                "Days": 61
            },
            "ID": "Y2Q1YzIwNGItY2YxNS00ZTdkLWFkMzQtYTYxNmExNTY3YWIz",
            "Prefix": "daily-backup",
            "Status": "Enabled",
            "AbortIncompleteMultipartUpload": {
                "DaysAfterInitiation": 7
        }
        },
        {
            "Expiration": {
                "Days": 40
            },
            "ID": "Y2Q1YzIwNGItY2YxNS00asdfasfsfYxNmExNTY3YWIz",
            "Prefix": "dbbackups",
            "Status": "Enabled",
            "AbortIncompleteMultipartUpload": {
                "DaysAfterInitiation": 7
            }
        }
    ]
}

aws s3api put-bucket-lifecycle-configuration --bucket mybucket --lifecycle-configuration file://s3bucket-lifecycle-policy.json

Shouldn't be hard to implement for s3cmd :)