open-io / oio-swift

Openio flavor of the Openstack Swift Gateway and middlewares, allowing access to OpenIO SDS via an Openstack Swift gateway.
https://www.openio.io
Apache License 2.0
23 stars 21 forks source link

[WIP] Allow to use bucket storage policy #220

Open AymericDu opened 4 years ago

AymericDu commented 4 years ago

Enable or disable the use of bucket storage policy.

When enabled, and if a storage policy is defined for a bucket (root container), all objects in this bucket will be uploaded with this storage policy.

The bucket storage policy can optionally be kept in the cache with its old value. To force the update of this value, it is enough to modify a property of this bucket via the AWS CLI (for example: add a tag).

$ $AWS s3 mb "s3://mybucket0"
make_bucket: mybucket0
$ $AWS s3 cp /etc/magic "s3://mybucket0/dir/magic"
upload: ../../../etc/magic to s3://mybucket0/dir/magic            
$ openio --oio-account AUTH_demo object show mybucket0%2Fdir magic
+---------------------------------------+-------------------------------------------------------------------------------------+
| Field                                 | Value                                                                               |
+---------------------------------------+-------------------------------------------------------------------------------------+
| account                               | AUTH_demo                                                                           |
| container                             | mybucket0%2Fdir                                                                     |
| ctime                                 | 1592432652                                                                          |
| hash                                  | 272913026300E7AE9B5E2D51F138E674                                                    |
| id                                    | D3758F1E4FA80500EED85C409D1F62E1                                                    |
| meta.x-object-sysmeta-oio-bucket-name | mybucket0                                                                           |
| meta.x-object-sysmeta-swift3-acl      | {"Owner":"demo:demo","Grant":[{"Grantee":"demo:demo","Permission":"FULL_CONTROL"}]} |
| mime-type                             | application/octet-stream                                                            |
| mtime                                 | 1592432652                                                                          |
| object                                | magic                                                                               |
| policy                                | THREECOPIES                                                                         |
| size                                  | 111                                                                                 |
| version                               | 1592432652154332                                                                    |
+---------------------------------------+-------------------------------------------------------------------------------------+      
$ openio --oio-account AUTH_demo container set mybucket0 --storage-policy TWOCOPIES
$ $AWS s3 cp /etc/magic "s3://mybucket0/dir/magic"
upload: ../../../etc/magic to s3://mybucket0/dir/magic            
$ openio --oio-account AUTH_demo object show mybucket0%2Fdir magic
+---------------------------------------+-------------------------------------------------------------------------------------+
| Field                                 | Value                                                                               |
+---------------------------------------+-------------------------------------------------------------------------------------+
| account                               | AUTH_demo                                                                           |
| container                             | mybucket0%2Fdir                                                                     |
| ctime                                 | 1592432652                                                                          |
| hash                                  | 272913026300E7AE9B5E2D51F138E674                                                    |
| id                                    | C46CD5224FA805005D12AD8A9D1F0FDF                                                    |
| meta.x-object-sysmeta-oio-bucket-name | mybucket0                                                                           |
| meta.x-object-sysmeta-swift3-acl      | {"Owner":"demo:demo","Grant":[{"Grantee":"demo:demo","Permission":"FULL_CONTROL"}]} |
| mime-type                             | application/octet-stream                                                            |
| mtime                                 | 1592432723                                                                          |
| object                                | magic                                                                               |
| policy                                | THREECOPIES                                                                         |
| size                                  | 111                                                                                 |
| version                               | 1592432723848390                                                                    |
+---------------------------------------+-------------------------------------------------------------------------------------+
$ $AWS s3api put-bucket-tagging --tagging 'TagSet=[{Key=storage,Value=policy}]' --bucket mybucket0
$ $AWS s3 cp /etc/magic "s3://mybucket0/dir/magic"
upload: ../../../etc/magic to s3://mybucket0/dir/magic            
$ openio --oio-account AUTH_demo object show mybucket0%2Fdir magic
+---------------------------------------+-------------------------------------------------------------------------------------+
| Field                                 | Value                                                                               |
+---------------------------------------+-------------------------------------------------------------------------------------+
| account                               | AUTH_demo                                                                           |
| container                             | mybucket0%2Fdir                                                                     |
| ctime                                 | 1592432652                                                                          |
| hash                                  | 272913026300E7AE9B5E2D51F138E674                                                    |
| id                                    | B5A448324FA80500BF750ADA9D1F32E1                                                    |
| meta.x-object-sysmeta-oio-bucket-name | mybucket0                                                                           |
| meta.x-object-sysmeta-swift3-acl      | {"Owner":"demo:demo","Grant":[{"Grantee":"demo:demo","Permission":"FULL_CONTROL"}]} |
| mime-type                             | application/octet-stream                                                            |
| mtime                                 | 1592432983                                                                          |
| object                                | magic                                                                               |
| policy                                | TWOCOPIES                                                                           |
| size                                  | 111                                                                                 |
| version                               | 1592432983057591                                                                    |
+---------------------------------------+-------------------------------------------------------------------------------------+