search

aldy120 / s3-note

Note for Amazon S3
0 stars 0 forks source link

Lifecycle #7

Open aldy120 opened 4 years ago

aldy120 commented 4 years ago

lifecycle 要付請求費，相當於轉換額外付錢

S3 Lifecycle Transition request pricing above represents requests to that storage class. 詳見 Lifecycle Transition requests into (per 1,000 requests) https://aws.amazon.com/s3/pricing/

使用情境

轉換成 IA or Onezone-IA 最少要設定 30 天以後

(https://docs.aws.amazon.com/en_us/AmazonS3/latest/dev/lifecycle-transition-general-considerations.html#lifecycle-general-considerations-transition-sc)

Objects must be stored at least 30 days in the current storage class before you can transition them to STANDARD_IA or ONEZONE_IA. For example, you cannot create a lifecycle rule to transition objects to the STANDARD_IA storage class one day after you create them.

Amazon S3 doesn't transition objects within the first 30 days because newer objects are often accessed more frequently or deleted sooner than is suitable for STANDARD_IA or ONEZONE_IA storage.

看不太懂

You can't specify a single lifecycle rule for both INTELLIGENT_TIERING (or STANDARD_IA or ONEZONE_IA) and GLACIER or DEEP_ARCHIVE transitions when the GLACIER or DEEP_ARCHIVE transition occurs less than 30 days after the INTELLIGENT_TIERING, STANDARD_IA, or ONEZONE_IA transition. That's because there is a minimum 30-day storage charge associated with the INTELLIGENT_TIERING, STANDARD_IA, and ONEZONE_IA storage classes.

The same 30-day minimum applies when you specify a transition from STANDARD_IA storage to ONEZONE_IA or INTELLIGENT_TIERING storage. You can specify two rules to accomplish this, but you pay minimum storage charges. For more information about cost considerations, see Amazon S3 Pricing.

S3 的 Glacier 跟 Deep Archive 只能從 S3 介面看到

Glacier 介面看不到。

加密不會變

transition 不會改變 S3 上的加密方式。

Glacier/Deep Archive 相關

Glacier 跟 Deep Archive 的 object 無法及時存取。需要申請解凍。解凍指的是會幫你暫時複製一份物件到 Standard 以供讀取。你可以趁這個機會複製一份到 standard class 。

轉成 Glacier 額外檔案變大了

For each object that is archived to GLACIER or DEEP_ARCHIVE, Amazon S3 adds 32 KB of storage for index and related metadata. This extra data is necessary to identify and restore your object. You are charged GLACIER or DEEP_ARCHIVE rates for this additional storage.

每次放到 Glacier/Deep Archive 都需要額外的一筆錢

Each object that you transition to the GLACIER or DEEP_ARCHIVE storage class constitutes one transition request. There is a cost for each such request. If you plan to transition a large number of objects, consider the request costs. If you are archiving small objects, consider aggregating many small objects into a smaller number of large objects to reduce transition request costs.

解凍的時會會同時支付 Glacier/Deep Archive 儲存費用，和一份解凍的 Reduced Redundancy 的費用

When you restore an archive, you are paying for both the archive (GLACIER or DEEP_ARCHIVE rate) and a copy that you restored temporarily (REDUCED_REDUNDANCY storage rate). For information about pricing, see Amazon S3 Pricing.

aldy120 commented 4 years ago

Expiration

有一些 storage class 有最小的存放時間，如果在這段時間內，使用 lifecycle expiration 刪掉的話，儲存費用還是照算到這段時間過完。

Storage Class Minimum Storage Time
Standard-IA 30 days
Intelligent-Tiering 30 days
Onezone-IA 30 days
Glacier 90 days
Deep Archive 180 days
aldy120 commented 4 years ago

如果用 lifecycle 來 archive 的話，那請求的費用是根據 lifecycle rule 的數量來算還是 object 的數量來算?

object 的數量

aldy120 commented 4 years ago

Expiration 不會刪掉 delete marker 除非沒有舊版了

If the current object version is the only object version and it is also a delete marker (also referred as an expired object delete marker, where all object versions are deleted and you only have a delete marker remaining), Amazon S3 removes the expired object delete marker. You can also use the expiration action to direct Amazon S3 to remove any expired object delete markers. For an example, see Example 7: Removing Expired Object Delete Markers.

aldy120 commented 3 years ago

https://aws.amazon.com/premiumsupport/knowledge-center/s3-lifecycle-rule-intelligent-tiering/

Objects smaller than 128 KB – For the following transitions, Amazon S3 does not transition objects that are smaller than 128 KB:

From the S3 Standard or S3 Standard-IA storage classes to S3 Intelligent-Tiering or S3 Glacier Instant Retrieval.

From the S3 Standard storage class to S3 Standard-IA or S3 One Zone-IA.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-transition-general-considerations.html

aldy120 commented 2 years ago

[] Transitioning objects using Amazon S3 Lifecycle - Constraints - https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-transition-general-considerations.html#lifecycle-configuration-constraints

某些情況下小於 128KB 的無法被 transition 。

Objects smaller than 128 KB – For the following transitions, Amazon S3 does not transition objects that are smaller than 128 KB:

From the S3 Standard or S3 Standard-IA storage classes to S3 Intelligent-Tiering or S3 Glacier Instant Retrieval.

From the S3 Standard storage class to S3 Standard-IA or S3 One Zone-IA.

aldy120 commented 2 years ago

無法建立未滿 30 天的 lifecycle transition 到 IA 系列 (Standard IA/Onezone IA)。

Before you transition objects from the S3 Standard or S3 Standard-IA storage classes to S3 Standard-IA or S3 One Zone-IA, you must store them at least 30 days in the S3 Standard storage class. For example, you cannot create a Lifecycle rule to transition objects to the S3 Standard-IA storage class one day after you create them. Amazon S3 doesn't transition objects within the first 30 days because newer objects are often accessed more frequently or deleted sooner than is suitable for S3 Standard-IA or S3 One Zone-IA storage.

aldy120 commented 2 years ago

IA 系列 (Standard IA/Onezone IA) 有 30 天低消。

The S3 Standard-IA and S3 One Zone-IA storage classes have a minimum 30-day storage charge. Therefore, you can't specify a single Lifecycle rule for both an S3 Standard-IA or S3 One Zone-IA transition and a S3 Glacier Flexible Retrieval or S3 Glacier Deep Archive transition when the S3 Glacier Flexible Retrieval or S3 Glacier Deep Archive transition occurs less than 30 days after the S3 Standard-IA or S3 One Zone-IA transition.

The same 30-day minimum applies when you specify a transition from S3 Standard-IA storage to S3 One Zone-IA. You can specify two rules to accomplish this, but you pay minimum storage charges. For more information about cost considerations, see Amazon S3 pricing.

aldy120 commented 2 years ago

Glacier Flexible Retrieval and Deep Archive 需要額外的 8 + 32 KB 的空間。 8KB (Standard 計價)用來 list 32KB (原計價類別計價) 用來 restore

Glacier Flexible Retrieval and Deep Archive 有儲存費用低消 Flexible: 90 days Deep Archive: 180 days

轉到 Glacier Flexible/Deep Archive 的轉換費雖然是一次性的，但其實沒有很便宜。

aldy120 commented 2 years ago

Restore 會生一份 Reduced Redundancy Storage 級別的暫存，是要收儲存費的。

aldy120 commented 2 years ago

delay 原則上收費是依照 lifecycle 定的來算，不會多扣錢。舉例來說延遲刪除不會多收 storage 費用。

但有例外。

The one exception to this behavior is if you have a lifecycle rule to transition to the S3 Intelligent-Tiering storage class. In that case, billing changes do not occur until the object has transitioned to S3 Intelligent-Tiering. https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html

aldy120 commented 2 years ago

已經存在的舊的 object 也會受到 lifecycle rule 影響。

Existing and new objects

When you add a Lifecycle configuration to a bucket, the configuration rules apply to both existing objects and objects that you add later. For example, if you add a Lifecycle configuration rule today with an expiration action that causes objects with a specific prefix to expire 30 days after creation, Amazon S3 will queue for removal any existing objects that are more than 30 days old.

aldy120 commented 2 years ago

似乎只有 expiration 有時間，但 transition 沒有辦法用 head-object 取得到期時間。

aldy120 commented 1 year ago

EFS

lifecycle 可以設定進去 IA (Infrequent Access) 跟出來的條件。進去的條件通常是一段時間沒有存取 (此時間是 EFS 內部自行計算，不會在檔案的 metadata )。

從 IA 出來的條件可以選擇被存取的自動出來，或是不要出來。

小於 128 KB 不會進入 IA 。

https://docs.aws.amazon.com/efs/latest/ug/lifecycle-management-efs.html