Open cookie33 opened 6 months ago
It only does it for files which are in both tiers.
What should happen if there are three or more tiers? Would your sentence read ... 'all tiers'?
So this policy would remove the oldest duplicated files, in order, if they are larger than a certain size, and until a threshold value is met?
Policy for the ordering could be customizable - or just part of an enum... ['oldest', 'largest', 'smallest']
It only does it for files which are in both tiers.
What should happen if there are three or more tiers? Would your sentence read ... 'all tiers'?
I would say if it is in more than one tier. If it is in more than one tier it can do a trim of the resource if it is on a other resource. f.i. if you have three tiers: Tier0, Tier1, Tier2 and a file: file_A. file_A has 2 copies file_A ==> Tier0 file_A ==> Tier2 So it can trim it from Tier0 because there is a copy on Tier2 and a metadata AVU which says so on resource Tier0
So this policy would remove the oldest duplicated files, in order, if they are larger than a certain size, and until a threshold value is met?
Policy for the ordering could be customizable - or just part of an enum... ['oldest', 'largest', 'smallest']
For now my rule as implemented just does following:
Got it. But a 'minimum filesize to trim' might also be a handy knob to have.
FEATURE REQUEST
VERSIONS
iRODS 4.3.1
proposed BEHAVIOUR
A system with tier 0 and tier 1. The parameter
irods::storage_tiering::preserve_replicas
is set to true for all tiers.A new parameter
irods::storage_tiering::preserve_replicas_size
is set to the number of bytes to keep on tier0When the storage tiering rule fires of it also checks for the object size in the tier0. If it is too much it trims files based on date (oldest first) till it reaches the limit. This might also be configarable like the query to find files to migrate. It only does it for files which are in both tiers. During the operation it also updates the attribute
irods::storage_tiering::group
with a new unit if it is neededAn example of how we now implement it in a python iRODS rule