tiering feature for iput

[peggy0135]

hi @trel , I tried different way to use the tiering plugin. but I found that the function only work with "iget" (to trigger the rule working) If I want it to work with "iput" when I want to overwrite the same file in some kind of slow tier. What can I do? or what should I configure?

Thank you

[korydraughn]

@peggy0135 Did something change for you to close this?

[peggy0135]

sorry mistake , I still can not fix the problem.

[trel]

Can you share what you're typing?

What is your expected result?

What is the actual result - are there any errors?

[peggy0135]

step1:

iget my_test_file

then in other terminal I can see the queue show that my test file is moving to the tier0

670188 {"delay_conditions":"<INST_NAME>irods_rule_engine_plugin-unified_storage_tiering-instance</INST_NAME>irods::storage_tiering::maximum_delay_time_in_seconds<PLUSET>24s</PLUSET>","destination-resource":"st_ufs0","group-name":"example_group","md5":"dadd4831ede77bd66ffacbc01cea3609","object-path":"/QCTZone/home/chao/tiering.r","preserve-replicas":false,"rule-engine-instance-name":"irods_rule_engine_plugin-unified_storage_tiering-instance","rule-engine-operation":"irods_policy_data_movement","source-replica-number":"14","source-resource":"st_ufs2","user-name":"chao","verification-type":"catalog"}

---

step2 : after I wait the test file move to the tier2 by tiering plugin

iput -f my_test_file my_test_file

but nothing show in the queue.

Every 3.0s: iqstat
irods01.openlab: Wed Apr 12 12:09:32 2023

No delayed rules pending for user chao

after the two steps , I tried to check the metadata of the test file.

"iget" didn't change the access_time, but "iput" did.

Thank you

[peggy0135]

Hi @trel , Is there any way that we can trigger the "tiering" with "iput"? Thank you.

[trel]

Is there any way that we can trigger the "tiering" with "iput"?

Hmmm.... Not really.

Tiering 'out' happens due to the 'violating query' finding things to enqueue for movement. This query runs regularly in the background under normal configurations.

Restaging happens due to the 'get' request.

[peggy0135]

Is there any way that we can trigger the "tiering" with "iput"?

Hmmm.... Not really.

Tiering 'out' happens due to the 'violating query' finding things to enqueue for movement. This query runs regularly in the background under normal configurations.

Restaging happens due to the 'get' request.

so if I want to make "iput" do the same thing (like overwrite) what can I config? Thank you

[trel]

I believe an iput -f will select a replica (based on voting) and overwrite it. That should happen regardless of how tiering is configured. You could specify the particular replica you want to overwrite with an additional -R option.

I'm still not quite sure what you want/expect to happen. Restaging on an iput doesn't yet make sense to me.

[peggy0135]

I just expect that when I "iput" with no flags, it will write to the tier 0 ( I mean during the overwrite) Thank you.

[trel]

Oh, I see.

No - from the client, you are writing to the 'logical data object' rather than targeting a particular physical device.

So if it happens to be an overwrite of an existing logical data object, the system will need to overwrite a particular physical instantiation of that object, known as a replica - which could be on any storage resource (in a tier or otherwise).

There may be a way to get the behavior you seek, perhaps with a pre-PEP, but it may be too heavyweight/slow to meet your expectations as it would have to do more bookkeeping in the catalog (and/or data movement) to keep things consistent.

[peggy0135]

I see, em......

[peggy0135]

Hi @trel another problem is, is there any other attribute, that we can use as the condition, not just "irods::access_time". And why "using iput to overwite a data" isn''t a "accessing data". ( due to the access time didn''t change) Thank you.