search

irods / irods_capability_storage_tiering

BSD 3-Clause "New" or "Revised" License
5 stars 10 forks source link

access time should be updated on additional events #200

Open trel opened 2 years ago

trel commented 2 years ago

For 2.7.0, this code determines when access time is applied: https://github.com/irods/irods_capability_storage_tiering/blob/2.7.0/libirods_rule_engine_plugin-storage_tiering.cpp#L74

For 2.7.0, I see:

Which suggests we also need to handle:

alanking commented 2 years ago

I think maybe trim should not update access time.

We should also put touch in the list of APIs which should be added.

kript commented 2 years ago

Is there a plan to release an update as 2.7.1 to address this, and if so do you have an ETA?

Also, could the documentation for the plugin be updated to show which PEP's trigger it please?

cheers

John

cookie33 commented 9 months ago

Are there plan's to implement this for: pep_api_data_obj_rsync_post?

if I do:

[irodstest2]:~
robertv$ irsync i:test_20231127_02.txt i:/frank/home/robertv#igor/test_from_igor_20231128_04.txt -R eudatCache

[irodstest2]:~
robertv$ imeta ls -d /frank/home/robertv#igor/test_from_igor_20231128_04.txt
AVUs defined for dataObj /frank/home/robertv#igor/test_from_igor_20231128_04.txt:
None

The needed AVU is not set.

What needs to be changed to get it to work for pep_api_data_obj_rsync_post. If you give me a pointer I will try to do so.

alanking commented 9 months ago

Yes, pep_api_data_obj_rsync_post is listed in OP. Please let us know if you think of any other PEPs which should be considered.

I think that we just need to add these PEPs to the big if ladder in this function: https://github.com/irods/irods_capability_storage_tiering/blob/67a218520fdc01d07a254a5b4f21aa674a2546ae/libirods_rule_engine_plugin-unified_storage_tiering.cpp#L243

cookie33 commented 9 months ago

Updated code with a debug statement:

    void apply_access_time_policy(
        const std::string&           _rn,
        ruleExecInfo_t*              _rei,
        const std::list<boost::any>& _args) {

        rodsLog(LOG_ERROR, "Entering because of [PEP=%s].", _rn.c_str());

So it shows whith which pep it entered

Did a small test with irsync between two zones (from iRODS 4.2.12 to 4.3.1):

[irodstest2]:~
robertv$ irsync i:test_20231127_02.txt i:/frank/home/robertv#igor/test_20231128_07.txt -R eudatCache

[irodstest2]:~
robertv$ imeta ls -d /frank/home/robertv#igor/test_20231128_07.txt
AVUs defined for dataObj /frank/home/robertv#igor/test_20231128_07.txt:
None

[irodstest2]:~
robertv$ irsync i:test_20231127_02.txt i:/frank/home/robertv#igor/test_20231128_08.txt -R eudatCache

[irodstest2]:~
robertv$ imeta ls -d /frank/home/robertv#igor/test_20231128_08.txt
AVUs defined for dataObj /frank/home/robertv#igor/test_20231128_08.txt:
None

The logfile shows that it entered with: pep_api_data_obj_close_post. I would have expected an rsync pep. But no. 

{"log_category":"legacy","log_level":"error","log_message":"Entering because of [PEP=pep_api_data_obj_close_post].","request_api_name":"DATA_OBJ_CLOSE_AN","request_api_number":673,"request_api_version":"d","request_client_user":"robertv","request_host":"145.100.3.239","request_proxy_user":"rods","request_release_version":"rods4.2.12","server_host":"irodstest1.storage.surfsara.nl","server_pid":31935,"server_timestamp":"2023-11-28T14:18:40.569Z","server_type":"agent","server_zone":"frank"}
{"log_category":"legacy","log_level":"error","log_message":"Entering because of [PEP=pep_api_data_obj_close_post].","request_api_name":"DATA_OBJ_CLOSE_AN","request_api_number":673,"request_api_version":"d","request_client_user":"robertv","request_host":"145.100.3.239","request_proxy_user":"rods","request_release_version":"rods4.2.12","server_host":"irodstest1.storage.surfsara.nl","server_pid":476,"server_timestamp":"2023-11-28T14:24:11.637Z","server_type":"agent","server_zone":"frank"}

It should have updated the AVU if it matches:

            else if("pep_api_data_obj_close_post" == _rn) {
                //TODO :: only for create/write events
                auto it = _args.begin();
                std::advance(it, 2);
                if(_args.end() == it) {
                    THROW(
                        SYS_INVALID_INPUT_PARAM,
                        "invalid number of arguments");
                }

                const auto opened_inp = boost::any_cast<openedDataObjInp_t*>(*it);
                const auto l1_idx = opened_inp->l1descInx;
                if(opened_objects.find(l1_idx) != opened_objects.end()) {
                    auto [object_path, resource_name] = opened_objects[l1_idx];

                    set_access_time_metadata(
                        _rei->rsComm,
                        object_path,
                        "",
                        config->access_time_attribute);
                }
            }

Will test some more.

korydraughn commented 9 months ago

There's also pep_api_replica_close_* which uses JSON as its input structure.