irods / irods_resource_plugin_rados

Cacheless Ceph/rados resource plugin for iRODS
BSD 3-Clause "New" or "Revised" License
7 stars 6 forks source link

can't remove files from radosResc #6

Closed kwaegema closed 6 years ago

kwaegema commented 6 years ago

Hi, While testing the plugin, I ran into this issue:

[kwaegema@vulpix01 ~]$ iput -R radosRescLum radostest44
[kwaegema@vulpix01 ~]$ irm radostest44 
remote addresses: xxx.xxx.xxx.xxx ERROR: rmUtil: rm error for /UGent/home/kwaegema/radostest44, status = -1803000 status = -1803000 HIERARCHY_ERROR

putting files is no problem, they are successfully written to the ceph pool. Deleting files from other resources are no problem.

client.irads1
    caps: [mon] allow r
    caps: [osd] allow rw pool=irads1
kwaegema commented 6 years ago

I didn't found this in the log of resource server, but I found this in the icat server's log:

Jan 26 11:50:46 pid:15975 NOTICE: IRADOS_DEBUG irados_resolve_hierarchy_plugin called UNLINK -> test.ugent.be
Jan 26 11:50:46 pid:15975 remote addresses: 127.0.0.1, xxx.xxx.xxx.xxx ERROR: [-] /tmp/tmp0dyJrt/server/api/src/rsDataObjUnlink.cpp:78:int rsDataObjUnlink(rsComm_t *, dataObjInp_t *) :  status [HIERARCHY_ERROR]  errno [] -- message [Failed to find resource hierarchy [for [/UGent/home/kwaegema/radostest49]]
        [-]     /tmp/tmp0dyJrt/server/core/src/irods_resource_redirect.cpp:668:irods::error irods::resolve_resource_hierarchy(const std::string &, rsComm_t *, dataObjInp_t *, std::string &, dataObjInfo_t **) :  status [HIERARCHY_ERROR]  errno [] -- message [Failed to find resource hierarchy [for [/UGent/home/kwaegema/radostest49]]

I tested running the ceph-client test included in the plugin, looks ok:

[root@vulpix01 ceph-test]# ./ceph-client 
Created a cluster handle.
Read the Ceph configuration file.
Parsed command line options.
Connected to the cluster.
Created an ioctx for the pool.
Wrote new object 'hw' 
Set the xattr 'lang' on our object!
Read object hw asynchronously with contents.
Hello World!
Got the xattr 'lang' from object hw!en_US
Removed the xattr 'lang' from our object!
Removed object 'hw'.
trel commented 6 years ago

We'll need a quick audit of the hierarchy and resc_name vs. resc_id logic.

We changed how this works in 4.2+ and this plugin has not been told of those changes.

trel commented 6 years ago

Try this...

diff --git a/irados/libirados.cpp b/irados/libirados.cpp
index 55f6665..b8ee877 100644
--- a/irados/libirados.cpp
+++ b/irados/libirados.cpp
@@ -1298,7 +1298,8 @@ int get_next_fd() {
                      // =-=-=-=-=-=-=-
                      // test the operation to determine which choices to make
                      if ( irods::OPEN_OPERATION == ( *_opr ) ||
-                             irods::WRITE_OPERATION == ( *_opr ) ) {
+                             irods::WRITE_OPERATION == ( *_opr ) ||
+                             irods::UNLINK_OPERATION == ( *_opr ) ) {
                          // =-=-=-=-=-=-=-
                          // call redirect determination for 'get' operation
                          ret = irados_redirect_open( _ctx.prop_map(), file_obj, resc_name, ( *_curr_host ), ( *_out_vote ) );

We added an unlink operation to hierarchy resolution in 4.2. This gives the plugin a chance to have an opinion about the unlink operation.

kwaegema commented 6 years ago

Thanks, this works! I'll add a PR

trel commented 6 years ago

excellent. thanks!