create-default-dirs-DPM.sh cron

[jrevillard]

Dear all,

I just have a quick question about the old "create-default-dirs-DPM.sh" cron job that was created by yaim before. Should it still be created ? and if yes, shoudn't it be managed by this module too ?

Best, Jerome

[andrea-manzi]

Hi Jerome, the cron was needed on old versions of DPM < 1.8.7. Now the folders are created by dmlite when writing if they are not present. cheers Andrea

[jrevillard]

Hi Andrea,

Thanks for the quick answer. Then I have a problem because this not the case for me .... well, the system try to create it but fail because the user who try to create this directory does not have to correct group id ....

Best, Jerome

[jrevillard]

In fact, the problem occurs for a user who is in multiple VO groups, and the primary group is not the one allowed to create the files

09/12 11:38:48.648  2524,0 Cns_srv_getgrpbygids: NS092 - getgrpbygids request by dpmmgr (0,0) from ng-hug-server3.neuro.hcuge.ch
09/12 11:38:48.648  2524,0 Cns_srv_getgrpbygids: NS098 - getgrpbygids 104       
09/12 11:38:48.648  2524,0 Cns_srv_getgrpbygids: NS098 - getgrpbygids 105       
09/12 11:38:48.648  2524,0 Cns_srv_getgrpbygids: NS098 - getgrpbygids 101       
09/12 11:38:48.648  2524,0 Cns_srv_getgrpbygids: NS098 - getgrpbygids 120       
09/12 11:38:48.648  2524,0 Cns_srv_getgrpbygids: NS098 - getgrpbygids 122       
09/12 11:38:48.648  2524,0 Cns_srv_getgrpbygids: returns 0                      
09/12 11:38:48.696  2524,0 Cns_srv_getgrpbygid: NS092 - getgrpbygid request by /O=dutchgrid/O=users/O=vu/OU=vumc/CN=XXX from ng-hug-server3.neuro.hcuge.ch
09/12 11:38:48.696  2524,0 Cns_srv_getgrpbygid: NS098 - getgrpbygid 104         
09/12 11:38:48.696  2524,0 Cns_srv_getgrpbygid: returns 0                       
09/12 11:38:48.742  2524,0 Cns_srv_stat: NS092 - stat request by /O=dutchgrid/O=users/O=vu/OU=vumc/CN=XXX from ng-hug-server3.neuro.hcuge.ch
09/12 11:38:48.742  2524,0 Cns_srv_stat: NS098 - stat 0 /dpm/neuro.hcuge.ch/home/vo.neugrid.eu/generated/2016-09-12//file-3984846e-a4e7-487d-85c6-d6fe497a6db4
09/12 11:38:48.744  2524,0 Cns_srv_stat: returns 2                              
09/12 11:38:50.518  2524,0 Cns_srv_creat: NS092 - creat request by /O=dutchgrid/O=users/O=vu/OU=vumc/CN=XXX from ng-hug-server3.neuro.hcuge.ch
09/12 11:38:50.518  2524,0 Cns_srv_creat: NS098 - creat /dpm/neuro.hcuge.ch/home/vo.neugrid.eu/generated/2016-09-12//file-3984846e-a4e7-487d-85c6-d6fe497a6db4  664 0
09/12 11:38:50.520  2524,0 Cns_srv_creat: returns 2                             
09/12 11:38:50.569  2524,0 Cns_srv_unlink: NS092 - unlink request by /O=dutchgrid/O=users/O=vu/OU=vumc/CN=XXX from ng-hug-server3.neuro.hcuge.ch
09/12 11:38:50.569  2524,0 Cns_srv_unlink: NS098 - unlink /dpm/neuro.hcuge.ch/home/vo.neugrid.eu/generated/2016-09-12//file-3984846e-a4e7-487d-85c6-d6fe497a6db4
09/12 11:38:50.569  2524,0 Cns_srv_unlink: returns 2  

> dpns-ls -l /dpm/neuro.hcuge.ch/home/vo.neugrid.eu
drwxrwxr-x 1301 root     101                       0 Sep 13 17:43 generated

101 != 104.

It seems that it was working previously so something changed somewhere... :-) (certainly not in the puppet module...)

Best, Jerome

[andrea-manzi]

from what i see the generated folder should be owned by the VO group ( in this case vo.neugrid.eu). Is the vo.neugrid.eu group corresponding to 101?

[jrevillard]

Yes it is. but as the user primary group is 104, it fails

[andrea-manzi]

well, i don't think that something has changed on DPNS w.r.t. this behaviour.

the cron has always created the generated folder with the vo gid ownership

cat <<'EOF' >> create-default-dirs-DPM.sh for i in seq 0 62; do for vo in echo $VOS | tr '[:upper:]' '[:lower:]'; do mydate=date +%Y-%m-%d -d "+$i days" mypath="/dpm/$MYDOMAIN/${DPNSBASEDIR}/$vo/generated/$mydate" mygenpath="/dpm/$MYDOMAIN/${DPNSBASEDIR}/$vo/generated/" echo "Creating and setting correct permissions on: $mypath" ${LCG_LOCATION}/bin/dpns-mkdir -p ${mypath} ${LCG_LOCATION}/bin/dpns-chmod 775 ${mypath} ${LCG_LOCATION}/bin/dpns-chown root:$vo ${mypath} ${LCG_LOCATION}/bin/dpns-setacl -m d:u::7,d:g::7,d:o:5 ${mypath} ${LCG_LOCATION}/bin/dpns-chmod 775 ${mygenpath} ${LCG_LOCATION}/bin/dpns-chown root:$vo ${mygenpath} ${LCG_LOCATION}/bin/dpns-setacl -m d:u::7,d:g::7,d:o:5 ${mygenpath} done done

which version of DPM are you running and which protocol the user is using for writing?

maybe is better to open a GGUS ticket for this, as i don't think this is an issue with this puppet module.

[jrevillard]

[root@ng-hug-server3:~] # rpm -qa |grep dpm
dpm-libs-1.8.11-1.el6.x86_64
dpm-server-mysql-1.8.11-1.el6.x86_64
dpm-1.8.11-1.el6.x86_64
dpm-python-1.8.11-1.el6.x86_64
dpm-name-server-mysql-1.8.11-1.el6.x86_64
dpm-srm-server-mysql-1.8.11-1.el6.x86_64
dpm-rfio-server-1.8.11-1.el6.x86_64
dpm-dsi-1.9.7-7.el6.x86_64

I will open a GGUS ticket thanks

[jrevillard]

FYI I found the problem.

This is only happening in our graphical user interface. It uses JSAGA to interact with the DPM. doing this, the stat or create commands contain double slashes which is not the case using the glite UI.

So, in the following case, DPNS is not able to create the parent directory:

09/14 16:38:54.308  6978,0 Cns_srv_creat: NS098 - creat /dpm/neuro.hcuge.ch/home/vo.neugrid.eu/generated/2016-09-14//file-2c575d8b-a0b5-4b93-90cd-842762d8fb57  664 0
09/14 16:38:54.309  6978,0 Cns_srv_creat: returns 2

whereas in this case, it works:

09/14 16:36:40.662  6978,0 Cns_srv_creat: NS098 - creat /dpm/neuro.hcuge.ch/home/vo.neugrid.eu/generated/2016-09-14/filecc950c70-a9da-4c4b-b9d0-07968b0a11fa  664 0
09/14 16:36:40.665  6978,0 Cns_srv_creat: file 620299 created
09/14 16:36:40.665  6978,0 Cns_srv_creat: returns 0

Best, Jerome

[jrevillard]

FYI: https://ggus.eu/index.php?mode=ticket_info&ticket_id=123897