Using together with XRootD http UID is correct, but not GID

[juztas]

From the mailing list

J: Using HTTPS and writing file, it uses correct uid, but not gid. This kind of allows anyone to delete file and gid becomes root... Should there be something more in sec.protocol?
B: No, this should be managed by Authfile, not sec.protocol.  That limits user's permissions prior to getting to the HDFS layer (note that XRootD has a set of authorization it applies as well as HDFS... getting the two to line up is tough).

[1]
lcmaps[27549]  LOG_NOTICE: 2018-07-03.07:53:38Z: LCMAPS CRED FINAL: DN:"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jbalcas/CN=751133/CN=Justas Balcas"->mapped uid:'26455',pgid:'504',sgid:'504'
lcmaps[27549]   LOG_DEBUG: 2018-07-03.07:53:38Z: lcmaps_run_with_stack_of_x509_and_return_account(): LCMAPS could not find the poolindex
lcmaps[27549]   LOG_DEBUG: 2018-07-03.07:53:38Z: lcmaps_run_with_stack_of_x509_and_return_account(): succeeded
INFO in AuthzFun: Got uid 26455
[2]
[root@transfer-2 ~]# ls -l /mnt/hadoop/store/temp/user/jbalcas.11111111/
total 135
-rw-rw-r--. 1 cmsuser root 69173 Jul  3 00:43 listing10
-rw-rw-r--. 1 cmsuser root 69173 Jul  3 00:53 listing11
[3]
[root@transfer-2 ~]# id -a cmsuser
uid=26455(cmsuser) gid=504(cms) groups=504(cms)

More info on the server setup:

auth_file (this was just dummy for testing):
u * /store ldrwi
from xrootd config file:
# Enable https over XrootD
if exec xrootd
  xrd.protocol http:1095 /usr/lib64/libXrdHttp.so
  http.cadir /etc/grid-security/certificates
  http.cert /etc/grid-security/xrootd/xrootdcert.pem
  http.key /etc/grid-security/xrootd/xrootdkey.pem
  http.secxtractor /usr/lib64/libXrdLcmaps.so
  http.listingdeny yes
  http.desthttps no
  http.trace all -debug
  http.selfhttps2http no
  http.staticpreload http://static/robots.txt /etc/xrootd/robots.txt
fi

Let me know if you need more log for this

[bbockelm]

Hi Justas,

Actually, any problem would be in the xrootd-hdfs plugin, not LCMAPS. Looking at the xrootd-hdfs code, the API we use only provides the username:

https://github.com/opensciencegrid/xrootd-hdfs/blob/master/src/XrdHdfs.cc#L62

So it's the HDFS library that is providing the group information. IIRC, when not provided, it uses the group information from the OS on the HDFS NN.

So:

1. What is the output of id -a cmsuser on the NN?
2. What's the output of ls -ld /mnt/hadoop/store/temp/user/jbalcas.11111111? That is, what are the directory permissions of the parent directory?
3. Depending on the answer to (2), it may be useful to create the files as the cmsuser using the HDFS CLI.

It may be that there isn't actually an issue here.

Brian

[juztas]

[root@transfer-2 ~]# id -a cmsuser
uid=26455(cmsuser) gid=504(cms) groups=504(cms)
[root@transfer-2 ~]# ls -ld /mnt/hadoop/store/temp/user/jbalcas.11111111
drwxrwxrwx. 8 root root 4096 Jul  3 02:51 /mnt/hadoop/store/temp/user/jbalcas.11111111
[root@transfer-2 ~]# ls -ld /mnt/hadoop/store/temp/user
drwxrwxrwx. 920 cmsprod cms 4096 Jul 12 01:53 /mnt/hadoop/store/temp/user
[root@transfer-2 ~]# sudo -u cmsuser ls -l /mnt/hadoop/store/temp/user/jbalcas.11111111
total 405
-rw-rw-r--. 1 cmsuser root 69173 Jul  3 00:43 listing10
-rw-rw-r--. 1 cmsuser root 69173 Jul  3 00:53 listing11
-rw-rw-r--. 1 cmsuser root 69173 Jul  3 02:43 listing11111111
-rw-rw-r--. 1 cmsuser root 69173 Jul  3 02:51 listing11231
-rw-rw-r--. 1 cmsuser root 69173 Jul  3 01:02 listing12
-rw-rw-r--. 1 cmsuser root 69173 Jul  3 01:03 listing4
[root@transfer-2 ~]# rm -rf /mnt/hadoop/store/temp/user/jbalcas.11111111
[root@transfer-2 ~]# stat /mnt/hadoop/store/temp/user
  File: ‘/mnt/hadoop/store/temp/user’
  Size: 4096        Blocks: 8          IO Block: 512    directory
Device: 2ch/44d Inode: 106         Links: 919
Access: (0777/drwxrwxrwx)  Uid: (20007/ cmsprod)   Gid: (  504/     cms)
Context: system_u:object_r:fusefs_t:s0
Access: 1969-12-31 16:00:00.000000000 -0800
Modify: 2018-07-13 01:14:46.000000000 -0700
Change: 2018-07-13 01:14:46.000000000 -0700
 Birth: -

I am cleaning up, but somehow 777 does not sound right to me

[juztas]

Boh... closing, Not sure where the issue was, but testing now with phedex, cmsuser all works out of the box. (Maybe because of the auth misconfig, not sure )

Thanks