Open vokac opened 1 year ago
Not to detract from your request, but some notes.
The chown
Chimera command accepts a two-part argument: the (user-)owner and the group-owner, separated by a colon. Unless you want to change the group-ownership of files with different user-owners, the chown
Chimera command should work for your chgrp
use-case.
For all your desired changes, one option is to NFS-mount dCache and make changes using regular Linux admin tools (see chown(1)
, chgrp(1)
, chmod(1)
and nfs4_setfacl(1)
). These all support recursive operations (-R
). Note that you need to use nfs4_setfacl
(and not setfacl(1)
) for "reasons".
Tested NFS ACL update on ~ 6M directories and it is 3.6x slower (~ 34 hours) than updating individual directories with chimera
(~ 9 hours). Our typical storage have roughly 30M directories so even NFS ACL update should take less than a week, but it would still be nice to have more efficient recursive chimera
option.
Thanks for benchmarking the different options, @vokac.
It's perhaps not a surprise that NFS introduces "some" overhead. NFS is quite a "chatty" protocol. I would imagine that the overhead will likely depend on the network RTT between client and server, and also how much effort the client makes to optimised the NFS requests.
I don't know if the NFS approach being over three times slower is expected. It could be that dCache's NFS interface could (inadvertently) be introducing some delays in processing these requests. Even small delays could be significant when compared to the direct database operations from the chimera client.
Just to be clear, I agree with you. Having a recursive option for more of the chimera
client commands would definitely be of benefit to sites with a large number of directories.
I mentioned the NFS "work-around" as we (the dCache team) have limited resources and are not always able to implement enhancements as quickly as we would like. In the future, should a dCache admin find this issue and the chimera
client hasn't been updated, at least they would know one way to achieve their goal without writing scripts.
I found that suggested NFS ACL update is sufficiently easy and also storage sites that support just grid use-cases (no storage for local users) may not even need to add / change permissions when they would like to support both X.509 and token authorization. Basically this -R
tweak is now a lower priority for me.
Currently only
chown
support recursive operation with-R
parameter. Would it be possible to add same behavior also to thechgrp
,chmod
andsetfacl
?