search

dCache / dcache

dCache - a system for storing and retrieving huge amounts of data, distributed among a large number of heterogenous server nodes, under a single virtual filesystem tree with a variety of standard access methods
https://dcache.org
282 stars 135 forks source link

Missing recursive chimera operations #6844

Open vokac opened 1 year ago

vokac commented 1 year ago

Currently only chown support recursive operation with -R parameter. Would it be possible to add same behavior also to the chgrp, chmod and setfacl?

paulmillar commented 1 year ago

Not to detract from your request, but some notes.

The chown Chimera command accepts a two-part argument: the (user-)owner and the group-owner, separated by a colon. Unless you want to change the group-ownership of files with different user-owners, the chown Chimera command should work for your chgrp use-case.

For all your desired changes, one option is to NFS-mount dCache and make changes using regular Linux admin tools (see chown(1), chgrp(1), chmod(1) and nfs4_setfacl(1)). These all support recursive operations (-R). Note that you need to use nfs4_setfacl (and not setfacl(1)) for "reasons".

vokac commented 1 year ago

Tested NFS ACL update on ~ 6M directories and it is 3.6x slower (~ 34 hours) than updating individual directories with chimera (~ 9 hours). Our typical storage have roughly 30M directories so even NFS ACL update should take less than a week, but it would still be nice to have more efficient recursive chimera option.

paulmillar commented 1 year ago

Thanks for benchmarking the different options, @vokac.

It's perhaps not a surprise that NFS introduces "some" overhead. NFS is quite a "chatty" protocol. I would imagine that the overhead will likely depend on the network RTT between client and server, and also how much effort the client makes to optimised the NFS requests.

I don't know if the NFS approach being over three times slower is expected. It could be that dCache's NFS interface could (inadvertently) be introducing some delays in processing these requests. Even small delays could be significant when compared to the direct database operations from the chimera client.

Just to be clear, I agree with you. Having a recursive option for more of the chimera client commands would definitely be of benefit to sites with a large number of directories.

I mentioned the NFS "work-around" as we (the dCache team) have limited resources and are not always able to implement enhancements as quickly as we would like. In the future, should a dCache admin find this issue and the chimera client hasn't been updated, at least they would know one way to achieve their goal without writing scripts.

vokac commented 1 year ago

I found that suggested NFS ACL update is sufficiently easy and also storage sites that support just grid use-cases (no storage for local users) may not even need to add / change permissions when they would like to support both X.509 and token authorization. Basically this -R tweak is now a lower priority for me.