Documentation of gplazma.oidc.provider plugin is insufficient

[XMol]

Hello dCache.org,

our WLCG customers want to use tokens relatively soon and for that several features of the OIDC providers need to be included in the dCache configuration. However, which settings those are we cannot find in the documentation shipped with dCache, just in other outlets like the CERN Twiki, GitHub notes, workshop presentations or plan word of mouth. Even in the current latest commit of the master branch - ad191846 at the point in time of writing this issue - gplazma.properties still only exposes -suppress. As a matter of fact, no functional configuration can be done without -profile and -prefix. -auth-id is another one that enables neat use-cases. And as Tigran foreshadowed in today's T1 support conference, upcoming releases will have yet another option that will influence whether the wlcg.group and (storage) scope claims should be combined or not. In summary, I believe the documentation specifically on the gplazma.oidc.provider property needs to be extended/updated.

As a side story, this week I had set up OIDC for a new VO at our site and at first I omitted the -prefix for their IDP. The consequence was, that gPlazma broke! In the log I found...

failed to create oidc: Bad OIDC provider gplazma.oidc.provider!KIT-ETP-DARWIN: Missing 'prefix' argument.

Even though gPlazma was running, no successful authentication was possible anymore, even without OIDC. Thus the -prefix argument at some point became mandatory, which strikes me as odd - / as a default value is safe, no?

Best regards,
Xavier.

/cc @samuambroj

[paulmillar]

Hi Xavier,

Did you have a chance to review the oidc documentation that was recently added?

This documentation is currently only available on the dCache v9.2 version of The Book, in the gPlazma chapter, under OIDC section.

(the documentation probably should be back-ported, so it appears under The Book for older versions of dCache.)

It would be great if you could check the documentation, to see if anything is badly explained (or just wrong!).

For your specific side-question: no / isn't a safe default and a value must be supplied.

The wlcg and scitokens profile includes support for explicit authorisation; for example, the token can say "the bearer of this token is allowed to upload data into directory /foo". In dCache, this is honoured by switching off the namespace security checks if the token has such explicit authorisation statements: if the token says the bearer can upload data (or delete data, or ...) then dCache honours that authorisation decision, irrespective of what the namespace says.

Currently, under wlcg and scitokens, an issuer/OP corresponds to a VO. The prefix option is (in essence) the subtree within dCache namespace where you are delegating authorisation decisions to that VO. For example, if the token could say "the bearer is authorised to delete all data" then the prefix limits this decision to all data underneath some VO-specific path (e.g., /pnfs/kit.edu/atlas).

Specifying / as the prefix would allow the VO to do anything on any file/directory on the storage. That might be OK, if the dCache instance is intended for a single VO, but I think it wouldn't be acceptable for dCache instances shared between multiple VOs. Therefore, I don't think / is a safe default value.

Hopefully the docs (in the above link) describes this better.

Cheers, Paul.

[XMol]

Hi Paul,

this update of The Book is already very good and much appreciated! 👏 I could not find anything wrong with it (aside from minor typos).

The documentation also relativizes the necessity of the -prefix argument: Only the wlcg and scitoken profiles support it and thus only for them it is mandatory.

Though, now you have the problem that the properties file is not as informative as The Book. The details on how to configure gPlazma in general - especially when it comes down to examples! - ought to be found in The Book, in my humble opinion. However, the properties should be described in the properties files. If I had any say in this, I would put all from the "General configuration" section into the gplazma.properties file (except for the paragraph on the audience claim, which is not a property).

One more question:

Specifying / as the prefix would allow the VO to do anything on any file/directory on the storage.

Does this mean, the user could access any part of the dCache namespace, even outside its root/home?
The justification for omitting -prefix in our setup was, that the user we're mapping the token to has a path (unequal to /) set as root (through storage-authzdb). My assumption is, that the behavior should be the same as when the identical path was set as -prefix for the OP. That is, as long as tokens are used. With our approach the behavior should be identical from the perspective of the user with other authentication methods, too, right? I really hope that -prefix=/ does not overrule the user's root from storage-authzdb. 😨

Best regards,
Xavier.

[XMol]

I really hope that -prefix=/ does not overrule the user's root from storage-authzdb. 😨

Reading the updated passages from The Book, it seems clear that the -prefix setting on the OIDC provider does not alter/overrule the home or root directory of the user.

It is an absolute path within dCache’s namespace under which authorisation paths are resolved. For example, if an OP is configured with -prefix=/wlcg/CMS and the token allows reading under the /users/paul path (storage.read:/home/paul) then the token is allowed to read content under the /wlcg/CMS/home/paul path.

I deduce here, that the token does not change the home or root directory of the client. In other words, /wlcg/CMS is not considered the home or root of paul. And neither is /home/paul. As such, paul must still prepend the entire LFN for access to any file (/wlcg/CMS/home/paul/...). Consequently each directory of that must be visible and accessible to paul, too.

If there was a home and/or root directory for paul, then that is not represented in this example. By which I mean, it could be that there are more super directories to /wlcg/CMS, which are not disclosed because of home/root directory settings, right?

[paulmillar]

Sorry, this isn't terribly clear in the documentation.

The -prefix argument is an absolute path in dCache's namespace. The value is interpreted independently of any user-specific path (the user-specific root path or user's home directory).

One way of looking at this is that the user's root path and the door root path are paths that the door uses to figure out what is the absolute path in the dCache namespace. (e.g, user requests /reference-data.root but the user-root path is /data/atlas then the request is actually targeting /data/atlas//reference-data.root). There is a similar process if the door has a door-root path configured: the path from a user's request would be modified to take into account the door's root path.

In contrast, the oidc plugin takes the -prefix path and the token's AuthZ statements to build a list of allowed activities that PnfsManager (not the door) enforces. This list of allowed activities also has absolute paths in it, but these are independent from how the door understands (or modifies) the user's request path.

[XMol]

So I was right, if I understand you correctly. Which means -prefix needs to be set to at least the user's root directory (enforced by the doors). Otherwise the settings are incompatible.

All in all, the best option for us seems to be, to never use user specific root or home directories. And in a dCache instance that is shared by multiple VOs a root enforced by the door is problematic, too. The -prefix merely saves characters for the bearer token's storage scope claims. The clients still need to know and apply the prefix themselves in the sURLs.

[paulmillar]

So I was right, if I understand you correctly. Which means -prefix needs to be set to at least the user's root directory (enforced by the doors). Otherwise the settings are incompatible.

Certainly, setting the user's (or door's) root and the OP's -prefix to the same absolute path results in a relatively straight-forward deployment, one that is easy for users to understand.

However, I think saying the settings are otherwise "incompatible" is overly harsh: I can imagine scenarios where it would make sense for these to be different.

The -prefix merely saves characters for the bearer token's storage scope claims

No, it fundamentally limits what a VO can do.

I'm not sure how much this is already obvious, but one way to look at this is from "damage limitation" point-of-view. Imagine a scenario where a VO (let's say ATLAS) issued a token to a client that authorised deletion of all data and that client "went mad" (sorry, suffered a software bug) resulting in it trying to actually delete all the files it could find (starting from what it considers the filesystem's root). This would be very bad for ATLAS, but the resulting damage should be limited to only ATLAS data. This misbehaving client should not be able to delete any files from some other VO.

It would be possible to implement this separation by using a user-specific root (or, perhaps, using a door specific root). But would you be 100% sure that is correctly configured; that it would work in all circumstances? In any case, there are dCache instances that do not use user-specific roots for VOs and modifying their behaviour would require updating all files in the catalogue (as the URLs would change).

Put another way, the way -prefix is meant to provide a very simple way to understand how token-based authorisation is supported. Simple here is good: there is a security-based maxim is that "complicity is the enemy of security." Including the user-root would make it much harder to understand what is authorised.

The clients still need to know and apply the prefix themselves in the sURLs.

Yes, that is certainly true, unless dCache is configured to have a user-specific root for all members of a VO, or a VO-specific door with a corresponding door-specific root.

HTH, Paul.

[XMol]

Hi Paul,

thanks for your clarifications. Then I think I've understood how -prefix is supposed to work. It does the same as a user specific root directory setting would in authorization context only. And at least for GridKa, we don't have any user specific root settings yet. Introducing them now for VOs with existing data requires that all LFNs - for dCache as well as the VO's catalog - need to be updated. -prefix here is the solution that is compatible to existing data and workflows.

Certainly, setting the user's (or door's) root and the OP's -prefix to the same absolute path results in a relatively straight-forward deployment, one that is easy for users to understand.

I tend to disagree. Unless I still don't get how user-specific root and home settings combine with -prefix, that is.
If for a user the root directory was set to the same value as -prefix, then none of the storage claims from the token can match, unless the namespace looks like $root/$prefix, which would be identical to $root/$root. Or rather, maybe I've misunderstood the meaning of root directory? Because, I cannot detect any functional difference in setting it (in our test dCache SE) or leaving it on /. And -prefix is relevant for PnfsManager exclusively, which always works with the fully qualified path of a file, irrespective of root or home directory.
The only setting that I have even less experience with is the door-specific root directory. Because all our dCache SEs have to support at least two different VOs, we can never deploy that (unless we multiply the doors, too, and make them VO specific). So if a door did have a non-/-root, is that prepended to the tURL of the client? That way PnfsManager would get the fully qualified path and the client wouldn't need to know that root exists, since it is transparent to them.

Ciao,
Xavier.

[XMol]

In order to avoid deviating this issue into a different discussion, I've started a new RT ticket—#10529—for our problems regarding the user specific root and home directories. So from here on we can return to the original complaint: suboptimal documentation of dCache's OIDC properties and features.

[kofemann]

Hi Xavier,

I think the main confusion comes from how you expect tokens to work. Unfortunately, this is a moving target, as WLCG time-to-time changes the semantics. However, the current behaviour for WLCG tokens is (-profile=wclg):

We support two types of tokens: Authorization tokens and NonAutheorization tokens. All tokens with any of scopes storage.*:/<path> OR compute.* are Authorization tokens. For such tokens, any file system checks are ignored. The prefix+<path> is used to construct the file path within the namespace, e.g, -prefix=/atlas and storage.read:/data123 gives read access to the /atlas/data123 directory and its subdirectories. User roots should be ignored. This gives experiments central control access to files without a need to know how the namespace is organized (ideally). The doo's root is an attempt to achieve the same unification, without tokents. The -authz-id section, allows to specify principals that are added during the authentication step, mainly uid and gid that should be used for newly created files, as there no user mapping should happen.

Profile scitokens shouldn't be used and will be removed soon. Yeah, the wording in book/property files might be not optimal. But we have an excuse (read above) - all this moving target.

[XMol]

Hi Tigran,

are user specific root and home directories ignored with tokens only? According to my testing, they are always ignored (hence RT #10529)...

And I understand that your biggest customer community has no clear idea of what they want yet, which renders it very difficult to implement all this stuff coherently. But I get more and more the impression that sites will have to set up dedicated door and gPlazma services for access by either tokens or the traditional x509 certificates.

This [-prefix] gives experiments central control access to files without a need to know how the namespace is organized .(ideally).

Well, according to our experience, that is only half true. With our new DARWIN VO, that uses tokens for authentication exclusively (i.e. no complications to support x509 in additional or from tradition), we had set the prefix /pnfs/gridka.de/darwin on their provider. Yet, they still uploaded data to roots://dcachexrootd-kit.gridka.de/pnfs/gridka.de/darwin/tescht.txt, which produced /pnfs/gridka.de/darwin/tescht.txt (rather then /pnfs/gridka.de/darwin/pnfs/gridka.de/darwin/tescht.txt). This supports Paul's wording, in that, the -prefix is used to build the full path to allow access for, not to actually access. With your example, the client still explicitly needs to include /atlas/data123 in the LFN, contradicting the "experiments don't need to know" bit.

Ciao,
Xavier.

[kofemann]

This is not contradicting. NOTE -prefix is used ONLY in combination with the storage.*:/path scope. If the scope is missing, then the non-Authorization token is used and the rules are different.

[XMol]

In your example storage.read/data123 is given with -prefix=/atlas. Thus, clients still need to access /atlas/data123 explicitly and not just /data123, right?

EDIT: I redact my question, since Tigran already made the answer clear, but I failed to comprehend it at the time.

[XMol]

Since The Book is updated on this subject with dCache 9.2, I guess there is no need for this issue anymore.