search

dCache / dcache

dCache - a system for storing and retrieving huge amounts of data, distributed among a large number of heterogenous server nodes, under a single virtual filesystem tree with a variety of standard access methods
https://dcache.org
277 stars 132 forks source link

webdav door reports a login error when trying to request a macaroon #7533

Open amisk opened 3 months ago

amisk commented 3 months ago

Hello,

I'm out of ideas about this one. Maybe it's a bug, maybe I'm doing something wrong, but here we go. We have a webdav door which I want to use to request macaroons. The layout of it is:


[webdavDomain2893]
[webdavDomain2893/webdav]
webdav.authn.hostcert.cert=/etc/grid-security/hostcert.pem
webdav.authn.hostcert.key=/etc/grid-security/hostkey.pem
webdav.authn.hostcert.cacert=/etc/grid-security/certificates
webdav.redirect.on-read=true
webdav.redirect.on-write=true
#webdav.authn.basic=true
webdav.authn.protocol=https
webdav.authn.accept-client-cert=true
webdav.authn.require-client-cert=false
webdav.net.port=2893
webdav.authz.readonly = false
webdav.enable.macaroons = true
webdav.macaroons.max-lifetime=1
webdav.macaroons.max-lifetime.unit=DAYS
webdav.authz.anonymous-listing = false
webdav.enable.overwrite=false
webdav.authz.allowed-paths = /pnfs/fz-juelich.de/data/utr2/

in /etc/grid-security/grid-mapfile and grid-vorolemap I have the line

"/C=DE/O=GridGermany/OU=Forschungszentrum Juelich GmbH/CN=Arpad Miskolczi" lofar001 and "/C=DE/O=GridGermany/OU=Forschungszentrum Juelich GmbH/CN=Arpad Miskolczi" "/lofar/ops" lofar001

When I try to request a macaroon using curl, I used: curl -vvv -k --cert ./grid_cert.pem --key ./grid_key.pem -X POST -H 'Content-Type: application/macaroon-request' -d '{"caveats": ["activity:LIST","path:/pnfs/fz-juelich.de/data/utr2/"]}' https://dcache-lofar.fz-juelich.de:2893 where grid_cert/key.pem is the certificate and key that corresponds to the line in the files. I also tried adding --capath /etc/grid-security/certificates/ to the command, but it has no effect.

After the request, I see in the log: level=WARN ts=2024-03-15T13:12:42.358+0100 event=org.dcache.webdav.request request.method=POST request.url=https://dcache-lofar.fz-juelich.de:2893/ response.code=401 response.reason="login failed" socket.remote=134.94.0.52:45536 user-agent=curl/7.29.0 user.dn="CN=Arpad Miskolczi,OU=Forschungszentrum Juelich GmbH,O=GridGermany,C=DE" duration=0

Now what I noticed is that the order of the dn entries is different and the fields are delimited by a comma instead of a dash. I don't know if that is important. But in case it is not, I also put in this version of the dn in to the above mentioned files, exactly how the log says.

Then I used "explain login" from the gPlazma domain:

[dcache-lofar] (gPlazma@gPlazma-dcache-lofarDomain) admin > explain login "dn:CN=Arpad Miskolczi,OU=Forschungszentrum Juelich GmbH,O=GridGermany,C=DE" fqan:/lofar/ops
LOGIN OK
 |    in: CN=Arpad Miskolczi,OU=Forschungszentrum Juelich GmbH,O=GridGermany,C=DE
 |        FQANPrincipal[/lofar/ops,primary]
 |   out: CN=Arpad Miskolczi,OU=Forschungszentrum Juelich GmbH,O=GridGermany,C=DE
 |        GroupNamePrincipal[lofar001,primary]
 |        UserNamePrincipal[lofar001]
 |        UidPrincipal[22004]
 |        FQANPrincipal[/lofar/ops,primary]
 |        GidPrincipal[1004,primary]
 |
 +--AUTH OK
 |   |
 |   +--x509 OPTIONAL:FAIL (no X.509 certificate chain) => OK
 |   |
 |   +--voms OPTIONAL:FAIL (no X509 certificate chain) => OK
 |   |
 |   +--kpwd OPTIONAL:FAIL (no username and password) => OK
 |
 +--MAP OK
 |   |    added: GroupNamePrincipal[lofar001,primary]
 |   |           UserNamePrincipal[lofar001]
 |   |           UidPrincipal[22004]
 |   |           GidPrincipal[1004,primary]
 |   |
 |   +--vorolemap OPTIONAL:OK => OK
 |   |      added: GroupNamePrincipal[lofar001,primary]
 |   |
 |   +--authzdb SUFFICIENT:OK => OK (ends the phase)
 |          added: UserNamePrincipal[lofar001]
 |                 UidPrincipal[22004]
 |                 GidPrincipal[1004,primary]
 |
 +--ACCOUNT OK
 |
 +--SESSION OK
 |   |
 |   +--roles OPTIONAL:OK => OK
 |   |
 |   +--authzdb SUFFICIENT:OK => OK (ends the phase)
 |
 +--VALIDATION OK

So the login using this dn is working according to gPlazma. But yet again, requesting the macaroon results in an login failed error. Even if the group or user id, or a path would be wrong, I would expect a permission denied instead of a login failed. If some firewall issue would be present, I would expect a connection error. Though I'm trying this from the same server where the door is on.

If I enable basic authorization and use a username and password it works flawlessly. We use dcache 7.2.12

I did try to set the logging verbosity to DEBUG, but then it's so many lines that I can't make heads or tails of it.

Is this some bug? Am I missing something? Do you need more information from other files?

Thanks!

Cheers, Arpad

paulmillar commented 3 months ago

Hi Arpad,

dCache will provide a detailed explanation of a login failure in the log file (of the domain hosting gPlazma) the first time gPlazma sees a failed login attempt. Subsequent failed login attempts (with the same credentials and door-supplied principals) are not logged.

My guess is that there was a login attempt (possible from a macaroon request) that failed some point in the past.
You could either search through the gPlazma log file for the detailed report, or get gPlazma to create a new detailed report. For the former, you will probably find the earlier report by searching for your DN; however, perhaps the most reliable way is to get gPlazma to provide a new detailed report.

gPlazma will reset the login failures (and so, provide a detailed log report on failures) when it reloads its configuration. The easiest way to do that is to update the last-modified time of the gPlazma configuration file (e.g., touch /etc/dcache/gplazma.conf). This tells gPlazma to reload its configuration. gPlazma will then log a detailed report for subsequent failed logins.

So, touch the gplazma.conf file, resubmit your macaroon request and look at the log file (for the domain hosting the gPlazma service) to see what went wrong.

HTH, Paul.

amisk commented 3 months ago

Hi Paul,

Thanks. But I still only see that login failed, I see which parts of the authorization worked, and which parts didn't. But I still don't see why it didn't. Like, the CN is in grid-mapfile, in the vorolemap file, it has a username attached, but I don't get the username from gPlazma. The best I got so far was that I got a UserNamePrincipal, but still not a uid or gid, and I can't see why not, since all necessary information should be in the files.

paulmillar commented 3 months ago

Hi Arpad,

Could you copy (into this issue) the login failure report, as recorded in the domain log file (from the domain hosting gPlazma)?

It should look like the explain login admin command output you posted earlier.

Cheers, Paul.

amisk commented 3 months ago 
03 Apr. 2024 10:10:30 (gPlazma) [WebDAV-dcache-lofar Login] Login attempt failed; detailed explanation follows:
LOGIN FAIL
 |    in: Origin[134.94.0.52]
 |        X509 Certificate chain:
 |          |
 |          +--CN=Arpad Miskolczi,OU=Forschungszentrum Juelich GmbH,O=GridGermany,C=DE [12572464135320552966040670602]
 |               |
 |               +--Issuer: CN=DFN-Verein PCA Grid - G01,OU=DFN-PKI,O=DFN-Verein,C=DE
 |               +--Validity: OK for 91 days, 23 hours, 33 minutes and 35,7 seconds
 |               +--Algorithm: SHA-256 with RSA
 |               +--Public key: RSA 4096 bits
 |               +--Subject alternative names: email: a.miskolczi@fz-juelich.de
 |               +--Key usage: digital signature, key encipherment, SSL client, email protection
 |        
 |   out: EmailAddressPrincipal[a.miskolczi@fz-juelich.de]
 |        /C=DE/O=GridGermany/OU=Forschungszentrum Juelich GmbH/CN=Arpad Miskolczi
 |        Origin[134.94.0.52]
 |        LoAPrincipal[IGTF-AP:Classic]
 |        LoAPrincipal[REFEDS:IAP:low]
 |        LoAPrincipal[REFEDS:IAP:medium]
 |        EntityDefinitionPrincipal[Person]
 |        LoAPrincipal[IGTF:CEDAR]
 |
 +--AUTH OK
 |   |    added: EmailAddressPrincipal[a.miskolczi@fz-juelich.de]
 |   |           /C=DE/O=GridGermany/OU=Forschungszentrum Juelich GmbH/CN=Arpad Miskolczi
 |   |           LoAPrincipal[IGTF-AP:Classic]
 |   |           LoAPrincipal[REFEDS:IAP:low]
 |   |           LoAPrincipal[REFEDS:IAP:medium]
 |   |           EntityDefinitionPrincipal[Person]
 |   |           LoAPrincipal[IGTF:CEDAR]
 |   |
 |   +--x509 OPTIONAL:OK => OK
 |   |      added: EmailAddressPrincipal[a.miskolczi@fz-juelich.de]
 |   |             /C=DE/O=GridGermany/OU=Forschungszentrum Juelich GmbH/CN=Arpad Miskolczi
 |   |             LoAPrincipal[IGTF-AP:Classic]
 |   |             LoAPrincipal[REFEDS:IAP:low]
 |   |             LoAPrincipal[REFEDS:IAP:medium]
 |   |             EntityDefinitionPrincipal[Person]
 |   |             LoAPrincipal[IGTF:CEDAR]
 |   |
 |   +--voms OPTIONAL:FAIL (no FQANs) => OK
 |   |
 |   +--kpwd OPTIONAL:FAIL (no username and password) => OK
 |
 +--MAP OK
 |   |
 |   +--vorolemap OPTIONAL:FAIL (no record) => OK
 |   |
 |   +--authzdb SUFFICIENT:FAIL (no mappable principal) => OK
 |   |
 |   +--kpwd SUFFICIENT:FAIL (no login name) => OK
 |
 +--ACCOUNT OK
 |
 +--SESSION OK
 |   |
 |   +--roles OPTIONAL:OK => OK
 |   |
 |   +--authzdb SUFFICIENT:FAIL (no username principal) => OK
 |   |
 |   +--kpwd SUFFICIENT:FAIL (no record found) => OK
 |
 +--VALIDATION FAIL (no username, no UID, no primary GID)

Here there is no username given because "map optional gridmap" is deactivated. It seems like we don't staging requests when it's active. But even with it being active, I would only get the username, but still no UID or GID.

amisk commented 3 months ago

I activated it to produce the log and then deactivated it again:

03 Apr. 2024 10:19:37 (gPlazma) [WebDAV-dcache-lofar Login] Login attempt failed; detailed explanation follows:
LOGIN FAIL
 |    in: Origin[134.94.0.52]
 |        X509 Certificate chain:
 |          |
 |          +--CN=Arpad Miskolczi,OU=Forschungszentrum Juelich GmbH,O=GridGermany,C=DE [12572464135320552966040670602]
 |               |
 |               +--Issuer: CN=DFN-Verein PCA Grid - G01,OU=DFN-PKI,O=DFN-Verein,C=DE
 |               +--Validity: OK for 91 days, 23 hours, 24 minutes and 28,1 seconds
 |               +--Algorithm: SHA-256 with RSA
 |               +--Public key: RSA 4096 bits
 |               +--Subject alternative names: email: a.miskolczi@fz-juelich.de
 |               +--Key usage: digital signature, key encipherment, SSL client, email protection
 |        
 |   out: LoAPrincipal[REFEDS:IAP:low]
 |        EmailAddressPrincipal[a.miskolczi@fz-juelich.de]
 |        EntityDefinitionPrincipal[Person]
 |        UserNamePrincipal[lofar001]
 |        LoAPrincipal[IGTF:CEDAR]
 |        /C=DE/O=GridGermany/OU=Forschungszentrum Juelich GmbH/CN=Arpad Miskolczi
 |        Origin[134.94.0.52]
 |        LoAPrincipal[IGTF-AP:Classic]
 |        LoAPrincipal[REFEDS:IAP:medium]
 |
 +--AUTH OK
 |   |    added: LoAPrincipal[REFEDS:IAP:low]
 |   |           EmailAddressPrincipal[a.miskolczi@fz-juelich.de]
 |   |           EntityDefinitionPrincipal[Person]
 |   |           LoAPrincipal[IGTF:CEDAR]
 |   |           /C=DE/O=GridGermany/OU=Forschungszentrum Juelich GmbH/CN=Arpad Miskolczi
 |   |           LoAPrincipal[IGTF-AP:Classic]
 |   |           LoAPrincipal[REFEDS:IAP:medium]
 |   |
 |   +--x509 OPTIONAL:OK => OK
 |   |      added: LoAPrincipal[REFEDS:IAP:low]
 |   |             EmailAddressPrincipal[a.miskolczi@fz-juelich.de]
 |   |             EntityDefinitionPrincipal[Person]
 |   |             LoAPrincipal[IGTF:CEDAR]
 |   |             /C=DE/O=GridGermany/OU=Forschungszentrum Juelich GmbH/CN=Arpad Miskolczi
 |   |             LoAPrincipal[IGTF-AP:Classic]
 |   |             LoAPrincipal[REFEDS:IAP:medium]
 |   |
 |   +--voms OPTIONAL:FAIL (no FQANs) => OK
 |   |
 |   +--kpwd OPTIONAL:FAIL (no username and password) => OK
 |
 +--MAP OK
 |   |    added: UserNamePrincipal[lofar001]
 |   |
 |   +--vorolemap OPTIONAL:FAIL (no record) => OK
 |   |
 |   +--authzdb SUFFICIENT:FAIL (no mappable principal) => OK
 |   |
 |   +--gridmap OPTIONAL:OK => OK
 |   |      added: UserNamePrincipal[lofar001]
 |   |
 |   +--kpwd SUFFICIENT:FAIL (UserNamePrincipal and X509 principals found) => OK
 |
 +--ACCOUNT OK
 |
 +--SESSION OK
 |   |
 |   +--roles OPTIONAL:OK => OK
 |   |
 |   +--authzdb SUFFICIENT:OK => OK (ends the phase)
 |
 +--VALIDATION FAIL (no UID, no primary GID)
paulmillar commented 3 months ago

In your explain login, you asserted that you're a member of /lofar/ops group:

explain login "dn:CN=[...],C=DE" fqan:/lofar/ops

However, for the Macaroon request, you are presenting your EEC. There's no VOMS proxy, so the voms plugin didn't add any FQAN principals.

This should explain the discrepancy between explain login output and what you're seeing in real life (with the Macaroon requests).

Removing the FQAN principal from the explain login command should replicate the problem.

If you want to support EEC-based AuthN then the problem you're facing is from the gridmap plugin adding a username. The kpwd plugin requires exactly one mapping principal: either a Login name, an X.509 DN, a Kerberos principal, a username principal or a gid principal.

So, you should be able to configure the kpwd plugin to support X.509 DN and omit the gridmap plugin.

Alternatively, don't use kpwd plugin, but use something else to map the username principal to uid and gid. The authzdb plugin should work.

HTH, Paul.