Kerberos fails with Credential Collection Cache

olifre commented 7 years ago

Using a credential collection, xrootd seems to try to use the first identity it finds (apparently the latest in the collection?) instead of the one matching the domain, leading to an authentication error.

How to reproduce: My /etc/krb5.conf contains:

[libdefaults]
        default_realm = PHYSIK.UNI-BONN.DE
        default_ccache_name = KEYRING:persistent:%{uid}

etc.

Now, I do:

$ kdestroy -A
$ kinit olifre@PHYSIK.UNI-BONN.DE -V
Using default cache: persistent:1000:krb_ccache_6Z2D9yK
Using principal: olifre@PHYSIK.UNI-BONN.DE
Password for olifre@PHYSIK.UNI-BONN.DE: 
Authenticated to Kerberos v5
$ kinit ofreyerm@CERN.CH -V         
Using new cache: persistent:1000:krb_ccache_PDaIPUf
Using principal: ofreyerm@CERN.CH
Password for ofreyerm@CERN.CH: 
Authenticated to Kerberos v5
$ xrdcp some_local.file root://eosuser.cern.ch//eos/user/o/ofreyerm/some_dest_file
zsh: correct 'root://eosuser.cern.ch//eos/user/o/ofreyerm/some_dest_file' to 'root//eosuser.cern.ch//eos/user/o/ofreyerm/some_dest_file' [nyae]? n
[0B/0B][100%][==================================================][0B/s]  
Run: [ERROR] Server responded with an error: [3006] Unable to create file /eos/user/o/ofreyerm/some_dest_file; File exists

Ok, so that works. Now, I do:

$ kdestroy -A
$ kinit ofreyerm@CERN.CH -V
Using default cache: persistent:1000:krb_ccache_PDaIPUf
Using principal: ofreyerm@CERN.CH
Password for ofreyerm@CERN.CH: 
Authenticated to Kerberos v5
$ kinit olifre@PHYSIK.UNI-BONN.DE -V
Using new cache: persistent:1000:krb_ccache_F4NVxI4
Using principal: olifre@PHYSIK.UNI-BONN.DE
Password for olifre@PHYSIK.UNI-BONN.DE: 
Authenticated to Kerberos v5
$ xrdcp some_local.file root://eosuser.cern.ch//eos/user/o/ofreyerm/some_dest_file
zsh: correct 'root://eosuser.cern.ch//eos/user/o/ofreyerm/some_dest_file' to 'root//eosuser.cern.ch//eos/user/o/ofreyerm/some_dest_file' [nyae]? n
[0B/0B][100%][==================================================][0B/s]  
Run: [ERROR] Server responded with an error: [3010] Unable to give access - user access restricted - unauthorized identity used ; Permission denied

So this does not work.

I am using xrootd 4.5.0 here, and mit-krb5 in version 1.14.2.

abh3 commented 6 years ago

Yes, we do not pick out the matching domain ticket should you have more than one such ticket. So, this is an enhancement to teh existing scheme.

olifre commented 6 years ago

Ok, this easily explains it. I am slightly astonished nobody encountered / reported this before me, since many people in physics are working in at least two realms (local University / Institute, CERN), at least if the local domain also uses Kerberos. Is it easy to fix / enhance xrootd to pick out the matching domain ticket?

olifre commented 6 years ago

With RHEL 7 based distros, this is the default:

default_ccache_name = KEYRING:persistent:%{uid}

so the issue should now bite anyone using two Kerberos realms.

abh3 commented 6 years ago

Well, perhaps. Will it not be the case that as long as the 1st ticket is the one we want then it will still work?

olifre commented 6 years ago

Will it not be the case that as long as the 1st ticket is the one we want then it will still work?

Luckily not - it seems to be the case that the ticket fetched last wins the race, at least that's what I observe in my example above, and also testing just now on Ubuntu 18.04 and Gentoo.

This means the most common case "works":

Kerberos ticket from local institution.
Fetch an addtional TGT from CERN.CH and use xrdcp against eosuser.

However, then any xrdcp to your local institution will fail, until you explicitly fetch a new TGT from them. So the nasty workaround would be to fetch a new, matching TGT directly before using xrdcp, which kind of defeats the purpose of Kerberos (to prevent having to enter passwords over and over again), but at least makes things work until the bug is fixed.

olifre commented 6 years ago

Checking the API documentation, it should be sufficient to:

krb5_cccol_cursor_new()
Iterate with krb5_cccol_cursor_next() over all credential caches
krb5_cccol_cursor_free()

However, a quick look at the xrootd codebase reveals the code is rather involved and there is no straightforward place to just "add a loop".

xrootd-dev commented 6 years ago

Hi Oliver,

Sigh, I was hoping you would come up with a patch. Sounds like you’d rather not, right?

Amdy

From: Oliver Freyermuth Sent: Tuesday, June 19, 2018 10:28 AM To: xrootd/xrootd Cc: Subscribed Subject: Re: [xrootd/xrootd] Kerberos fails with Credential Collection Cache (#535)

Checking the API documentation, it should be sufficient to:

a.. krb5_cccol_cursor_new() b.. Iterate with krb5_cccol_cursor_next() over all credential caches c.. krb5_cccol_cursor_free() However, a quick look at the xrootd codebase reveals the code is rather involved and there is no straightforward place to just "add a loop".

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

Use REPLY-ALL to reply to list

To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1

olifre commented 6 years ago

Hi Andy,

yes, a patch was my original intent. I was hoping it would be easy, but after a quick skim over the code, it's not. Just trying all Credential Collections would mean to modify several initialization functions, and rework the logic a bit so a loop over all credential collections can be done with tests of authentication in between.

The more correct way, to check against the domain and from that map the correct realm, would even mean changing the XrdSecInterface if I understand correctly to pass through the domain name / endpoint, and I'd need to learn more about the Kerberos API to actually query the realm <=> domain mapping.

For all of this, I'd need a significant time just to understand the program flow and the corner cases (there are special cases when variables in principal names are evaluated etc). While I would certainly be interested, I don't have the resources (time) at the moment :sad:.

Cheers, Oliver

xrootd-dev commented 6 years ago

Actually, you get the domain name via the NetAddrInfo object that part of he Entity structure passed to you. However, there is some noise out there saying that it's based on DNS and one should no longer trust DNS. So, you pick your fights where they are. Anyway, if you change your mind let me know! I might be able to get to this in a few months.

Andy

On Tue, 19 Jun 2018, Oliver Freyermuth wrote:

Hi Andy,

yes, a patch was my original intent. I was hoping it would be easy, but after a quick skim over the code, it's not. Just trying all Credential Collections would mean to modify several initialization functions, and rework the logic a bit so a loop over all credential collections can be done with tests of authentication in between.

The more correct way, to check against the domain and from that map the correct realm, would even mean changing the XrdSecInterface if I understand correctly to pass through the domain name / endpoint, and I'd need to learn more about the Kerberos API to actually query the realm <=> domain mapping.

For all of this, I'd need a significant time just to understand the program flow and the corner cases (there are special cases when variables in principal names are evaluated etc). While I would certainly be interested, I don't have the resources (time) at the moment :sad:.

Cheers, Oliver

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/xrootd/xrootd/issues/535#issuecomment-398556172

######################################################################## Use REPLY-ALL to reply to list

To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1

xrootd / xrootd

Kerberos fails with Credential Collection Cache #535