systemd / systemd

The systemd System and Service Manager
https://systemd.io
GNU General Public License v2.0
12.9k stars 3.7k forks source link

systemd breaks user directory on AFS #7261

Closed rzlourenco closed 6 years ago

rzlourenco commented 6 years ago

Submission type

systemd version the issue has been seen with

$ systemctl --version
systemd 234
+PAM -AUDIT +SELINUX -IMA +APPARMOR -SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID -ELFUTILS +KMOD -IDN2 -IDN default-hierarchy=hybrid
$ uname -a
Linux dred 4.12.8-1-default #1 SMP PREEMPT Thu Aug 17 05:30:12 UTC 2017 (4d7933a) x86_64 x86_64 x86_64 GNU/Linux

Used distribution

openSUSE Tumbleweed.

In case of bug report: Expected behaviour you didn't see

I should be able to login into GNOME with GDM. I should be able to use systemctl --user.

In case of bug report: Unexpected behaviour you saw

After login I'm brought back to GDM. Here's a snippet of journalctl when I login in the console. A few things of note, when I login in the console:

$ env | grep XDG
XDG_DATA_DIRS=/afs/l2f/home/rzl/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share
XDG_CONFIG_DIRS=/etc/xdg
$ env | grep DBUS
1 $ tree /run/user/`id -u`
/run/user/50421
├── krb5cc_CCMdRW [error opening dir]
└── krb5cc_d2OGVV
    ├── primary
    └── tktOaPFiW
$ systemctl --user status
Failed to read server status: Process org.freedesktop.systemd1 exited with status 1

My PAM config:

$ cat /etc/pam.d/systemd-user
# This file is part of systemd.
#
# Used by systemd --user instances.
account  include common-account
session  required pam_selinux.so close
session  required pam_selinux.so nottys open
session  include common-session
$ cat /etc/pam.d/common-session
# [...]
session optional        pam_systemd.so  debug
session required        pam_limits.so
session required        pam_unix.so     try_first_pass
session optional        pam_krb5.so     minimum_uid=500
session optional        pam_umask.so
session optional        pam_gnome_keyring.so    auto_start only_if=gdm,gdm-password,lxdm,lightdm
session optional        pam_env.so

In case of bug report: Steps to reproduce the problem

Broadly:

  1. Install Tumbleweed with GNOME.
  2. Install OpenAFS. You can get it from the filesystems repo, though you'll need an older kernel since the package doesn't build since 4.12.8.
  3. Get users from LDAP.
  4. Setup Kerberos authentication.
  5. Reboot.
  6. Try to login.

This used to work. More detailed instructions for Fedora, but it's basically the same. It seems Debian has stumbled into this problem sometime ago.

If I understood correctly, the problem is that systemd changed behavior and started forking a "user manager" daemon from PID 1, off of which things like D-Bus are launched. But since it does not pass through PAM, it gets no AFS token, so the user directory is effectively inaccessible.

My temporary workaround is to login in the console and then in GDM. This is a non-solution because it requires me to leave an open console with my user... It works somewhat, but GNOME crashes often, from D-Bus-related problems, it seems. I don't think it's a problem with GNOME because I have a box at home with the same GNOME, but no AFS and it is very stable.

What can I do to help? Is there a way to not launch a user manager outside my PAM session?

poettering commented 6 years ago

But since it does not pass through PAM, it gets no AFS token, so the user directory is effectively inaccessible.

it does pass through PAM. The "systemd-user" PAM fragment (which you even pasted above) is the one that is used for the PAM invocation for the "systemd --user" instance.

I really don't know AFS, not sure we can help you much on this. It appears this is something that the OpenSUSE folks should be contacted about.

Looking at your log output it appears to me that ther's simply something wrong with your PAM setup, and one of the hooks you have in there is failing. Which one isn't that obvious...

What it is interesting though: current kerberos uses the kernel keyring (at least on fedora), instead of a directory in /run to store its tickets. That solves some ordering issues, as pam_systemd will make the user dir in /run available, but the kerberos code used to want to write there even before pam_system ran. Maybe that's the same issue?

Eitherway, please bring this to the attention of your downtsream distro folks. From upstream we have no experience with kerberos, openafs or SUSE, and can't help you much.

Is there a way to not launch a user manager outside my PAM session?

Well, as mentioned, the user manager runs in its own PAM session, so this should not be an issue. The user manager is shared by all your login sessions (i.e. all PAM session of your user, with the one exception of the PAM session of the user manager itself), as a singleton, lasting from your first (concurrent) session until you log out entirely again. And because things are designed that way it can't really live in the original PAM session, as it would then be lifecycled with it, and that's undesired...

poettering commented 6 years ago

Anyway, I think we should close this here. Please talk to your distro downstream folks. Feel free to reopen if they indicate that this is really an upstream systemd issue. Other than that, such integration issues are really what distros are for, and which they need to make sure work correctly.

Closing hence. I hope that makes some sense?

grawity commented 6 years ago

What it is interesting though: current kerberos uses the kernel keyring (at least on fedora), instead of a directory in /run to store its tickets.

As far as I know (which might be outdated),

  1. The default Kerberos configuration uses the session keyring, not the user keyring. (This might have changed with the advent of "user persistent" keyring type, hopefully.)
  2. OpenAFS assigns credentials to "PAGs", which are also a kind of session – inheritable, but not uid-wide. I'm not sure if an existing PAG can be joined or not.
  3. From my experience, this is even the same for cifscreds/cifs.ko, which insists on storing CIFS/SMB passwords in the session keyring, and won't use the user keyring no matter what.

So in both cases the problem comes from two separate PAM stacks, and the inability for one process to "join" another process tree and obtain its credentials. (And of course the filesystems being so insistent on session-type credentials...)

I don't think it is something a distro can fix by shuffling PAM modules around.

rzlourenco commented 6 years ago

But since it does not pass through PAM, it gets no AFS token, so the user directory is effectively inaccessible.

it does pass through PAM. The "systemd-user" PAM fragment (which you even pasted above) is the one that is used for the PAM invocation for the "systemd --user" instance.

I'm sorry, I wrote something that makes no sense. It does go through PAM. What I mean was that since systemd --user doesn't go through the authentication part of Kerberos, it gets no AFS tokens. I don't understand the exact mechanism, but AFS tokens are stored in "Process Authentication Groups", and these are set by session, not by user. When systemd user daemons changed from being per session to per user, this broke existing systems.

Looking at your log output it appears to me that ther's simply something wrong with your PAM setup, and one of the hooks you have in there is failing. Which one isn't that obvious...

pam_systemd is failing:

Nov 06 10:07:22 dred login[1884]: pam_systemd(login:session): pam-systemd initializing
Nov 06 10:07:22 dred login[1884]: pam_systemd(login:session): Asking logind to create session: uid=50421 pid=1884 service=login type=tty class=user desktop= seat= vtnr=0 tty=tty1 display= remote=no remote_user= remote_host=
Nov 06 10:07:22 dred systemd[1]: user@50421.service: Trying to enqueue job user@50421.service/start/replace
Nov 06 10:07:22 dred systemd[1]: user@50421.service: Installed new job user@50421.service/start as 1975
Nov 06 10:07:22 dred systemd[1]: user@50421.service: Enqueued job user@50421.service/start as 1975
Nov 06 10:07:22 dred systemd[1]: user@50421.service: Passing 0 fds to service
Nov 06 10:07:22 dred systemd[1]: user@50421.service: About to execute: /usr/lib/systemd/systemd --user
Nov 06 10:07:22 dred systemd[1]: user@50421.service: Forked /usr/lib/systemd/systemd as 1892
Nov 06 10:07:22 dred systemd[1]: user@50421.service: Changed dead -> start
Nov 06 10:07:22 dred systemd[1]: user@50421.service: User lookup succeeded: uid=50421 gid=100
Nov 06 10:07:22 dred login[1884]: pam_systemd(login:session): Failed to create session: Start job for unit user@50421.service failed with 'failed'
Nov 06 10:07:22 dred login[1884]: pam_unix(login:session): session opened for user rzl by LOGIN(uid=0)
Nov 06 10:07:22 dred systemd[1892]: user@50421.service: Failed at step PAM spawning /usr/lib/systemd/systemd: Operation not permitted
Nov 06 10:07:22 dred systemd[1]: user@50421.service: Child 1892 belongs to user@50421.service
Nov 06 10:07:22 dred systemd[1]: user@50421.service: Main process exited, code=exited, status=224/PAM
Nov 06 10:07:22 dred systemd[1]: user@50421.service: Changed start -> failed
Nov 06 10:07:22 dred login[1884]: pam_krb5[1884]: error destroying ccache "DIR:/run/user/50421/krb5cc_iMs5RE"
Nov 06 10:07:22 dred login[1884]: pam_krb5[1884]: error destroying ccache "DIR:/run/user/50421/krb5cc_iMs5RE"
Nov 06 10:07:22 dred systemd[1]: user@50421.service: Job user@50421.service/start finished, result=failed
Nov 06 10:07:22 dred systemd[1]: user@50421.service: Unit entered failed state.
Nov 06 10:07:22 dred systemd[1]: user@50421.service: Failed with result 'protocol'.
Nov 06 10:07:22 dred systemd[1]: user@50421.service: cgroup is empty
Nov 06 10:07:22 dred systemd-logind[882]: Sent message type=error sender=n/a destination=:1.63 object=n/a interface=n/a member=n/a cookie=196 reply_cookie=2 error=Start job for unit user@50421.service failed with 'failed'
Nov 06 10:07:22 dred systemd[1]: Got cgroup empty notification for: /user.slice/user-50421.slice/user@50421.service
Nov 06 10:07:22 dred systemd[1665]: Got cgroup empty notification for: /user.slice/user-50421.slice/user@50421.service

systemd is failing to execute something because it (or something it executes) has no permissions, which is consistent with not having access to my $HOME because systemd --user and its children are not in the PAG. I think having the option to get back the old behavior (launching user daemons per session) is reasonable, and I'm willing to take a shot at it, though I'll need some guidance.

poettering commented 6 years ago

The default Kerberos configuration uses the session keyring, not the user keyring. (This might have changed with the advent of "user persistent" keyring type, hopefully.)

Why isn't the user keyring good enough?

I don't think it is something a distro can fix by shuffling PAM modules around.

Well, but it's still outside of the scope of systemd... If what you are say is true, then OpenAFS is generally incompatible with systemd? how has this ever worked?

grawity commented 6 years ago

then OpenAFS is generally incompatible with systemd

With systemd --user specifically, perhaps. But yes, I wouldn't call it a systemd bug either, just two incompatible designs. Ideally the credentials would be per-uid...

I'm kinda curious how AFS deals with cron jobs. It seems like it'd have the same problem... (My own AFS experience is very limited.)

poettering commented 6 years ago
systemd[1892]: user@50421.service: Failed at step PAM spawning /usr/lib/systemd/systemd: Operation not permitted

This indicates that PAM failed, and systemd just propagated that. I can't really tell you why EPERM was seen by PAM though

poettering commented 6 years ago

I think having the option to get back the old behavior (launching user daemons per session) is reasonable, and I'm willing to take a shot at it, though I'll need some guidance.

We never had such a mode. The "--user" instance of systemd has always been a singleton, refcounted by the active user sessions.

grawity commented 6 years ago

Well, if you turn off the user bus entirely, you should at least get the old GNOME behavior back.

poettering commented 6 years ago

sure, you can try "systemctl mask user@.service". YMMV, though

mbiebl commented 6 years ago

In the downstream Debian bug report, a user reported uninstalling the dbus-user-session package and using dbus-x11 instead, was apparently enough to get AFS working again. https://packages.debian.org/sid/dbus-user-session

rzlourenco commented 6 years ago

Well, if you turn off the user bus entirely, you should at least get the old GNOME behavior back.

sure, you can try "systemctl mask user@.service". YMMV, though

This solves my GNOME problem, thanks! I'll talk to the folk at OpenAFS, this interaction is something that should be documented. Are there any downsides? I've noticed that the following still happens:

$ systemctl --user status
Failed to read server status: Process org.freedesktop.systemd1 exited with status 1

In the downstream Debian bug report, a user reported uninstalling the dbus-user-session package and using dbus-x11 instead, was apparently enough to get AFS working again.

As far as I know, openSUSE doesn't have this, at least on Tumbleweed.

We never had such a mode. The "--user" instance of systemd has always been a singleton, refcounted by the active user sessions.

Thanks for the clarifications. Has the "--user" instance always been forked from PID 1? Also, there is perhaps a better way to handle, or at least report, this error? Reporting "permission denied" on /usr/lib/systemd/systemd is puzzling.

grawity commented 6 years ago

sure, you can try "systemctl mask user@.service". YMMV, though

I've noticed that the following still happens:

$ systemctl --user status
Failed to read server status: Process org.freedesktop.systemd1 exited with status 1

Well, yes, because you masked the service which provides systemd --user functionality. There is no direct bus activation support for it and AFAIK that's intentional.

(Even if it was allowed, systemd --user itself doesn't support more than one instance per uid, so it'd break as soon as you tried to use it from two dbus sessions.)

Has the "--user" instance always been forked from PID 1?

As far as I can remember, it has always been a service.

poettering commented 6 years ago

This solves my GNOME problem, thanks! I'll talk to the folk at OpenAFS, this interaction is something that should be documented. Are there any downsides?

Yes, many. You turned off user service management entirely. Hence "systemctl --user" and all that stuff won't work anymore.

poettering commented 6 years ago

Thanks for the clarifications. Has the "--user" instance always been forked from PID 1?

Yes, this always has been that way.

Also, there is perhaps a better way to handle, or at least report, this error? Reporting "permission denied" on /usr/lib/systemd/systemd is puzzling.

Well, the permission error is received from PAM, which in turn got it from some PAM module. How to debug PAM issues is something to discuss with the PAM community.

rzlourenco commented 6 years ago

Thanks for your patience and help.

heini commented 6 years ago

@poettering, I am the author of the Debian bug mentioned above. Just to clarify things a bit here: In an AFS filesystem, root is just a normal user, without special privileges. To access anything in /afs (which doesn't have explicit "system:anyuser rl" permissions), a user needs to acquire both a kerberos 5 ticket and an AFS token. This usually happens when the user logs in, by means of some pam modules, so that he can finally access his $HOME which is located in somewhere in /afs.

That being said: Since systemd --user is spawned from PID 1, it didn't go through PAM and thus can't have aquired neither a kerberos ticket, nor an AFS token. Which means it doesn't have access to the user's $HOME.

So, the permission error doesn't really come from PAM, and I don't think it involves /usr/lib/systemd. It rather comes from AFS and denies access to the users $HOME.

See also: https://docs.google.com/document/d/1P27fP1uj-C8QdxDKMKtI-Qh00c5_9zJa4YHjnpB6ODM/pub

poettering commented 6 years ago

That being said: Since systemd --user is spawned from PID 1, it didn't go through PAM and thus can't have aquired neither a kerberos ticket, nor an AFS token. Which means it doesn't have access to the user's $HOME.

As mentioned before, systemd --user also goes through PAM. If you look at the definition of user@.service you'll see that PAMName= is used, i.e. the systemd instance is run after PAM is set up for it too.

heini commented 6 years ago

Yes, but since it's somehow decoupled from the actual login process, it takes a completely different path through PAM than login, for example. On my Debian system /etc/pam.d/systemd-user looks like this:

# This file is part of systemd.
#
# Used by systemd --user instances.

@include common-account

session  required pam_selinux.so close
session  required pam_selinux.so nottys open
session  required pam_loginuid.so
session  required pam_limits.so
@include common-session-noninteractive
session optional pam_systemd.so

This is missing common-auth, which does all the user authentication. So it can never get kerberos tickets and AFS tokens.

heini commented 6 years ago

Hmm, what about having a pam module that spawns systemd --user processes instead of having them started by PID 1? Or starting them from some bash snippet in /etc/profile.d, or ...? There's enough alternatives.

poettering commented 6 years ago

@heini please work with the debian folks to make sure the pam snippet used pulls in the necessary modules. That's the correct approach really.

mbiebl commented 6 years ago

@poettering can you be more specific? Are you talking about /etc/pam.d/systemd-user? Are you saying that auth ... is required?

@heini does it solve your problem, if you hadd @include common-auth?

grawity commented 6 years ago

@mbiebl No, systemd doesn't run the auth stack as it never calls pam_authenticate().

@rzlourenco Does your system use pam_afs_session.so to obtain the tokens based on Kerberos tickets, or does it require the password directly?

If it uses Kerberos, I can see some ways of making it work.

mbiebl commented 6 years ago

@grawity well, I wasn't sure what @poettering meant in https://github.com/systemd/systemd/issues/7261#issuecomment-368457803 tbh, that's why I asked for clarification. It's not clear to me, who exactly (debian folks == debian systemd maintainers?) is supposed to fix which file (/etc/pam.d/systemd-user?) including what changes?

poettering commented 6 years ago

@poettering can you be more specific? Are you talking about /etc/pam.d/systemd-user? Are you saying that auth ... is required?

Oh, this was simply in reference to @heini's comments that the PAM snippet in use for it wouldn't pull in modules necessary for AFS, that's all…

poettering commented 6 years ago

@grawity well, I wasn't sure what @poettering meant in #7261 (comment) tbh, that's why I asked for clarification. It's not clear to me, who exactly (debian folks == debian systemd maintainers?) is supposed to fix which file (/etc/pam.d/systemd-user?) including what changes?

Well, I have no clue about AFS really, all I wanted to point out is that if the right modules aren't pulled in then this should be fixed by pulling them in, not by making PAM modules fork off systemd --user...

heini commented 6 years ago

@poettering, as outlined above, this can't work in the OpenAFS scenario. The process (systemd --user in this case) needs a Kerberos ticket + AFS token, which it can only obtain when being authenticated. There's no way for the Debian folks to solve this as long as systemd --user bypasses standard user authentication (by taking it's own path through the PAM stack). Pulling in other modules doesn't help here, either, because the user can't type in his username/password (the process is non-interactive, isn't it). Using PAM to fork systemd --user was just an example, btw. Whatever the solution would be, it would need to be one that spawns systemd --user from an authenticated process that's already running with Kerberos/AFS credentials or it won't ever get access to anything in /afs.

@mbiebl, tried to include it, but doesn't work. Doesn't even get me a systemd --user process, then.

poettering commented 6 years ago

@poettering, as outlined above, this can't work in the OpenAFS scenario. The process (systemd --user in this case) needs a Kerberos ticket + AFS token, which it can only obtain when being authenticated.

That sounds like a shortcoming of the design of AFS, no? I mean, what we are doing here with systemd is pretty much the same how things like crond do PAM: i.e. authentication doesn't take place but the account management and session hooks are run.

There's no way for the Debian folks to solve this as long as systemd --user bypasses standard user authentication (by taking it's own path through the PAM stack).

Uh, "we bypass standard user authentication" as much has crond and suchlike do. We take a pretty "normal" path through the PAM stack, but it doesn't involve authentication, as there's nothing to authenticate for the systemd --user instance, as it only runs for users that already authenticated independently beforehand.

Or let me ask explicitly: how does AFS work with crond?

Pulling in other modules doesn't help here, either, because the user can't type in his username/password (the process is non-interactive, isn't it). Using PAM to fork systemd --user was just an example, btw. Whatever the solution would be, it would need to be one that spawns systemd --user from an authenticated process that's already running with Kerberos/AFS credentials or it won't ever get access to anything in /afs.

kerberos places their key material in the per-user kernel keyring these days (and even before could place the material in a per-user location, so that it could be shared between sessions). Maybe AFS should do the same in order to be compatible with crond and systemd --user?

heini commented 6 years ago

Or let me ask explicitly: how does AFS work with crond?

For non-interactive users (aka system accounts), there's a tool called k5start that first obtains a Kerberos ticket by using a key file which then enables it to obtain an AFS token. However, this only works when the key contained in that file is passwordless. But the problem isn't non-interactive users here...

Maybe AFS should do the same in order to be compatible with crond and systemd --user?

Oh, it does:


% keyctl show
Session Keyring
 818743416 ---lswrv      0     0  keyring: _ses.17046
 346861339 ----s--v      0     0   \_ afs_pag: _pag```

But AFAIK this can only be accessed by child processes of the creator.
poettering commented 6 years ago

For non-interactive users (aka system accounts), there's a tool called k5start that first obtains a Kerberos ticket by using a key file which then enables it to obtain an AFS token. However, this only works when the key contained in that file is passwordless. But the problem isn't non-interactive users here...

Well, but what about regular users which install cronjobs?

poettering commented 6 years ago

But AFAIK this can only be accessed by child processes of the creator.

So, why not fix that and add them to the per-user keyring rather than the session keyring?

heini commented 6 years ago

Never had the use case for regular user cronjobs. But that's not the topic here, is it?

Why not fix systemd --user being side-loaded into the user's environment? Sounds wrong to me by all means.

poettering commented 6 years ago

Never had the use case for regular user cronjobs. But that's not the topic here, is it?

Ah, come on. It's certainly easy to discount everything that doesn't fit into your specific usecase as "never had a use for" and hence irrelevant, but this is not how we do things.

Why not fix systemd --user being side-loaded into the user's environment? Sounds wrong to me by all means.

Well, the --user instance is generally a singleton that is ref-counted by active, concurrent sessions, i.e. it is started in the bg on first concurrent login and stays around until the last logout of the user. It may thus survive many sessions of the same user as long as there's always one around to keep it around. Moreover if "lingering" is enabled for a user an implicit reference is kept from boot to shutdown, so that it ends up being started during boot without the user having logged in and until shutdown even if the user didn't stay logged in that long.

Long story short, yes, it is usually started on the event of a session login, but it's lifecycle is detached from it, and it quite likely survives it, hence it needs to be started independently of the session, as it shall not be terminated when the session goes away.

Moreover the --user instance shall explicitly not inherit arbitrary process properties (such as env vars, selinux labels, audit creds, X11 auth info…) from the original session that triggered its starting, since it shall be shared by other sessions with possibly very different process properties.

Anyway, please work with the AFS folks to fix their PAM hookup. They really should provide their key material to the user independently of the PAM authentication hook, since that is pureley for authentication and not for non-authenticated PAM sessions such as those managed by crond and similar, as well as systemd. Thank you.

heini commented 6 years ago

Well, that would mean to change the design of a 30+ year old, albeit still great, piece of software. Don't think this is going to happen...

grawity commented 6 years ago

You're talking about a very small part of said software. In the past "30+ years" I'm pretty sure it has been redesigned to fit different operating systems several times. Merely being old doesn't mean it's set in stone.

heini commented 6 years ago

No, it hasn't. It has been ported to different OS, but not redesigned.

poettering commented 6 years ago

Well, it's hardly a "redesign". And some other 30+ year old software is in the same boat here: crond.

(In general, I'd be careful with running 30+ year old network-facing software that has a maintainance problem)

filbranden commented 6 years ago

@heini For some more context, kernel keyring feature was added circa 2006 so AFS's use of it is at most ~10 years old...

Considering they already switched from some other mechanism to using the keyring, adapting it to use the per-user keyring instead of the per-session one is probably a minor switch...

Contact AFS about it, it's quite possible there's already a way to accomplish that or that the fix to do so is easy.

heini commented 6 years ago

@poettering, it didn't say it has a maintenance problem. It's still actively maintained.

@filbranden, I'll see what I can do. Thanks a lot.

heini commented 6 years ago

Started a discussion on the openafs-info mailing list. First reply indicates that it was an explicit design decision to prefer the session keyring over user keyring. Anyway, I proposed to add an option to the kernel module so that admins have the choice...

grawity commented 6 years ago

First reply indicates that it was an explicit design decision to prefer the session keyring over user keyring

Using the session keyring makes sense to me, too.

However, the session keyring almost always has a link to the user keyring, and the whole point of it is that a single recursive search would find keys in whichever keyring happens to have them. In theory, you'd need a session/user knob only in the tool which inserts credentials, and the kernel driver would find them with no extra options needed.

But it seems that e.g. cifs.ko (the SMB driver) ignores such links – if I manually add the exact same credentials to my user keyring, cifs.ko will not find them. Do you know if it's the same with OpenAFS too, and if it is, is that also an explicit decision?

heini commented 6 years ago

No, i don't. Sorry.

jaltman commented 6 years ago

OpenAFS, the Linux kernel AFS, and AuriStorFS file systems track tokens via a 30 year old technique known as Process Authentication Groups. When Linux keyrings are available, a session keyring is used to store the reference to the AFS tokens assigned to the PAG. A PAG has the following properties:

  1. A child parent inherits the PAG of its parent
  2. A process can leave a PAG (and have no PAG)
  3. A process can create a new PAG
  4. No one can join or otherwise access a PAG

PAGs ensure that two login sessions using the same local uid will not have access to the same network authentication credentials. Likewise, a process that changes its effective uid will not have access to the network authentication credentials of the user. uids are local and have no context when used with network services.

Since systemd --user is not part of the PAG its file system requests will not be associated with the user's AFS tokens. As a result, the AFS fileservers will see the process as the anonymous AFS user and deny access.

Executing pam_afs_session from systemd --user could obtain AFS tokens are process start but those tokens would require periodic renewal.

One question: is the systemd --user process tree supposed to represent the user identity or a system identity? If its a system identity, I believe the right thing would be for systemd --user to execute pam_afs_session using a system identity such as host/server@REALM stored in /etc/krb5.keytab.

Then users that wish to store $HOME in /afs could add the host's Kerberos identity to the ACL for the configuration files.

If on the otherhand, the systemd --user process tree is meant to have the same rights as the user, then it must be launched from the user's logon session.

poettering commented 6 years ago

One question: is the systemd --user process tree supposed to represent the user identity or a system identity?

Not sure what precisely you mean by "user identity" or a "system identity", but if by that you mean whether it is run for each "human" user, or whether it's only used for system users (i.e. UIDs below 1000), then it's the former: for each "human" user it's a system service that is spawned and reference counted by the user's login sessions, and those are typically getty, ssh, gnome or similar sessions, i.e. real, interactive ones.

poettering commented 6 years ago

If on the otherhand, the systemd --user process tree is meant to have the same rights as the user, then it must be launched from the user's logon session.

The systemd --user service is started when the user first logs in, and then maintained until all its sessions ended, it is ref-counted by open sessions of the same user if you so will. It is detached from the every user session otherwise: it runs in its own fresh executoin environment, it won't inherit anything from the sessions, as it very likely will survive the initial login session of the user, and will be reused for any other session.

Note that systemd --user itself is run as system service, though on behalf of the specific user, i.e. it won't get direct access to the sessions TTY or such, because it only manages background user services for the suer, and those shall not be effected by HUPs or so on the TTY.

jaltman commented 6 years ago

The systemd --user service is started when the user first logs in, and then maintained until all its sessions ended, it is ref-counted by open sessions of the same user if you so will. It is detached from the every user session otherwise: it runs in its own fresh executoin environment, it won't inherit anything from the sessions, as it very likely will survive the initial login session of the user, and will be reused for any other session.

There is a very important assumption being made here:

The security identity of all login sessions associated with the same uid are identical.

This assumption is not guaranteed to be valid under all circumstances. There is no requirement that the same network authentication identity be used to login with a particular uid. It is often the case that multiple Kerberos identities are permitted to share a local uid because authorization decisions when accessing network services such as remote file systems, web services, databases, etc are derived from the Kerberos identity and not the local uid.

I clearly am not familiar with the system architecture but I fear that systemd --user without proper care might be used to exploit the credentials of one login session by another sharing the same uid.

Note that systemd --user itself is run as system service, though on behalf of the specific user, i.e. it won't get direct access to the sessions TTY or such, because it only manages background user services for the user, and those shall not be effected by HUPs or so on the TTY.

So systemd --user is a shared system process. How are the background user services it manages supposed to access their configuration information that is stored in the login session's $HOME directory? Changing the effective uid of the process to that of the login user is ineffective when $HOME is stored in the network.

There isn't even a guarantee that the file system path $HOME evaluates to refers to the same object when the network authentication identities are distinct.

grawity commented 6 years ago

@jaltman: I guess the main question in this thread is whether all of the mentioned restrictions make sense nowadays.

I can see having multiple PAGs be useful for accessing different systems with different credentials (much like setting $KRB5CCNAME for Kerberos), but it seems like that should be voluntary. That is, it doesn't seem to be particularly useful as an enforced security feature – there isn't much that prevents one process in PAG A from tampering with another in PAG B. (And, does it even matter if AFS tokens are obtainable off Kerberos tickets and those are shared across the entire uid?)

One question: is the systemd --user process tree supposed to represent the user identity or a system identity?

I suppose it represents the user identity, although it is launched "in background" and so doesn't receive authentication details from PAM. (It still runs the account/session PAM modules though.) You could say it's very much like cron – with the addition that interactive apps are now often hosted under it as well. How is this problem usually solved by cron?

So systemd --user is a shared system process.

It's not shared – there's one instance per uid, each having its own environment (envp, keyrings, etc.)

poettering commented 6 years ago

The security identity of all login sessions associated with the same uid are identical.

Yupp, and nothing else makes much sense. On Linux access control is fundamentally based on UIDs, its the one credential concept that is universal on Linux, and to that pretty much everything adheres, from file access control, to shared memory access control, IPC access, and so on. And given that users can ptrace their own processes ignoring that won't do much good, but just complicate stuff.

poettering commented 6 years ago

I clearly am not familiar with the system architecture but I fear that systemd --user without proper care might be used to exploit the credentials of one login session by another sharing the same uid.

Yeah, as mentioned, on Linux any process can do that really. gdb is your exploit tool.

poettering commented 6 years ago

So systemd --user is a shared system process. How are the background user services it manages supposed to access their configuration information that is stored in the login session's $HOME directory? Changing the effective uid of the process to that of the login user is ineffective when $HOME is stored in the network.

On Linux home directories are defined per-user, not per-session. systemd --user gets $HOME set the same way any login session of the same user would get it set.

There isn't even a guarantee that the file system path $HOME evaluates to refers to the same object when the network authentication identities are distinct.

Well, you are in big trouble anyway if AFS introduces per-session $HOME directories. This is going to break with half our stack, because secondary user creds tend to be stored in $HOME (X11, dbus tokens and suchlike), and programs of the same user tend to pass around directory paths between themselves all the time under the assumption they refer to the same resources.