BUG: client hangs on missing mount resource

rgbriggs commented 6 years ago

The process hangs when trying to unmount a missing filesystem (ceph, cifs, 9p, lustre, fuse (gluster) or NFS) when the task is auditable. This happens because user_path_mountpoint_at() calls audit_copy_inode() (via filename_mountpoint() and audit_inode()) which attempts to record the mounted filesystem's root directory fcaps to report in a PATH record.

Sample backtrace:

[3371449.641117]  [<ffffffff816ab6d9>] schedule+0x29/0x70
[3371449.641123]  [<ffffffffc04e6035>] __fuse_get_req+0x1a5/0x280 [fuse]
[3371449.641125]  [<ffffffff810b34b0>] ? wake_up_atomic_t+0x30/0x30
[3371449.641128]  [<ffffffffc04e6120>] fuse_get_req+0x10/0x20 [fuse]
[3371449.641130]  [<ffffffffc04e918e>] fuse_getxattr+0x5e/0x190 [fuse]
[3371449.641132]  [<ffffffff81213019>] ? path_mountpoint+0x359/0x430
[3371449.641135]  [<ffffffff812b2159>] get_vfs_caps_from_disk+0x69/0x120
[3371449.641136]  [<ffffffff8111a9fd>] audit_copy_inode+0x6d/0xb0
[3371449.641138]  [<ffffffff811215fa>] __audit_inode+0x18a/0x3c0
[3371449.641140]  [<ffffffff81213dea>] filename_mountpoint+0x8a/0xd0
[3371449.641141]  [<ffffffff81214e32>] user_path_mountpoint_at+0x32/0x40
[3371449.641144]  [<ffffffff8122579e>] SyS_umount+0x7e/0x400

Steps to Reproduce:

Set up trivial fuse filesystem:

git clone git://github.com/libfuse/libfuse.git -b fuse_2_9_bugfix
cd libfuse/
./makeconf.sh
./configure
make
cd example

OR

dnf -y install fuse-devel
wget https://raw.githubusercontent.com/libfuse/libfuse/fuse_2_9_bugfix/example/fusexmp.c
gcc -Wall fusexmp.c -DHAVE_SETXATTR `pkg-config fuse --cflags --libs` -o fusexmp

Set up an audit rule to trigger it:

auditctl -A exit,always -S umount2 -F key=umount2

Then run:
```
./fusexmp -f -s /mnt
^Z
```
At this point the filesystem is hung (simulates network failure).
```
umount -l /mnt
```
Normally this should succeed, but if audit is doing a getxattr, then it will hang. Note the comment above user_path_mountpoint_at() that warns:
- A umount is a special case for path walking. We're not actually interested
- in the inode in this situation, and ESTALE errors can be a problem. We
- simply want track down the dentry and vfsmount attached at the mountpoint
- and avoid revalidating the last component.

Suggested solutions:

Upon reading the comment above user_path_mountpoint_at() this looks like a bad idea and at minimum the MNT_FORCE flag should signal to audit not to fetch the mount's root directory fcaps.
Address this the same way we addressed https://github.com/linux-audit/audit-kernel/issues/8 by allowing certain types of filesystems to be bypassed, adding this functionality to __audit_inode() the way it was implemented for __audit_inode_child().

rgbriggs commented 6 years ago

Paul, please assign to me... ref bz1570612

pcmoore commented 6 years ago

Paul, please assign to me

Done.

Sorry for the delay (vacation).