[BUG] [DISK] Soft-fails whith `df: [...]: Operation not permitted`

[SomethingGeneric]

Describe the bug System says df: /run/user/1000/doc: Operation not permitted, which as far as I know, means that it's trying to list the temp files related to your session.

Expected behavior No error message, and be able to see the disk usage.

Screenshots

Environment

• Version used (4.Y.Z) : 4.8.1
• Method of installation (distribution package, PyPI, sources) : Arch .pkg.tar.xz
• Hardware type (laptop, server, Raspberry, hyper-visor) : Custom built PC
• Python version (3.Y.Z) : 3.8.6
• Operating system and version : Arch Linux (rolling), Kernel v 5.9.8-arch1-1
• Graphical environment name and version : Gnome Shell & Gnome DE. GNOME 3.38.1
• Connectivity (off-line, LAN only, Internet access) : LAN & Internet over Ethernet

Additional context I assume a package upgrade is at fault here, or something, because disk usage worked correctly until latest system update. Not sure what I'm missing.

[HorlogeSkynet]

Welcome back @SomethingGeneric !

Quick diagnostic : do you use Flatpak on your system ?

PS : You really should hide your public IPs from the screenshots you share here.

Bye :wave:

[SomethingGeneric]

I have the gnome software center installed with my DE, but haven't installed anything. Do flatpaks cause that output?

[HorlogeSkynet]

Yes Flatpak may cause this output, see flatpak/xdg-desktop-portal#512. We can silence df STDERR as a workaround here, nevertheless it'd only hide the underlying error.

Also see :

• https://askubuntu.com/questions/1227667/df-command-throws-error-on-run-user-1000-doc-folder
• https://discussion.fedoraproject.org/t/output-of-df-with-df-run-user-1000-doc-operation-not-permitted/20424

Anyway, this issue does not seem to be related to Archey :shrug:

[SomethingGeneric]

Yeah, I think you're right. I can look into other tools to replace df or to ignore those folders.

[HorlogeSkynet]

"Other tools" are also impacted (see muesli/duf#35), but mostly not widely available so it'd break many distros :confused:

And about working with a whitelist/blacklist of paths, it's actually already done (see https://github.com/HorlogeSkynet/archey4/blob/v4.8.1/archey/entries/disk.py#L41), and /run/* are indeed "ignored". The problem is only about df walking through each available mounted point, including them.

[SomethingGeneric]

I think it's also related to permissions. sudo df ... doesn't throw the error.

Also, I might be rusty on regex, but it doesn't seem like device_path_regexp = re.compile(r'^\/dev\/(?:(?!loop|[rs]?vnd|lofi|dm).)+$') targets /run too?

[SomethingGeneric]

Wait I'm now realizing it's a whitelist and not a blacklist. That's extra weird. :/

[HorlogeSkynet]

I think it's also related to permissions. sudo df ... doesn't throw the error.

Indeed, but df is supposed to run smoothly as non-root too :confused:

Also, I might be rusty on regex, but it doesn't seem like device_path_regexp = re.compile(r'^\/dev\/(?:(?!loop|[rs]?vnd|lofi|dm).)+$') targets /run too?

Yeah @ingrinder works on this REGEXP and implicitly excluded /run* paths here. The pattern is used for post-processing purposes based on df (global) output (usage is df -P -k since v4.8.0, for a very wide support).

[SomethingGeneric]

This might seem dumb, but would it be common for other distros to have different versions of df? On arch, it's from GNU coreutils, but others might have from a different source.

[HorlogeSkynet]

This might seem dumb, but would it be common for other distros to have different versions of df?

Yes, it's a PITA. The df from BusyBox, for instance, does not support some flags Archey used to specify...

---

I've been thinking about your issue though, and despite the error being displayed, you should not get a Not detected value for Disk. Could you please attach below your df -P -k output (as non-root user) ?

[adjagu]

[doctor@who ~]$ archey4
df: /run/user/1000/doc: Operation not permitted
 $$$$$$$$$$$$$$$$  $$$$$$$   User: doctor
 M77777777777777M  M77777M   Hostname: who
 M77777777777777M  M77777M   Model: Inspiron 3252 2.13.0
 M77777MMMMMMMMMM  M77777M   Distro: Manjaro Linux [x86_64]
 M77777M           M77777M   Kernel: 5.4.77-1-MANJARO
 M77777M  $$$$$$$  M77777M   Uptime: 2 hours and 9 minutes
 MMMMMMM  M77777M  M77777M   Processes: 173
          M77777M  M77777M   WindowManager: KWin
 $$$$$$$  M77777M  M77777M   DesktopEnvironment: KDE
 M77777M  M77777M  M77777M   Shell: /bin/bash
 M77777M  M77777M  M77777M   Terminal: Konsole ██ ██ ██ ██ ██ ██ ██
 M77777M  M77777M  M77777M   Packages: 1270
 M77777M  M77777M  M77777M   Temperature: 93.2 F (Max. 93.2 F)
 M77777M  M77777M  M77777M   CPU: Intel(R) Pentium(R) CPU N3700 @ 1.60GHz
 M77777M  M77777M  M77777M   GPU: Intel Corporation Atom/Celeron/Pentium Processor...
 M77777M  M77777M  M77777M   RAM: 1087 MiB / 3851 MiB
 M77777M  M77777M  M77777M   Disk: Not detected
 MMMMMMM  MMMMMMM  MMMMMMM   LAN_IP: xxx.xxx.xxx.xxx, xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx
                             WAN_IP: xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx

[doctor@who ~]$ df -P -k
df: /run/user/1000/doc: Operation not permitted
Filesystem     1024-blocks     Used Available Capacity Mounted on
dev                1962460        0   1962460       0% /dev
run                1972044     1232   1970812       1% /run
/dev/sda1        102687672 16051724  81376684      17% /
tmpfs              1972044        0   1972044       0% /dev/shm
tmpfs                 4096        0      4096       0% /sys/fs/cgroup
tmpfs              1972044    47632   1924412       3% /tmp
/dev/sda3           523248      280    522968       1% /boot/efi
/dev/sda4        371811132 70272932 282581476      20% /home
tmpfs               394408       96    394312       1% /run/user/1000

[SomethingGeneric]

Here's my df -P -k.

In addition, I do get Not detected? Both reinstalled with git clone ... && sudo pip3 install . and using arch package??

Also, what does PITA mean? And do you think the coreutils matches most other distros where it works?

[SomethingGeneric]

Here's a better formatted screenshot. Sorry abt that

[adjagu]

@SomethingGeneric PITA = pain in the ass. Something that is more difficult than it needs to be.

[SomethingGeneric]

@SomethingGeneric PITA = pain in the ass. Something that is more difficult than it needs to be.

Ah, I assumed there was some technical term. Forgot about that good old acronym. Lol

[HorlogeSkynet]

Could you run df -P -k && echo $? on your setup(s) please ?

[SomethingGeneric]

No change?

[HorlogeSkynet]

Yep, that's what I feared. df is hard-failing so no post-processing is performed at all, that accounts for your Not detected value.

https://github.com/HorlogeSkynet/archey4/blob/v4.8.1/archey/entries/disk.py#L113

df is holding exit status code, this means it "fails" from the moment an error occurs : https://github.com/coreutils/coreutils/blob/v8.32/src/df.c#L1846.

[SomethingGeneric]

Ah, so as far as Archey is concerned, df just doesn't exist?

[adjagu]

[doctor@who ~]$ df -P -k && echo $?
df: /run/user/1000/doc: Operation not permitted
Filesystem     1024-blocks     Used Available Capacity Mounted on
dev                1962460        0   1962460       0% /dev
run                1972044     1236   1970808       1% /run
/dev/sda1        102687672 16051728  81376680      17% /
tmpfs              1972044        0   1972044       0% /dev/shm
tmpfs                 4096        0      4096       0% /sys/fs/cgroup
tmpfs              1972044    47632   1924412       3% /tmp
/dev/sda3           523248      280    522968       1% /boot/efi
/dev/sda4        371811132 70286300 282568108      20% /home
tmpfs               394408       96    394312       1% /run/user/1000

[SomethingGeneric]

[doctor@who ~]$ df -P -k && echo $?
df: /run/user/1000/doc: Operation not permitted
Filesystem     1024-blocks     Used Available Capacity Mounted on
dev                1962460        0   1962460       0% /dev
run                1972044     1236   1970808       1% /run
/dev/sda1        102687672 16051728  81376680      17% /
tmpfs              1972044        0   1972044       0% /dev/shm
tmpfs                 4096        0      4096       0% /sys/fs/cgroup
tmpfs              1972044    47632   1924412       3% /tmp
/dev/sda3           523248      280    522968       1% /boot/efi
/dev/sda4        371811132 70286300 282568108      20% /home
tmpfs               394408       96    394312       1% /run/user/1000

I assume we're identical because @AdJaGu is on Manjaro (downstream Arch)

[SomethingGeneric]

So I tested the Flatpak theory with sudo pacman -R gnome-software and rebooting. Indeed, the issue is Flatpak. Personally, I might end up just going with this since I prefer Snaps, and use them instead. I'd still be happy to work on a fix though, for others like @AdJaGu

[adjagu]

Ran sudo pacman -R flatpak pamac-flatpak-plugin and rebooted. No change here on Manjaro.

[doctor@who ~]$ archey4
df: /run/user/1000/doc: Operation not permitted
 $$$$$$$$$$$$$$$$  $$$$$$$   User: doctor
 M77777777777777M  M77777M   Hostname: who
 M77777777777777M  M77777M   Model: Inspiron 3252 2.13.0
 M77777MMMMMMMMMM  M77777M   Distro: Manjaro Linux [x86_64]
 M77777M           M77777M   Kernel: 5.4.77-1-MANJARO
 M77777M  $$$$$$$  M77777M   Uptime: 8 minutes
 MMMMMMM  M77777M  M77777M   Processes: 177
          M77777M  M77777M   WindowManager: KWin
 $$$$$$$  M77777M  M77777M   DesktopEnvironment: KDE
 M77777M  M77777M  M77777M   Shell: /bin/bash
 M77777M  M77777M  M77777M   Terminal: Konsole ██ ██ ██ ██ ██ ██ ██
 M77777M  M77777M  M77777M   Packages: 1268
 M77777M  M77777M  M77777M   Temperature: 96.8 F (Max. 98.6 F)
 M77777M  M77777M  M77777M   CPU: Intel(R) Pentium(R) CPU N3700 @ 1.60GHz
 M77777M  M77777M  M77777M   GPU: Intel Corporation Atom/Celeron/Pentium Processor...
 M77777M  M77777M  M77777M   RAM: 1029 MiB / 3851 MiB
 M77777M  M77777M  M77777M   Disk: Not detected
 MMMMMMM  MMMMMMM  MMMMMMM   LAN_IP: 192.168.1.22, xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx
                             WAN_IP: xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx

[doctor@who ~]$ df -P -k
df: /run/user/1000/doc: Operation not permitted
Filesystem     1024-blocks     Used Available Capacity Mounted on
dev                1962456        0   1962456       0% /dev
run                1972040     1228   1970812       1% /run
/dev/sda1        102687672 16042476  81385932      17% /
tmpfs              1972040        0   1972040       0% /dev/shm
tmpfs                 4096        0      4096       0% /sys/fs/cgroup
tmpfs              1972044    46920   1925124       3% /tmp
/dev/sda3           523248      280    522968       1% /boot/efi
/dev/sda4        371811132 70295004 282559404      20% /home
tmpfs               394408       96    394312       1% /run/user/1000

[doctor@who ~]$ df -P -k && echo $?
df: /run/user/1000/doc: Operation not permitted
Filesystem     1024-blocks     Used Available Capacity Mounted on
dev                1962456        0   1962456       0% /dev
run                1972040     1228   1970812       1% /run
/dev/sda1        102687672 16042476  81385932      17% /
tmpfs              1972040        0   1972040       0% /dev/shm
tmpfs                 4096        0      4096       0% /sys/fs/cgroup
tmpfs              1972044    46920   1925124       3% /tmp
/dev/sda3           523248      280    522968       1% /boot/efi
/dev/sda4        371811132 70295004 282559404      20% /home
tmpfs               394408       96    394312       1% /run/user/1000

[SomethingGeneric]

@AdJaGu I think KDE software center is also Flatpak based?

[SomethingGeneric]

Might be wrong about last. Just reinstalled Arch (not related, I'm just dumb), with KDE, and had no issue running archey. Disk detected just fine

[adjagu]

@SomethingGeneric yeah KDE Discover has an optional and make dependency of flatpak but, Manjaro uses Pamac, so KDE Discover (that's the KDE Software Center you speak of correct?) isn't even installed on my system. Either way I just use pacman unless I'm installing something from the AUR.

When I removed flatpak and pamac-flatpak-plugin that removed flatpak from my system entirely.

[SomethingGeneric]

That's weird. Still not working?

[adjagu]

@SomethingGeneric correct. As outlined in a previous comment of mine ( https://github.com/HorlogeSkynet/archey4/issues/92#issuecomment-730608896 ) removing flatpak and pamac-flatpak-plugin the issue remains. Those were the only bits of flatpak installed on my system.

[HorlogeSkynet]

@AdJaGu Could you run systemctl --user status xdg-document-portal.service on your Manjaro from your regular user please ?

---

@SomethingGeneric I'll be working on an error-proof patch for the v4.8 branch today.

[adjagu]

@HorlogeSkynet yes I can. Results below.

[doctor@who ~]$ systemctl --user status xdg-document-portal.service
● xdg-document-portal.service - flatpak document portal service
     Loaded: loaded (/usr/lib/systemd/user/xdg-document-portal.service; static)
     Active: active (running) since Fri 2020-11-20 11:41:25 EST; 9min ago
   Main PID: 1090 (xdg-document-po)
     CGroup: /user.slice/user-1000.slice/user@1000.service/xdg-document-portal.service
             ├─1090 /usr/lib/xdg-document-portal
             └─1098 fusermount -o rw,nosuid,nodev,fsname=portal,auto_unmount,subtype=portal -- /run/user/1000/doc

Nov 20 11:41:25 who systemd[797]: Starting flatpak document portal service...
Nov 20 11:41:25 who systemd[797]: Started flatpak document portal service.

[HorlogeSkynet]

@AdJaGu Thanks ! So I guess Flatpak removal has not been complete as we would expect...

[adjagu]

@HorlogeSkynet yeah. Searching for xdg- in installed programs returns:

xdg-dbus-proxy
xdg-desktop-portal
xdg-desktop-portal-kde
xdg-su
xdg-user-dirs
xdg-utils

xdg-desktop-portal --which is required by manjaro-kde-settings and is explicitly installed-- is the offender that provides /usr/lib/systemd/user/xdg-desktop-portal.service, /usr/lib/systemd/user/xdg-document-portal.service, and /usr/lib/systemd/user/xdg-permission-store.service.

Stopping xdg-document-portal.service allows archey4 to run without error:

[doctor@who ~]$ systemctl --user stop xdg-document-portal.service

[doctor@who ~]$ systemctl --user status xdg-document-portal.service
● xdg-document-portal.service - flatpak document portal service
     Loaded: loaded (/usr/lib/systemd/user/xdg-document-portal.service; static)
     Active: inactive (dead)

Nov 20 11:41:25 who systemd[797]: Starting flatpak document portal service...
Nov 20 11:41:25 who systemd[797]: Started flatpak document portal service.
Nov 20 12:03:53 who systemd[797]: Stopping flatpak document portal service...
Nov 20 12:03:53 who systemd[797]: xdg-document-portal.service: Succeeded.
Nov 20 12:03:53 who systemd[797]: Stopped flatpak document portal service.

[doctor@who ~]$ df -P -k && echo $?  
Filesystem     1024-blocks     Used Available Capacity Mounted on
dev                1962460        0   1962460       0% /dev
run                1972044     1228   1970816       1% /run
/dev/sda1        102687672 15993660  81434748      17% /
tmpfs              1972044        0   1972044       0% /dev/shm
tmpfs                 4096        0      4096       0% /sys/fs/cgroup
tmpfs              1972044    46920   1925124       3% /tmp
/dev/sda3           523248      280    522968       1% /boot/efi
/dev/sda4        371811132 70292976 282561432      20% /home
tmpfs               394408       96    394312       1% /run/user/1000
0

[doctor@who ~]$ archey4
 $$$$$$$$$$$$$$$$  $$$$$$$   User: doctor
 M77777777777777M  M77777M   Hostname: who
 M77777777777777M  M77777M   Model: Inspiron 3252 2.13.0
 M77777MMMMMMMMMM  M77777M   Distro: Manjaro Linux [x86_64]
 M77777M           M77777M   Kernel: 5.4.77-1-MANJARO
 M77777M  $$$$$$$  M77777M   Uptime: 38 minutes
 MMMMMMM  M77777M  M77777M   Processes: 171
          M77777M  M77777M   WindowManager: KWin
 $$$$$$$  M77777M  M77777M   DesktopEnvironment: KDE
 M77777M  M77777M  M77777M   Shell: /bin/bash
 M77777M  M77777M  M77777M   Terminal: Konsole ██ ██ ██ ██ ██ ██ ██
 M77777M  M77777M  M77777M   Packages: 1261
 M77777M  M77777M  M77777M   Temperature: 95.5 F (Max. 98.6 F)
 M77777M  M77777M  M77777M   CPU: Intel(R) Pentium(R) CPU N3700 @ 1.60GHz
 M77777M  M77777M  M77777M   GPU: Intel Corporation Atom/Celeron/Pentium Processor x5-E8000/J3xxx/N3xxx Integrated Graphics Controller (rev 21)
 M77777M  M77777M  M77777M   RAM: 1315 MiB / 3851 MiB
 M77777M  M77777M  M77777M   Disk: 82.3 GiB / 453.0 GiB
 MMMMMMM  MMMMMMM  MMMMMMM   LAN_IP: 192.168.1.22, xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx
                             WAN_IP: xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx

Not sure what to do now.

[SomethingGeneric]

Seems like removing manjaro-kde-settings would be a bad idea. Any other thoughts, @HorlogeSkynet ?

[HorlogeSkynet]

Of course, we won't advise anyone to remove any package to silence an upstream error...

Now we know that flatpak/xdg-desktop-portal#512 is responsible for what you are experiencing, I'm currently testing a workaround for this kind of issues.

[HorlogeSkynet]

@AdJaGu @SomethingGeneric Could you please test latest master ? :rocket:

Also, do you think we should draw a v4.8.2 or this patch could wait for v4.9 expected soon ?

Bye :wave:

[SomethingGeneric]

@HorlogeSkynet Arch KDE doesn't have this issue, so I can't test the patch for you :(

[HorlogeSkynet]

@HorlogeSkynet Arch KDE doesn't have this issue, so I can't test the patch for you :(

With Flatpak installed for instance as when you reported :stuck_out_tongue_winking_eye:

[SomethingGeneric]

@HorlogeSkynet Arch KDE doesn't have this issue, so I can't test the patch for you :(

With Flatpak installed for instance as when you reported

In my issue report I had Arch & GNOME. Though, I'm not sure why Manjaro KDE Software causes this and Arch KDE software doesn't. So either way, it's a question now for @AdJaGu & others on Manjaro KDE

[SomethingGeneric]

(Pic or it didn't happen :stuck_out_tongue_winking_eye:

[adjagu]

@HorlogeSkynet yes I can and I have no issues waiting for this patch until 4.9.

[doctor@who github]$ git clone https://github.com/HorlogeSkynet/archey4.git
Cloning into 'archey4'...
remote: Enumerating objects: 60, done.
remote: Counting objects: 100% (60/60), done.
remote: Compressing objects: 100% (37/37), done.
remote: Total 2449 (delta 27), reused 46 (delta 19), pack-reused 2389
Receiving objects: 100% (2449/2449), 672.63 KiB | 3.50 MiB/s, done.
Resolving deltas: 100% (1517/1517), done.

[doctor@who github]$ cd archey4

[doctor@who archey4]$ sudo pip3 install .
Processing /home/doctor/develop/github/archey4
Requirement already satisfied: distro in /usr/lib/python3.8/site-packages (from archey4==4.8.1) (1.5.0)
Requirement already satisfied: netifaces in /usr/lib/python3.8/site-packages (from archey4==4.8.1) (0.10.9)
Using legacy 'setup.py install' for archey4, since package 'wheel' is not installed.
Installing collected packages: archey4
    Running setup.py install for archey4 ... done
Successfully installed archey4-4.8.1

[doctor@who archey4]$ archey
 $$$$$$$$$$$$$$$$  $$$$$$$   User: doctor
 M77777777777777M  M77777M   Hostname: who
 M77777777777777M  M77777M   Model: Inspiron 3252 2.13.0
 M77777MMMMMMMMMM  M77777M   Distro: Manjaro Linux [x86_64]
 M77777M           M77777M   Kernel: 5.4.77-1-MANJARO
 M77777M  $$$$$$$  M77777M   Uptime: 1 hour and 15 minutes
 MMMMMMM  M77777M  M77777M   Processes: 173
          M77777M  M77777M   WindowManager: KWin
 $$$$$$$  M77777M  M77777M   DesktopEnvironment: KDE
 M77777M  M77777M  M77777M   Shell: /bin/bash
 M77777M  M77777M  M77777M   Terminal: Konsole ██ ██ ██ ██ ██ ██ ██
 M77777M  M77777M  M77777M   Packages: 1260
 M77777M  M77777M  M77777M   Temperature: 36.5 C (Max. 39.0 C)
 M77777M  M77777M  M77777M   CPU: Intel(R) Pentium(R) CPU N3700 @ 1.60GHz
 M77777M  M77777M  M77777M   GPU: Intel Corporation Atom/Celeron/Pentium Processor x5-E8000/J3xxx/N3xxx Integrated Graphics Controller (rev 21)
 M77777M  M77777M  M77777M   RAM: 1167 MiB / 3851 MiB
 M77777M  M77777M  M77777M   Disk: 82.3 GiB / 453.0 GiB
 MMMMMMM  MMMMMMM  MMMMMMM   LAN_IP: 192.168.1.22, xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx
                             WAN_IP: xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx

Working as intended now :)

[HorlogeSkynet]

Perfect, thanks to you both. See you soon :bow: