Sudo does not work in Fedora 34

[alvarlagerlof]

Describe the bug Sudo commands fails with sudo: /usr/bin/sudo must be owned by uid 0 and have the setuid bit set

Steps how to reproduce the behaviour

toolbox create --distro fedora --release 34
toolbox enter --release 34
sudo

Expected behaviour sudo works

Actual behaviour sudo outputs sudo: /usr/bin/sudo must be owned by uid 0 and have the setuid bit set

Output of toolbox --version (v0.0.90+) toolbox version 0.0.99.1

Toolbox package info (rpm -q toolbox) toolbox-0.0.99.1-1.fc34.x86_64

Output of podman version

Version:      3.1.0
API Version:  3.1.0
Go Version:   go1.16
Built:        Tue Mar 30 15:29:36 2021
OS/Arch:      linux/amd64

Podman package info (rpm -q podman) podman-3.1.0-1.fc34.x86_64

Info about your OS Fedora Silverblue 34

[debarshiray]

Umm... I am confused.

I have a Fedora 34 Silverblue laptop here:

[rishi@bollard ~]$ rpm-ostree status
State: idle
Deployments:
● fedora:fedora/34/x86_64/silverblue
                   Version: 34.20211028.0 (2021-10-28T19:04:57Z)
                BaseCommit: 18cbafdadffd890038582490d7d7f22cebcd1be808ba72ef2df36016ddc38996
              GPGSignature: Valid signature by 8C5BA6990BDB26E19F2A1A801161AE6945719A39
           LayeredPackages: gstreamer1-plugins-ugly emacs mozilla-openh264 zsh gstreamer1-libav gstreamer1-plugin-openh264 gnupg1
             LocalPackages: rpmfusion-free-release-34-1.noarch

Did you really run:

$ toolbox create --distro fedora --release 34
...
$ toolbox enter --release 34 sudo
...

I am asking because that should have led to:

[rishi@bollard ~]$ toolbox enter --release 34 sudo
Error: container sudo not found
Use the '--container' option to select a toolbox.
Run 'toolbox --help' for usage.

Inside the container, I see:

⬢[rishi@toolbox ~]$ ls -l $(which sudo)
---s--x--x. 1 root root 185504 Jan 26  2021 /usr/bin/sudo

[alvarlagerlof]

I'm sorry, my command got jumbled. I ran three different commands, ending with sudo when inside the container.

[debarshiray]

I see.

As I showed above, the permissions of the sudo binary are correct inside the Toolbox container. Do you see something else?

[alvarlagerlof]

Looks a bit different for me for some reason. It's showing x instead of s on the first one. How can that be?

[alvar@rehoboam config]$ toolbox create --release 34 
Creating container fedora-toolbox-34: | Created container: fedora-toolbox-34
Enter with: toolbox enter fedora-toolbox-34
[alvar@rehoboam config]$ toolbox enter fedora-toolbox-34
⬢[alvar@toolbox config]$ ls -l $(which sudo)
---x--x--x. 1 root root 185504 Jan 26  2021 /usr/bin/sudo
⬢[alvar@toolbox config]$ sudo
sudo: /usr/bin/sudo must be owned by uid 0 and have the setuid bit set
⬢[alvar@toolbox config]$ 

[debarshiray]

Was it a custom image?

Could it have been an old buggy image? Although, I am not aware of any such buggy images.

I tested with a fresh new fedora-toolbox:34 image from the registry, and it seems fine.

Do you see the same thing with other images too? eg., Fedoras 33, 35 and 36.

[olivergs]

I was not able to reproduce this either.

Try to remove any images for f34 in your local registry using podman rmi image_name and try again just to check you're using the latest f34 image from the fedora regirstries.

[debarshiray]

Thanks for trying it out, @olivergs

I think this is probably something else, because if sudo(8) was really broken inside Toolbox containers then it would have set our bug trackers (both upstream and downstream) on fire.

I am tentatively closing this, but feel free to re-open or leave a comment if you have further updates.

[alvarlagerlof]

Was it a custom image?

No, I have no idea how to make those.

Do you see the same thing with other images too? eg., Fedoras 33, 35 and 36.

33 and 35 works file.

Try to remove any images for f34 in your local registry using podman rmi image_name and try again just to check you're using the latest f34 image from the fedora regirstries.

The thing is that I did this. And it was broken efter after seemingly deleteing, redowloading, and creating.

[alvar@rehoboam ~]$ podman rmi fedora-toolbox:34
Untagged: registry.fedoraproject.org/fedora-toolbox:34
Deleted: 7dbe6a8efc65fa9c62caa3c61f496647c717acb48013bf8f213c9106a2ddd300
[alvar@rehoboam ~]$ toolbox create --release 34 
Image required to create toolbox container.
Download registry.fedoraproject.org/fedora-toolbox:34 (500MB)? [y/N]: y
Creating container fedora-toolbox-34: | Created container: fedora-toolbox-34
Enter with: toolbox enter fedora-toolbox-34
[alvar@rehoboam ~]$ toolbox enter fedora-toolbox-34
⬢[alvar@toolbox ~]$ sudo
sudo: /usr/bin/sudo must be owned by uid 0 and have the setuid bit set
⬢[alvar@toolbox ~]$ 

I think this is probably something else, because if sudo(8) was really broken inside Toolbox containers then it would have set our bug trackers (both upstream and downstream) on fire.

Very likely something wrong on just my system. But I'd still like to know why and potentially fix a bug. Maybe podman rmi is not working?

[debarshiray]

Did you happen to copy ~/.local/share/containers from a different system?

@tpopela said that doing so reproduces this issue for him.

[alvarlagerlof]

Did you happen to copy ~/.local/share/containers from a different system?

Not that I recall? I may have changed permissions on that folder due to an earlier though. What are the right ones?

[olivergs]

This are the base permissions

drwx------ 4 oliver oliver 4096 nov 22 16:51 containers/ drwx------ 2 oliver oliver 4096 oct 15 12:47 containers/cache drwx------ 9 oliver oliver 4096 nov 22 16:51 containers/storage

But what I recommend just to check if this is the problem is to rename the containers directory to another name, then try to create a new toolbox and check what happens.

You can recover your old containers directory after that if you need, and then compare the permissions of the newly created one.

[tpopela]

I've started with an empty ~/.local/share/containers and recreated the F35 toolbox container (and downloaded the image) and things are working.

[alvarlagerlof]

So it's apparently one of the overlay that is not 10000:10000 and in on of the overlay-containers a userdata has incorrect permissions too. This makes toolbox rmi fail with Error: failed to inspect image fedora-toolbox-34.

Creating a new containers folder works. Might just do that now. I think I might have done is try to back up my files with Pika backup, and found permission errors on folders in the containers folder. So I natively changed it. Sorry for not remebering earlier.

[JohnAtl]

For other weary travelers that might arrive here: I changed permissions on .local/share/containers/* when I messed something up, and this caused the sudo permission error. Deleting .local/share/containers (as mentioned above) and creating new containers fixed the issue.

[debarshiray]

I am glad you managed to track it down, @alvarlagerlof @JohnAtl

We have encountered some bugs with permissions under ~/.local/share/containers in the past. See https://github.com/containers/toolbox/issues/1299 and https://github.com/containers/toolbox/issues/880 They usually turn out to be problems in containers/storage.