Does it work with gVisor?

[azazar]

I've set the environment variable UDOCKER_USE_RUNC_EXECUTABLE to point to runsc installed at $HOME/.local/bin/runsc, following the gVisor installation guide (with altered installation path). However, this change doesn't appear to take effect. Could someone guide me on what might be incorrect with my setup? Thank you.

[azazar]

Figured out that I should use UDOCKER_DEFAULT_EXECUTION_MODE=R1. But it still doesn't work.

$ udocker run --rm hello-world
Info: downloading layer sha256:d2c94e258dcb3c5ac2798d32e1249e42ef01cba4841c2234249495f87264ac5a
Info: downloading layer sha256:c1ec31eb59444d78df06a974d155e597c894ab4cda84f08294145e845394988e
Warning: check container content: 7d08cc80-f1d5-3d59-ab1a-48fd3b99a344

[jorge-lip]

The runsc CLI is different enough to create issues with udocker, there is a fix in the devel3 branch. Notice that runsc only runs in old 4.x kernels and seems to have still many issues.

[azazar]

It still fails, but now it's probably a gVisor issue.

$ UDOCKER_USE_RUNC_EXECUTABLE=/usr/local/bin/runsc UDOCKER_DEFAULT_EXECUTION_MODE=R1 udocker run --rm -t busybox

 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 %                                                                            %
 %               STARTING 724e99a6-84d6-3c68-b8a3-e6de04dcbd68                %
 %                                                                            %
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 executing: sh
running container: creating container: cannot create gofer process: gofer: fork/exec /proc/self/exe: operation not permitted: check whether /proc/sys/user/max_user_namespaces is set too low (gvisor.dev/issue/5964)

[azazar]

Or maybe it's still an udocker issue. Because gVisor seems to work fine.

$ cat > test.sh
mkdir -p /tmp/runsc
sudo runsc --debug --debug-log=/tmp/runsc/ do echo 123
sudo runsc --debug --debug-log=/tmp/runsc/ --TESTONLY-unsafe-nonroot do echo 123
runsc --debug --debug-log=/tmp/runsc/ --rootless --network=none do echo 123

$ bash test.sh
123
123
123

[jorge-lip]

It seems that with certain kernels and/or distributions runsc will break when the user namespace is present in the spec. Please try again with the latest fix in the Devel3 branch.

UDOCKER_USE_RUNC_EXECUTABLE=/home/jorge/.udocker/bin/runsc-x86_64 UDOCKER_DEFAULT_EXECUTION_MODE=R1 udocker run --rm -t busybox

[azazar]

It works. Thanks!

❯ UDOCKER_USE_RUNC_EXECUTABLE=/usr/local/bin/runsc  UDOCKER_DEFAULT_EXECUTION_MODE=R1  udocker run --rm -t busybox

 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 %                                                                            %
 %               STARTING e315d77b-a3a6-3318-80d7-457ff2752dee                %
 %                                                                            %
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 executing: sh
e315d77b#echo 123
123

[azazar]

It's broken again.

$ udocker run --rm hello-world
Traceback (most recent call last):
  File "maincmd.py", line 58, in <module>
    main()
  File "maincmd.py", line 36, in main
    exit_status = UMain(sys.argv).execute()
  File "/home/username/.opt/udocker-devel3/udocker/../udocker/umain.py", line 108, in execute
    exit_status = cmds[command](self.cmdp)  # executes command
  File "/home/username/.opt/udocker-devel3/udocker/../udocker/cli.py", line 794, in do_run
    if not container_id:
UnboundLocalError: local variable 'container_id' referenced before assignment

$ env | grep UDOCKER_
UDOCKER_DEFAULT_EXECUTION_MODE=R1
UDOCKER_USE_RUNC_EXECUTABLE=/home/username/.local/bin/runsc

[jorge-lip]

Should be ok now. Thanks

On Thu, 2024-03-28 at 09:57 -0700, azazar wrote:

udocker run --rm hello-world

[azazar]

It fails to start in a docker container with rhel 8 running.

[user@d54bb4983a80 ~]$ env|grep UDOCKER
UDOCKER_DEFAULT_EXECUTION_MODE=R1
UDOCKER_USE_RUNC_EXECUTABLE=/home/user/.local/bin/runsc
[user@d54bb4983a80 ~]$ udocker run --rm -i -t -v $HOME:/home/host debian

 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 %                                                                            %
 %               STARTING 3e7bec78-f1e5-394b-9665-c830723dabdc                %
 %                                                                            %
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 executing: bash
Error executing inside namespace: re-executing self: fork/exec /proc/self/exe: operation not permitted

[azazar]

I've tried to create a special docker image for reproducing that issue easier and found one more.

Problem

$ docker run --rm -it udocker-test
[user@fb6b7f2125ec ~]$ udocker run --rm hello-world
Warning: check container content: 702ca086-50aa-36ff-a5d0-a7cd946d7fd9

 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 %                                                                            %
 %               STARTING 702ca086-50aa-36ff-a5d0-a7cd946d7fd9                %
 %                                                                            %
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 executing: hello
flag provided but not defined: -ignore-cgroups
Usage: runsc <flags> <subcommand> <subcommand args>

Subcommands:
    checkpoint       checkpoint current state of container (experimental)
    create           create a secure container
    delete           delete resources held by a container
    do               Simplistic way to execute a command inside the sandbox. It's to be used for testing only.
    events           display container events such as OOM notifications, cpu, memory, and IO usage statistics
    exec             execute new process inside the container
    flags            describe all known top-level flags
    gofer            launch a gofer process that serves files over 9P protocol (internal use only)
    help             Print help documentation.
    kill             sends a signal to the container
    list             list containers started by runsc with the given root
    pause            pause suspends all processes in a container
    ps               ps displays the processes running inside a container
    restore          restore a saved state of container (experimental)
    resume           Resume unpauses a paused container
    run              create and run a secure container
    spec             create a new OCI bundle specification file
    start            start a secure container
    state            get the state of a container
    wait             wait on a process inside a container

Subcommands for helpers:
    install          adds a runtime to docker daemon configuration
    uninstall        removes a runtime from docker daemon configuration

Subcommands for internal use only:
    boot             launch a sandbox process (internal use only)
    debug            shows a variety of debug information
    gofer            launch a gofer process that serves files over 9P protocol (internal use only)
    state            shows information about a statefile

Use "runsc flags" for a list of top-level flags

Dockerfile

FROM redhat/ubi8

RUN yum -y install git python3.12-pip

RUN /usr/bin/python3.12 -m pip install pipx

RUN adduser user

WORKDIR /home/user

USER user

RUN pipx install udocker

RUN mkdir -p .local/bin \
 && cd .local/bin \
 && curl https://storage.googleapis.com/gvisor/releases/release/latest/runsc > runsc \
 && curl https://storage.googleapis.com/gvisor/releases/release/latest/containerd-shim-runsc-v1 > containerd-shim-runsc-v1 \
 && chmod a+rx runsc containerd-shim-runsc-v1

ENV UDOCKER_USE_RUNC_EXECUTABLE=/home/user/.local/bin/runsc
ENV UDOCKER_DEFAULT_EXECUTION_MODE=R1

RUN .local/bin/udocker pull hello-world

CMD [ "/bin/bash" ]