search

containers / podman

Podman: A tool for managing OCI containers and pods.
https://podman.io
Apache License 2.0
23.74k stars 2.41k forks source link

[Kubic] podman run 1.9.0 fails on Ubuntu 19.10 [workaround] #5891

Closed GeoffWilliams closed 4 years ago

GeoffWilliams commented 4 years ago

/kind bug

podman run fails on Ubuntu 19.10 with The following syscalls will be blocked by seccomp:

After upgrading from podman 1.8.2 today, I'm unable to run any containers.

Steps to reproduce the issue:

  1. Install podman on (X)Ubuntu 19.10 following the instructions from https://podman.io/getting-started/installation. I'm using the OpenSUSE/Kubic Project debs. Error occurs on both stable and testing channels

  2. Run a docker image

geoff@computer:~$ podman run -ti alpine
Error: /usr/bin/slirp4netns failed: "sent tapfd=7 for tap0\nWARNING: Support for sandboxing is experimental\nWARNING: Support for seccomp is experimental\nreceived tapfd=7\nenable_seccomp failed\ndo_slirp is exiting\ndo_slirp failed\nparent failed\nWARNING: Support for sandboxing is experimental\nWARNING: Support for seccomp is experimental\nStarting slirp\n* MTU:             65520\n* Network:         10.0.2.0\n* Netmask:         255.255.255.0\n* Gateway:         10.0.2.2\n* DNS:             10.0.2.3\n* Recommended IP:  10.0.2.100\nseccomp: The following syscalls will be blocked by seccomp:"

Describe the results you received:

Fails to run, message about syscalls blocked by seccomp

Describe the results you expected:

Expected container to start. This was the case with the previous version of podman. The repository seems to purge older version so I can't easily downgrade.

Output of podman version:

geoff@computer:~$ podman version
Version:            1.9.0
RemoteAPI Version:  1
Go Version:         go1.12.10
OS/Arch:            linux/amd64

Output of podman info --debug:

geoff@computer:~$ podman info --debug
debug:
  compiler: gc
  gitCommit: ""
  goVersion: go1.12.10
  podmanVersion: 1.9.0
host:
  arch: amd64
  buildahVersion: 1.14.8
  cgroupVersion: v1
  conmon:
    package: 'conmon: /usr/libexec/podman/conmon'
    path: /usr/libexec/podman/conmon
    version: 'conmon version 2.0.15, commit: '
  cpus: 8
  distribution:
    distribution: ubuntu
    version: "19.10"
  eventLogger: journald
  hostname: computer
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.3.0-46-generic
  memFree: 24361922560
  memTotal: 33523597312
  ociRuntime:
    name: runc
    package: 'cri-o-runc: /usr/lib/cri-o-runc/sbin/runc'
    path: /usr/lib/cri-o-runc/sbin/runc
    version: 'runc version spec: 1.0.1-dev'
  os: linux
  rootless: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: 'slirp4netns: /usr/bin/slirp4netns'
    version: |-
      slirp4netns version 0.4.3
      commit: unknown
  swapFree: 68719472640
  swapTotal: 68719472640
  uptime: 1h 26m 43.67s (Approximately 0.04 days)
registries:
  search:
  - docker.io
  - quay.io
store:
  configFile: /home/geoff/.config/containers/storage.conf
  containerStore:
    number: 31
    paused: 0
    running: 1
    stopped: 30
  graphDriverName: vfs
  graphOptions: {}
  graphRoot: /home/geoff/.local/share/containers/storage
  graphStatus: {}
  imageStore:
    number: 22
  runRoot: /run/user/1000/containers
  volumePath: /home/geoff/.local/share/containers/storage/volumes

Package info (e.g. output of rpm -q podman or apt list podman):

geoff@computer:~$ apt list podman
Listing... Done
podman/now 1.9.0~3 amd64 [installed,local]

Additional environment details (AWS, VirtualBox, physical, etc.):

This is a regular Xubuntu 19.10 linux workstation running on a laptop. Prevous podman version 1.8.2 worked with no issue:

Start-Date: 2020-04-20  08:01:07
Commandline: apt upgrade
Requested-By: geoff (1000)
Upgrade: podman:amd64 (1.8.2~144, 1.9.0~2)
End-Date: 2020-04-20  08:01:15

Same command in debug mode:


geoff@computer:~$ podman run  --log-level debug  -ti alpine
DEBU[0000] Using conmon: "/usr/libexec/podman/conmon"   
DEBU[0000] Initializing boltdb state at /home/geoff/.local/share/containers/storage/libpod/bolt_state.db 
DEBU[0000] Using graph driver vfs                       
DEBU[0000] Using graph root /home/geoff/.local/share/containers/storage 
DEBU[0000] Using run root /run/user/1000/containers     
DEBU[0000] Using static dir /home/geoff/.local/share/containers/storage/libpod 
DEBU[0000] Using tmp dir /run/user/1000/libpod/tmp      
DEBU[0000] Using volume path /home/geoff/.local/share/containers/storage/volumes 
DEBU[0000] Set libpod namespace to ""                   
DEBU[0000] No store required. Not opening container store. 
DEBU[0000] Initializing event backend journald          
DEBU[0000] using runtime "/usr/lib/cri-o-runc/sbin/runc" 
WARN[0000] Error initializing configured OCI runtime crun: no valid executable found for OCI runtime crun: invalid argument 
WARN[0000] Error initializing configured OCI runtime kata: no valid executable found for OCI runtime kata: invalid argument 
INFO[0000] running as rootless                          
DEBU[0000] Using conmon: "/usr/libexec/podman/conmon"   
DEBU[0000] Initializing boltdb state at /home/geoff/.local/share/containers/storage/libpod/bolt_state.db 
DEBU[0000] Using graph driver vfs                       
DEBU[0000] Using graph root /home/geoff/.local/share/containers/storage 
DEBU[0000] Using run root /run/user/1000/containers     
DEBU[0000] Using static dir /home/geoff/.local/share/containers/storage/libpod 
DEBU[0000] Using tmp dir /run/user/1000/libpod/tmp      
DEBU[0000] Using volume path /home/geoff/.local/share/containers/storage/volumes 
DEBU[0000] Set libpod namespace to ""                   
DEBU[0000] [graphdriver] trying provided driver "vfs"   
DEBU[0000] Initializing event backend journald          
WARN[0000] Error initializing configured OCI runtime crun: no valid executable found for OCI runtime crun: invalid argument 
WARN[0000] Error initializing configured OCI runtime kata: no valid executable found for OCI runtime kata: invalid argument 
DEBU[0000] using runtime "/usr/lib/cri-o-runc/sbin/runc" 
DEBU[0000] parsed reference into "[vfs@/home/geoff/.local/share/containers/storage+/run/user/1000/containers]docker.io/library/alpine:latest" 
DEBU[0000] parsed reference into "[vfs@/home/geoff/.local/share/containers/storage+/run/user/1000/containers]@a187dde48cd289ac374ad8539930628314bc581a481cdb41409c9289419ddb72" 
DEBU[0000] exporting opaque data as blob "sha256:a187dde48cd289ac374ad8539930628314bc581a481cdb41409c9289419ddb72" 
DEBU[0000] Using slirp4netns netmode                    
DEBU[0000] No hostname set; container's hostname will default to runtime default 
DEBU[0000] Loading seccomp profile from "/usr/share/containers/seccomp.json" 
DEBU[0000] created OCI spec and options for new container 
DEBU[0000] Allocated lock 63 for container 2d1337b305adc490d4ad646978362e6c89cbaca1b4204ede9fa33603cc582de3 
DEBU[0000] parsed reference into "[vfs@/home/geoff/.local/share/containers/storage+/run/user/1000/containers]@a187dde48cd289ac374ad8539930628314bc581a481cdb41409c9289419ddb72" 
DEBU[0000] exporting opaque data as blob "sha256:a187dde48cd289ac374ad8539930628314bc581a481cdb41409c9289419ddb72" 
DEBU[0000] created container "2d1337b305adc490d4ad646978362e6c89cbaca1b4204ede9fa33603cc582de3" 
DEBU[0000] container "2d1337b305adc490d4ad646978362e6c89cbaca1b4204ede9fa33603cc582de3" has work directory "/home/geoff/.local/share/containers/storage/vfs-containers/2d1337b305adc490d4ad646978362e6c89cbaca1b4204ede9fa33603cc582de3/userdata" 
DEBU[0000] container "2d1337b305adc490d4ad646978362e6c89cbaca1b4204ede9fa33603cc582de3" has run directory "/run/user/1000/containers/vfs-containers/2d1337b305adc490d4ad646978362e6c89cbaca1b4204ede9fa33603cc582de3/userdata" 
DEBU[0000] New container created "2d1337b305adc490d4ad646978362e6c89cbaca1b4204ede9fa33603cc582de3" 
DEBU[0000] container "2d1337b305adc490d4ad646978362e6c89cbaca1b4204ede9fa33603cc582de3" has CgroupParent "/libpod_parent/libpod-2d1337b305adc490d4ad646978362e6c89cbaca1b4204ede9fa33603cc582de3" 
DEBU[0000] Handling terminal attach                     
DEBU[0000] mounted container "2d1337b305adc490d4ad646978362e6c89cbaca1b4204ede9fa33603cc582de3" at "/home/geoff/.local/share/containers/storage/vfs/dir/394115d1597cd54b40fa095f687ef61b8267fd6df58d95a8142d528445defa2b" 
DEBU[0000] Made network namespace at /run/user/1000/netns/cni-68bbc104-3f6a-4cde-13da-6e654be68d27 for container 2d1337b305adc490d4ad646978362e6c89cbaca1b4204ede9fa33603cc582de3 
DEBU[0000] Created root filesystem for container 2d1337b305adc490d4ad646978362e6c89cbaca1b4204ede9fa33603cc582de3 at /home/geoff/.local/share/containers/storage/vfs/dir/394115d1597cd54b40fa095f687ef61b8267fd6df58d95a8142d528445defa2b 
DEBU[0000] slirp4netns command: /usr/bin/slirp4netns --disable-host-loopback --mtu 65520 --enable-sandbox --enable-seccomp -c -e 3 -r 4 --netns-type=path /run/user/1000/netns/cni-68bbc104-3f6a-4cde-13da-6e654be68d27 tap0 
DEBU[0001] unmounted container "2d1337b305adc490d4ad646978362e6c89cbaca1b4204ede9fa33603cc582de3" 
DEBU[0001] Tearing down network namespace at /run/user/1000/netns/cni-68bbc104-3f6a-4cde-13da-6e654be68d27 for container 2d1337b305adc490d4ad646978362e6c89cbaca1b4204ede9fa33603cc582de3 
DEBU[0001] Cleaning up container 2d1337b305adc490d4ad646978362e6c89cbaca1b4204ede9fa33603cc582de3 
DEBU[0001] Network is already cleaned up, skipping...   
DEBU[0001] Container 2d1337b305adc490d4ad646978362e6c89cbaca1b4204ede9fa33603cc582de3 storage is already unmounted, skipping... 
DEBU[0001] ExitCode msg: "/usr/bin/slirp4netns failed: \"sent tapfd=7 for tap0\\nwarning: support for sandboxing is experimental\\nwarning: support for seccomp is experimental\\nreceived tapfd=7\\nenable_seccomp failed\\ndo_slirp is exiting\\ndo_slirp failed\\nparent failed\\nwarning: support for sandboxing is experimental\\nwarning: support for seccomp is experimental\\nstarting slirp\\n* mtu:             65520\\n* network:         10.0.2.0\\n* netmask:         255.255.255.0\\n* gateway:         10.0.2.2\\n* dns:             10.0.2.3\\n* recommended ip:  10.0.2.100\\nseccomp: the following syscalls will be blocked by seccomp:\"" 
ERRO[0001] /usr/bin/slirp4netns failed: "sent tapfd=7 for tap0\nWARNING: Support for sandboxing is experimental\nWARNING: Support for seccomp is experimental\nreceived tapfd=7\nenable_seccomp failed\ndo_slirp is exiting\ndo_slirp failed\nparent failed\nWARNING: Support for sandboxing is experimental\nWARNING: Support for seccomp is experimental\nStarting slirp\n* MTU:             65520\n* Network:         10.0.2.0\n* Netmask:         255.255.255.0\n* Gateway:         10.0.2.2\n* DNS:             10.0.2.3\n* Recommended IP:  10.0.2.100\nseccomp: The following syscalls will be blocked by seccomp:"

Workaround In an unrelated issue I found a workaround: Run with --network=host, eg:

geoff@computer:~$ podman run  --network=host  -ti alpine
/ #

works

aleks-mariusz commented 4 years ago

same issue, CentOS 7.7.1908, podman updated from RPM (from the devel_kubic_libcontainers_stable repo):

$ podman restart squidproxy
Error: /usr/bin/slirp4netns failed: "sent tapfd=7 for tap0\nWARNING: Support for sandboxing is experimental\nWARNING: Support for seccomp is experimental\nreceived tapfd=7\ncannot remount / as read-only\nenable_seccomp failed\ndo_slirp is exiting\ndo_slirp failed\nparent failed\nWARNING: Support for sandboxing is experimental\nWARNING: Support for seccomp is experimental\nStarting slirp\n* MTU:             65520\n* Network:         10.0.2.0\n* Netmask:         255.255.255.0\n* Gateway:         10.0.2.2\n* DNS:             10.0.2.3\n* Recommended IP:  10.0.2.100\nseccomp: The following syscalls will be blocked by seccomp:"

(starting a new copy of the container has same issue)

Version:            1.9.0
RemoteAPI Version:  1
Go Version:         go1.13.6
OS/Arch:            linux/amd64

since using --network=host works for @GeoffWilliams , it's likely an issue with slirp4netns setting up rootless networking.. re that utility, i am running version 0.4.3 (from slirp4netns-0.4.3-23.8.x86_64 rpm (from the SuSE tumbleweed repo to work-around this issue), build date Wed Mar 18 23:16:43 2020).

aleks-mariusz commented 4 years ago

perusing issues in the slirp4netns project: perhaps a possibly cross-reference to https://github.com/rootless-containers/slirp4netns/issues/192 ?

A different work-around i came up with that i'd rather use instead (which doesn't require you to attach your container to the host networking), is creating a wrapper script for slirp4netns, which re-disables seccomp (with the standard caveats/disclaimers when it comes to disabling any kind of security feature):

sudo mv /usr/bin/slirp4netns{,.real}
echo -e '#!/usr/bin/env bash\n\nexec /usr/bin/slirp4netns.real ${*/--enable-seccomp/}' | sudo tee -a /usr/bin/slirp4netns >/dev/null
sudo chmod +x /usr/bin/slirp4netns

This would need to be undone whenever you update/re-install the slirp4netns package

Update: this workaround is not recommended as a panacea, as it does not seem to work on all my CentOS systems, i get a cgroup namespaces aren't enabled in the kernel: OCI runtime error error on other systems, when trying to not enable seccomp (perhaps seccomp is related to this check failing).

mheon commented 4 years ago

@giuseppe @AkihiroSuda Sounds like a slirp issue here - PTAL

AkihiroSuda commented 4 years ago

Can't reproduce the issue with Podman v1.9.0 + slirp4netns v1.0.0 + libseccomp v2.4.1 on Ubuntu 19.10. Does slirp4netns v1.0.0 work?

aleks-mariusz commented 4 years ago

I'll let the OP (@GeoffWilliams ) comment if that combination works for him on Ubuntu, however that combination does not work for me (on CentOS 7.7):

$ podman version
Version:            1.9.0
RemoteAPI Version:  1
Go Version:         go1.13.6
OS/Arch:            linux/amd64

latest slirp4netns binary sourced from here the releases page of the project 

$ slirp4netns --version
slirp4netns version 1.0.0
commit: a3be729152a33e692cd28b52f664defbf2e7810a
libslirp: 4.2.0

the official centos repo only provided libseccomp 2.3.1, i sourced 2.4.1 from here. 

$ rpm -qi libseccomp
Name        : libseccomp
Version     : 2.4.1
Release     : 0.el7
Architecture: x86_64
Install Date: Mon Apr 20 13:45:54 2020
Group       : Unspecified
Size        : 333624
License     : LGPLv2
Signature   : (none)
Source RPM  : libseccomp-2.4.1-0.el7.src.rpm
Build Date  : Sat Aug 17 19:36:02 2019
Build Host  : c1bd.rdu2.centos.org
Relocations : (not relocatable)
Packager    : CBS <cbs@centos.org>
Vendor      : CentOS
URL         : https://github.com/seccomp/libseccomp
Summary     : Enhanced seccomp library
Description :
The libseccomp library provides an easy to use interface to the Linux Kernel's
syscall filtering mechanism, seccomp.  The libseccomp API allows an application
to specify which syscalls, and optionally which syscall arguments, the
application is allowed to execute, all of which are enforced by the Linux
Kernel.
error about seccomp still happens ``` $ 'podman' 'run' --log-level debug '-d' '--name' 'squidproxy' '-p' '127.0.0.1:8787:3128' 'docker.io/mendlik/squid:latest' DEBU[0000] Found deprecated file /home/cynikal/.config/containers/libpod.conf, please remove. Use /home/cynikal/.config/.config/containers/containers.conf to override defaults. DEBU[0000] Reading configuration file "/home/cynikal/.config/containers/libpod.conf" DEBU[0000] Using conmon: "/usr/bin/conmon" DEBU[0000] Initializing boltdb state at /home/cynikal/.local/share/containers/storage/libpod/bolt_state.db DEBU[0000] Using graph driver vfs DEBU[0000] Using graph root /home/cynikal/.local/share/containers/storage DEBU[0000] Using run root /run/user/1000 DEBU[0000] Using static dir /home/cynikal/.local/share/containers/storage/libpod DEBU[0000] Using tmp dir /run/user/1000/libpod/tmp DEBU[0000] Using volume path /home/cynikal/.local/share/containers/storage/volumes DEBU[0000] Set libpod namespace to "" DEBU[0000] No store required. Not opening container store. DEBU[0000] Initializing event backend journald DEBU[0000] using runtime "/usr/bin/runc" WARN[0000] Error initializing configured OCI runtime crun: no valid executable found for OCI runtime crun: invalid argument WARN[0000] Error initializing configured OCI runtime kata: no valid executable found for OCI runtime kata: invalid argument DEBU[0000] Failed to add podman to systemd sandbox cgroup: exec: "dbus-launch": executable file not found in $PATH INFO[0000] running as rootless DEBU[0000] Found deprecated file /home/cynikal/.config/containers/libpod.conf, please remove. Use /home/cynikal/.config/.config/containers/containers.conf to override defaults. DEBU[0000] Reading configuration file "/home/cynikal/.config/containers/libpod.conf" DEBU[0000] Using conmon: "/usr/bin/conmon" DEBU[0000] Initializing boltdb state at /home/cynikal/.local/share/containers/storage/libpod/bolt_state.db DEBU[0000] Using graph driver vfs DEBU[0000] Using graph root /home/cynikal/.local/share/containers/storage DEBU[0000] Using run root /run/user/1000 DEBU[0000] Using static dir /home/cynikal/.local/share/containers/storage/libpod DEBU[0000] Using tmp dir /run/user/1000/libpod/tmp DEBU[0000] Using volume path /home/cynikal/.local/share/containers/storage/volumes DEBU[0000] Set libpod namespace to "" DEBU[0000] [graphdriver] trying provided driver "vfs" DEBU[0000] Initializing event backend journald WARN[0000] Error initializing configured OCI runtime kata: no valid executable found for OCI runtime kata: invalid argument DEBU[0000] using runtime "/usr/bin/runc" WARN[0000] Error initializing configured OCI runtime crun: no valid executable found for OCI runtime crun: invalid argument DEBU[0000] parsed reference into "[vfs@/home/cynikal/.local/share/containers/storage+/run/user/1000]docker.io/mendlik/squid:latest" DEBU[0000] parsed reference into "[vfs@/home/cynikal/.local/share/containers/storage+/run/user/1000]@a5a345bf9d8004f748e75e047cd4d8b8d050cd8b3a0047ee1182e760abad9650" DEBU[0000] exporting opaque data as blob "sha256:a5a345bf9d8004f748e75e047cd4d8b8d050cd8b3a0047ee1182e760abad9650" DEBU[0000] Using slirp4netns netmode DEBU[0000] No hostname set; container's hostname will default to runtime default DEBU[0000] Loading seccomp profile from "/usr/share/containers/seccomp.json" DEBU[0000] setting container name squidproxy DEBU[0000] created OCI spec and options for new container DEBU[0000] Allocated lock 1 for container 88e03d0228c9234d05a82a8be61ae327a334459e89c4d48a40411cc0fbf1d8ce DEBU[0000] parsed reference into "[vfs@/home/cynikal/.local/share/containers/storage+/run/user/1000]@a5a345bf9d8004f748e75e047cd4d8b8d050cd8b3a0047ee1182e760abad9650" DEBU[0000] exporting opaque data as blob "sha256:a5a345bf9d8004f748e75e047cd4d8b8d050cd8b3a0047ee1182e760abad9650" DEBU[0053] created container "88e03d0228c9234d05a82a8be61ae327a334459e89c4d48a40411cc0fbf1d8ce" DEBU[0053] container "88e03d0228c9234d05a82a8be61ae327a334459e89c4d48a40411cc0fbf1d8ce" has work directory "/home/cynikal/.local/share/containers/storage/vfs-containers/88e03d0228c9234d05a82a8be61ae327a334459e89c4d48a40411cc0fbf1d8ce/userdata" DEBU[0053] container "88e03d0228c9234d05a82a8be61ae327a334459e89c4d48a40411cc0fbf1d8ce" has run directory "/run/user/1000/vfs-containers/88e03d0228c9234d05a82a8be61ae327a334459e89c4d48a40411cc0fbf1d8ce/userdata" DEBU[0053] New container created "88e03d0228c9234d05a82a8be61ae327a334459e89c4d48a40411cc0fbf1d8ce" DEBU[0053] container "88e03d0228c9234d05a82a8be61ae327a334459e89c4d48a40411cc0fbf1d8ce" has CgroupParent "/libpod_parent/libpod-88e03d0228c9234d05a82a8be61ae327a334459e89c4d48a40411cc0fbf1d8ce" DEBU[0053] Made network namespace at /run/user/1000/netns/cni-91153c0e-8e65-fdfe-19f4-611618e950b8 for container 88e03d0228c9234d05a82a8be61ae327a334459e89c4d48a40411cc0fbf1d8ce DEBU[0053] slirp4netns command: /usr/bin/slirp4netns --disable-host-loopback --mtu 65520 --enable-sandbox --enable-seccomp -c -e 3 -r 4 --netns-type=path /run/user/1000/netns/cni-91153c0e-8e65-fdfe-19f4-611618e950b8 tap0 DEBU[0053] mounted container "88e03d0228c9234d05a82a8be61ae327a334459e89c4d48a40411cc0fbf1d8ce" at "/home/cynikal/.local/share/containers/storage/vfs/dir/163ee1d7088a1171e5e344ae8f2642cb87282ba07f4941e0ffc002b92e2c83bc" DEBU[0054] Created root filesystem for container 88e03d0228c9234d05a82a8be61ae327a334459e89c4d48a40411cc0fbf1d8ce at /home/cynikal/.local/share/containers/storage/vfs/dir/163ee1d7088a1171e5e344ae8f2642cb87282ba07f4941e0ffc002b92e2c83bc DEBU[0054] unmounted container "88e03d0228c9234d05a82a8be61ae327a334459e89c4d48a40411cc0fbf1d8ce" DEBU[0054] Tearing down network namespace at /run/user/1000/netns/cni-91153c0e-8e65-fdfe-19f4-611618e950b8 for container 88e03d0228c9234d05a82a8be61ae327a334459e89c4d48a40411cc0fbf1d8ce DEBU[0054] Cleaning up container 88e03d0228c9234d05a82a8be61ae327a334459e89c4d48a40411cc0fbf1d8ce DEBU[0054] Network is already cleaned up, skipping... DEBU[0054] Container 88e03d0228c9234d05a82a8be61ae327a334459e89c4d48a40411cc0fbf1d8ce storage is already unmounted, skipping... DEBU[0054] ExitCode msg: "/usr/bin/slirp4netns failed: \"warning: support for seccomp is experimental\\nsent tapfd=7 for tap0\\nreceived tapfd=7\\ncannot remount / as read-only\\nenable_seccomp failed\\ndo_slirp is exiting\\ndo_slirp failed\\nparent failed\\nstarting slirp\\n* mtu: 65520\\n* network: 10.0.2.0\\n* netmask: 255.255.255.0\\n* gateway: 10.0.2.2\\n* dns: 10.0.2.3\\n* recommended ip: 10.0.2.100\\nseccomp: the following syscalls will be blocked by seccomp:\"" ERRO[0054] /usr/bin/slirp4netns failed: "WARNING: Support for seccomp is experimental\nsent tapfd=7 for tap0\nreceived tapfd=7\ncannot remount / as read-only\nenable_seccomp failed\ndo_slirp is exiting\ndo_slirp failed\nparent failed\nStarting slirp\n* MTU: 65520\n* Network: 10.0.2.0\n* Netmask: 255.255.255.0\n* Gateway: 10.0.2.2\n* DNS: 10.0.2.3\n* Recommended IP: 10.0.2.100\nseccomp: The following syscalls will be blocked by seccomp:" └>retc:126 ```
aleks-mariusz commented 4 years ago

What's unusual is that, containers i created before with the older podman 1.8.2 can still be re-started, however i can't start new ones.. So to me this seems like there's something with the container setup process related to enabling seccomp by default now.

Is there anyway to stop from having seccomp always be enabled (probably because of this PR ) please ?? this is a pretty bad issue as podman is useless for launching containers now :-(

aleks-mariusz commented 4 years ago

Can't reproduce the issue with Podman v1.9.0 + slirp4netns v1.0.0 + libseccomp v2.4.1 on Ubuntu 19.10. Does slirp4netns v1.0.0 work?

@AkihiroSuda can you please also try on centos 7 system please? A lot of people are setting up podman via a repo from the opensuse kubic project (especially considering the official podman page recommends using it on the install page ), which now means they all end up with a version combination that is effectively useless for running podman in rootlessmode.

It would be good to help figure where the mismatch of versions is so they can update the package versions in the repo.

here's what the average user experience now looks like, this was done just now on a fresh EC2 instance on amazon.. ``` $ ssh -i .ssh/id_rsa centos@18.132.42.172 The authenticity of host '18.132.42.172 (18.132.42.172)' can't be established. ECDSA key fingerprint is SHA256:FFNEx+beqYiqsOWmjS1+EsaKytuIAw4v0h6T9W6+hlM. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '18.132.42.172' (ECDSA) to the list of known hosts. [centos@ip-172-31-1-30 ~]$ sudo curl -sLo /etc/yum.repos.d/kubic_libcontainers_stable.repo 'http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/CentOS_7/devel:kubic:libcontainers:stable.repo' [centos@ip-172-31-1-30 ~]$ sudo yum install -y podman Loaded plugins: fastestmirror Determining fastest mirrors * base: d36uatko69830t.cloudfront.net * extras: d36uatko69830t.cloudfront.net * updates: d36uatko69830t.cloudfront.net base | 3.6 kB 00:00:00 devel_kubic_libcontainers_stable | 1.3 kB 00:00:00 extras | 2.9 kB 00:00:00 updates | 2.9 kB 00:00:00 (1/5): base/7/x86_64/group_gz | 165 kB 00:00:00 (2/5): extras/7/x86_64/primary_db | 165 kB 00:00:00 (3/5): devel_kubic_libcontainers_stable/primary | 9.9 kB 00:00:00 (4/5): updates/7/x86_64/primary_db | 7.6 MB 00:00:00 (5/5): base/7/x86_64/primary_db | 6.0 MB 00:00:00 devel_kubic_libcontainers_stable 27/27 Resolving Dependencies --> Running transaction check ---> Package podman.x86_64 0:1.9.0-1.1.el7 will be installed --> Processing Dependency: podman-plugins = 1.9.0-1.1.el7 for package: podman-1.9.0-1.1.el7.x86_64 --> Processing Dependency: slirp4netns >= 0.3.0-2 for package: podman-1.9.0-1.1.el7.x86_64 --> Processing Dependency: containernetworking-plugins >= 0.7.5-1 for package: podman-1.9.0-1.1.el7.x86_64 --> Processing Dependency: runc for package: podman-1.9.0-1.1.el7.x86_64 --> Processing Dependency: nftables for package: podman-1.9.0-1.1.el7.x86_64 --> Processing Dependency: containers-common for package: podman-1.9.0-1.1.el7.x86_64 --> Processing Dependency: container-selinux for package: podman-1.9.0-1.1.el7.x86_64 --> Processing Dependency: conmon for package: podman-1.9.0-1.1.el7.x86_64 --> Running transaction check ---> Package criu.x86_64 0:3.12-2.el7 will be installed --> Processing Dependency: libprotobuf-c.so.1(LIBPROTOBUF_C_1.0.0)(64bit) for package: criu-3.12-2.el7.x86_64 --> Processing Dependency: libprotobuf-c.so.1()(64bit) for package: criu-3.12-2.el7.x86_64 --> Processing Dependency: libnet.so.1()(64bit) for package: criu-3.12-2.el7.x86_64 ---> Package libnftnl.x86_64 0:1.0.8-1.el7 will be installed --> Running transaction check ---> Package libnet.x86_64 0:1.1.6-7.el7 will be installed ---> Package protobuf-c.x86_64 0:1.0.2-3.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================================================================================================================================================================================================= Package Arch Version Repository Size ================================================================================================================================================================================================================================================================= Installing: podman x86_64 1.9.0-1.1.el7 devel_kubic_libcontainers_stable 22 M Installing for dependencies: conmon x86_64 2:2.0.15-2.1.el7 devel_kubic_libcontainers_stable 34 k container-selinux noarch 2:2.107-3.el7 extras 39 k containernetworking-plugins x86_64 0.8.5-145.1.el7 devel_kubic_libcontainers_stable 35 M containers-common x86_64 2:0.2.0-2.1.el7 devel_kubic_libcontainers_stable 53 k criu x86_64 3.12-2.el7 base 453 k libnet x86_64 1.1.6-7.el7 base 59 k libnftnl x86_64 1.0.8-1.el7 base 77 k nftables x86_64 1:0.8-14.el7 base 186 k podman-plugins x86_64 1.9.0-1.1.el7 devel_kubic_libcontainers_stable 2.3 M protobuf-c x86_64 1.0.2-3.el7 base 28 k runc x86_64 2:1.0.0-15.1.el7 devel_kubic_libcontainers_stable 4.5 M slirp4netns x86_64 0.4.3-22.1.el7 devel_kubic_libcontainers_stable 84 k Transaction Summary ================================================================================================================================================================================================================================================================= Install 1 Package (+12 Dependent packages) Total download size: 64 M Installed size: 186 M Downloading packages: warning: /var/cache/yum/x86_64/7/extras/packages/container-selinux-2.107-3.el7.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY Public key for container-selinux-2.107-3.el7.noarch.rpm is not installed (1/13): container-selinux-2.107-3.el7.noarch.rpm | 39 kB 00:00:00 warning: /var/cache/yum/x86_64/7/devel_kubic_libcontainers_stable/packages/conmon-2.0.15-2.1.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 75060aa4: NOKEY ] 0.0 B/s | 39 kB --:--:-- ETA Public key for conmon-2.0.15-2.1.el7.x86_64.rpm is not installed (2/13): conmon-2.0.15-2.1.el7.x86_64.rpm | 34 kB 00:00:00 Public key for criu-3.12-2.el7.x86_64.rpm is not installed (3/13): criu-3.12-2.el7.x86_64.rpm | 453 kB 00:00:00 (4/13): libnet-1.1.6-7.el7.x86_64.rpm | 59 kB 00:00:00 (5/13): containers-common-0.2.0-2.1.el7.x86_64.rpm | 53 kB 00:00:00 (6/13): libnftnl-1.0.8-1.el7.x86_64.rpm | 77 kB 00:00:00 (7/13): nftables-0.8-14.el7.x86_64.rpm | 186 kB 00:00:00 (8/13): podman-1.9.0-1.1.el7.x86_64.rpm | 22 MB 00:00:01 (9/13): containernetworking-plugins-0.8.5-145.1.el7.x86_64.rpm | 35 MB 00:00:02 (10/13): protobuf-c-1.0.2-3.el7.x86_64.rpm | 28 kB 00:00:00 (11/13): podman-plugins-1.9.0-1.1.el7.x86_64.rpm | 2.3 MB 00:00:00 (12/13): runc-1.0.0-15.1.el7.x86_64.rpm | 4.5 MB 00:00:00 (13/13): slirp4netns-0.4.3-22.1.el7.x86_64.rpm | 84 kB 00:00:00 ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 25 MB/s | 64 MB 00:00:02 Retrieving key from http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/CentOS_7/repodata/repomd.xml.key Importing GPG key 0x75060AA4: Userid : "devel:kubic OBS Project " Fingerprint: 2472 d6d0 d2f6 6af8 7aba 8da3 4d64 3903 7506 0aa4 From : http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/CentOS_7/repodata/repomd.xml.key Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 Importing GPG key 0xF4A80EB5: Userid : "CentOS-7 Key (CentOS 7 Official Signing Key) " Fingerprint: 6341 ab27 53d7 8a78 a7c2 7bb1 24c6 a8a7 f4a8 0eb5 Package : centos-release-7-7.1908.0.el7.centos.x86_64 (installed) From : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : 2:container-selinux-2.107-3.el7.noarch 1/13 Installing : libnet-1.1.6-7.el7.x86_64 2/13 Installing : podman-plugins-1.9.0-1.1.el7.x86_64 3/13 Installing : libnftnl-1.0.8-1.el7.x86_64 4/13 Installing : 1:nftables-0.8-14.el7.x86_64 5/13 Installing : slirp4netns-0.4.3-22.1.el7.x86_64 6/13 Installing : containernetworking-plugins-0.8.5-145.1.el7.x86_64 7/13 Installing : 2:containers-common-0.2.0-2.1.el7.x86_64 8/13 Installing : 2:conmon-2.0.15-2.1.el7.x86_64 9/13 Installing : protobuf-c-1.0.2-3.el7.x86_64 10/13 Installing : criu-3.12-2.el7.x86_64 11/13 Installing : 2:runc-1.0.0-15.1.el7.x86_64 12/13 Installing : podman-1.9.0-1.1.el7.x86_64 13/13 Verifying : podman-1.9.0-1.1.el7.x86_64 1/13 Verifying : protobuf-c-1.0.2-3.el7.x86_64 2/13 Verifying : 2:conmon-2.0.15-2.1.el7.x86_64 3/13 Verifying : 2:containers-common-0.2.0-2.1.el7.x86_64 4/13 Verifying : containernetworking-plugins-0.8.5-145.1.el7.x86_64 5/13 Verifying : slirp4netns-0.4.3-22.1.el7.x86_64 6/13 Verifying : 2:container-selinux-2.107-3.el7.noarch 7/13 Verifying : 1:nftables-0.8-14.el7.x86_64 8/13 Verifying : libnftnl-1.0.8-1.el7.x86_64 9/13 Verifying : criu-3.12-2.el7.x86_64 10/13 Verifying : podman-plugins-1.9.0-1.1.el7.x86_64 11/13 Verifying : libnet-1.1.6-7.el7.x86_64 12/13 Verifying : 2:runc-1.0.0-15.1.el7.x86_64 13/13 Installed: podman.x86_64 0:1.9.0-1.1.el7 Dependency Installed: conmon.x86_64 2:2.0.15-2.1.el7 container-selinux.noarch 2:2.107-3.el7 containernetworking-plugins.x86_64 0:0.8.5-145.1.el7 containers-common.x86_64 2:0.2.0-2.1.el7 criu.x86_64 0:3.12-2.el7 libnet.x86_64 0:1.1.6-7.el7 libnftnl.x86_64 0:1.0.8-1.el7 nftables.x86_64 1:0.8-14.el7 podman-plugins.x86_64 0:1.9.0-1.1.el7 protobuf-c.x86_64 0:1.0.2-3.el7 runc.x86_64 2:1.0.0-15.1.el7 slirp4netns.x86_64 0:0.4.3-22.1.el7 Complete! [centos@ip-172-31-1-30 ~]$ echo user.max_user_namespaces=15000 | sudo tee -a /etc/sysctl.d/95-user-namespaces.conf >/dev/null [centos@ip-172-31-1-30 ~]$ sudo sysctl --system >/dev/null [centos@ip-172-31-1-30 ~]$ sysctl user.max_user_namespaces user.max_user_namespaces = 15000 [centos@ip-172-31-1-30 ~]$ podman run -d --name nginx -p 8080:80 docker.io/library/nginx:alpine WARN[0000] The cgroupv2 manager is set to systemd but there is no systemd user session available WARN[0000] For using systemd, you may need to login using an user session WARN[0000] Alternatively, you can enable lingering with: `loginctl enable-linger 1000` (possibly as root) WARN[0000] Falling back to --cgroup-manager=cgroupfs WARN[0000] The cgroupv2 manager is set to systemd but there is no systemd user session available WARN[0000] For using systemd, you may need to login using an user session WARN[0000] Alternatively, you can enable lingering with: `loginctl enable-linger 1000` (possibly as root) WARN[0000] Falling back to --cgroup-manager=cgroupfs Trying to pull docker.io/library/nginx:alpine... Getting image source signatures Copying blob b14da7a62044 done Copying blob aad63a933944 done Copying config 29b49a39bc done Writing manifest to image destination Storing signatures Error: /usr/bin/slirp4netns failed: "sent tapfd=8 for tap0\nWARNING: Support for sandboxing is experimental\nWARNING: Support for seccomp is experimental\nreceived tapfd=8\ncannot mount tmpfs on /tmp\ncreate_sandbox failed\ndo_slirp is exiting\ndo_slirp failed\nparent failed\nWARNING: Support for sandboxing is experimental\nWARNING: Support for seccomp is experimental\nStarting slirp\n* MTU: 65520\n* Network: 10.0.2.0\n* Netmask: 255.255.255.0\n* Gateway: 10.0.2.2\n* DNS: 10.0.2.3\n* Recommended IP: 10.0.2.100\n" [centos@ip-172-31-1-30 ~]$ /usr/bin/slirp4netns --version slirp4netns version 0.4.3-beta.1 commit: b04291ba84ca35ccc60bd009372a28f9ea7ef841 ``` by default we get slirp4netns 0.4.3-beta which is afflicted by [this issue](https://github.com/containers/libpod/issues/5420), so i update it to the latest mentioned: ``` [centos@ip-172-31-1-30 ~]$ podman rm nginx WARN[0000] The cgroupv2 manager is set to systemd but there is no systemd user session available WARN[0000] For using systemd, you may need to login using an user session WARN[0000] Alternatively, you can enable lingering with: `loginctl enable-linger 1000` (possibly as root) WARN[0000] Falling back to --cgroup-manager=cgroupfs caee458aeeeed44f7d5c74fc476bc34a24f6034ceef585f1ee106c427c191a6a [centos@ip-172-31-1-30 ~]$ sudo curl -sLo /usr/bin/slirp4netns https://github.com/rootless-containers/slirp4netns/releases/download/v1.0.0/slirp4netns-x86_64 [centos@ip-172-31-1-30 ~]$ /usr/bin/slirp4netns --version slirp4netns version 1.0.0 commit: a3be729152a33e692cd28b52f664defbf2e7810a libslirp: 4.2.0 [centos@ip-172-31-1-30 ~]$ podman run -d --name nginx -p 8080:80 docker.io/library/nginx:alpine WARN[0000] The cgroupv2 manager is set to systemd but there is no systemd user session available WARN[0000] For using systemd, you may need to login using an user session WARN[0000] Alternatively, you can enable lingering with: `loginctl enable-linger 1000` (possibly as root) WARN[0000] Falling back to --cgroup-manager=cgroupfs Error: /usr/bin/slirp4netns failed: "WARNING: Support for seccomp is experimental\nsent tapfd=7 for tap0\nreceived tapfd=7\ncannot remount / as read-only\nenable_seccomp failed\ndo_slirp is exiting\ndo_slirp failed\nparent failed\nStarting slirp\n* MTU: 65520\n* Network: 10.0.2.0\n* Netmask: 255.255.255.0\n* Gateway: 10.0.2.2\n* DNS: 10.0.2.3\n* Recommended IP: 10.0.2.100\nseccomp: The following syscalls will be blocked by seccomp:" ```
AkihiroSuda commented 4 years ago

On CentOS 7, I can hit the issue with the slirp4netns upstream static binary (https://github.com/rootless-containers/slirp4netns/releases/download/v1.0.0/slirp4netns-x86_64), but the issue doesn't seem to happen when I build slirp4netns on CentOS 7.

AkihiroSuda commented 4 years ago

i.e. This is likely to be version mismatch of libseccomp .

AkihiroSuda commented 4 years ago

@cyphar @saschagrunert Could you update slirp4netns packages on Kubic? Also please make sure to use the latest CentOS 7 for building CentOS 7 RPMs, and the latest Ubuntu 19.10 for Ubuntu 19.10 dpkgs.

saschagrunert commented 4 years ago

@cyphar @saschagrunert Could you update slirp4netns packages on Kubic?

I think @lsm5 is currently maintaining the upstream packages in OBS. Can we somehow update the ubuntu dependencies there?

aleks-mariusz commented 4 years ago

I think lsm5 is currently maintaining the upstream packages in OBS. Can we somehow update the ubuntu dependencies there?

i've pinged him about updating slirp4netns already earlier and he mentioned it should now be updated

lsm5 commented 4 years ago

should be done for all except ubuntu 18.04 where it's giving me some weird error that I haven't quite figured out. Hope that one can wait ..

GeoffWilliams commented 4 years ago

Updated system and now have packages:

I'm still getting the original error though. System is fully up-to-date:

geoff@computer:~$ podman run -ti  alpine
Error: /usr/bin/slirp4netns failed: "sent tapfd=7 for tap0\nWARNING: Support for seccomp is experimental\nreceived tapfd=7\nenable_seccomp failed\ndo_slirp is exiting\ndo_slirp failed\nparent failed\nWARNING: Support for seccomp is experimental\nStarting slirp\n* MTU:             65520\n* Network:         10.0.2.0\n* Netmask:         255.255.255.0\n* Gateway:         10.0.2.2\n* DNS:             10.0.2.3\n* Recommended IP:  10.0.2.100\nseccomp: The following syscalls will be blocked by seccomp:"

Also tried with https://github.com/rootless-containers/slirp4netns/releases/tag/v1.0.0 ~which I tested by placing first on path - same issue~ by replacing the file at /usr/bin/slirp4netns and chmod +xing it and this works

lsm5 commented 4 years ago

Updated system and now have packages:

* libslirp0:amd64 4.2.0~4

* slirp4netns 1.0.0~1

I'm still getting the original error though. System is fully up-to-date:

geoff@computer:~$ podman run -ti  alpine
Error: /usr/bin/slirp4netns failed: "sent tapfd=7 for tap0\nWARNING: Support for seccomp is experimental\nreceived tapfd=7\nenable_seccomp failed\ndo_slirp is exiting\ndo_slirp failed\nparent failed\nWARNING: Support for seccomp is experimental\nStarting slirp\n* MTU:             65520\n* Network:         10.0.2.0\n* Netmask:         255.255.255.0\n* Gateway:         10.0.2.2\n* DNS:             10.0.2.3\n* Recommended IP:  10.0.2.100\nseccomp: The following syscalls will be blocked by seccomp:"

Also tried with https://github.com/rootless-containers/slirp4netns/releases/tag/v1.0.0 ~which I tested by placing first on path - same issue~ by replacing the file at /usr/bin/slirp4netns and chmod +xing it and this works

@AkihiroSuda could this be because the deb package didn't have a static binary? or is it an older seccomp to blame?

lsm5 commented 4 years ago

@giuseppe suggested building slirp4netns without seccomp enabled. I'll try to get those out on the testing repo soon..

giuseppe commented 4 years ago

the packages from https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/CentOS_7/x86_64/ work well for me on CentOS 7.

Please try these commands:

$ unshare -rn sleep 100 &
$ slirp4netns -c --enable-seccomp $! tap0

WARNING: Support for seccomp is experimental
sent tapfd=5 for tap0
received tapfd=5
Starting slirp
* MTU:             1500
* Network:         10.0.2.0
* Netmask:         255.255.255.0
* Gateway:         10.0.2.2
* DNS:             10.0.2.3
* Recommended IP:  10.0.2.100
WARNING: 127.0.0.1:* on the host is accessible as 10.0.2.2 (set --disable-host-loopback to prohibit connecting to 127.0.0.1:*)
seccomp: can't block execevat because __NR_execveat was not defined in the build environment
seccomp: The following syscalls will be blocked by seccomp: execve open_by_handle_at ptrace prctl process_vm_readv process_vm_writev mount name_to_handle_at setns umount umount2 unshare.

Does slirp4netns exit with error?

The seccomp messages are just warnings that you can ignore.

aleks-mariusz commented 4 years ago

the packages from https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/CentOS_7/x86_64/ work well for me on CentOS 7.

the podman 1.9.0 packages from this repo only now finally work cleanly on my CentOS 7 systems, in rootless mode, with the addition of the following /etc/containers/containers.conf contents:

[containers]
cgroupns = "host"

[engine]
cgroup_manager = "cgroupfs"
events_logger = "file"

with the first section to work around this issue and the second section is due to this issue

also while I appreciate the attention for CentOS, @GeoffWilliams 's issue is on Ubuntu 19.10 :-)

Does slirp4netns exit with error?

no, it stays in the foreground.. think that's good.

github-actions[bot] commented 4 years ago

A friendly reminder that this issue had no activity for 30 days.

mheon commented 4 years ago

Can anyone retry this with Podman v1.9.2 or later and report if it's fixed? I suspect that we've caught it already

rhatdan commented 4 years ago

Reopen if this is not fixed.

llebout commented 4 years ago

This issue is happening for me on a fully updated Fedora 32 as of today on ppc64le:

$ podman run -v guix:/src/guix:Z -it registry.gitlab.com/lle-bout/guix:latest /bin/bash
Trying to pull registry.gitlab.com/lle-bout/guix:latest...
Getting image source signatures
Copying blob e7edab824d5f done  
Copying blob fd827a7d77b1 done  
Copying blob b9183a6c55fb done  
Copying config f9db71b17f done  
Writing manifest to image destination
Storing signatures
Error: /usr/bin/slirp4netns failed: "sent tapfd=7 for tap0\nWARNING: Support for seccomp is experimental\nreceived tapfd=7\nseccomp: can't add extra arch (i=0)\nenable_seccomp failed\ndo_slirp is exiting\ndo_slirp failed\nparent failed\nWARNING: Support for seccomp is experimental\nStarting slirp\n* MTU:             65520\n* Network:         10.0.2.0\n* Netmask:         255.255.255.0\n* Gateway:         10.0.2.2\n* DNS:             10.0.2.3\n* Recommended IP:  10.0.2.100\n"

Version of podman is 2.0.1

I note:

seccomp: can't add extra arch (i=0)

but it used to work before on that same machine

twagtwig commented 4 years ago

Me too. I'm running podman on Debian 10 cloud with the newest Kubic package.

vrothberg commented 4 years ago

Please open a new issue for that. The issue here is very different and has been closed many weeks ago.

@giuseppe @AkihiroSuda, you may be interested in it.

llebout commented 4 years ago

@vrothberg Seeing the comments and the fact it was closed after being stale, I wasnt sure whether it was certain it was solved or not. Opened another here: https://github.com/containers/podman/issues/6922