search

containers / podman

Podman: A tool for managing OCI containers and pods.
https://podman.io
Apache License 2.0
23.82k stars 2.42k forks source link

"unable to upgrade to tcp, received 409" when using docker compatibility mode #23774

Open d-m opened 2 months ago

d-m commented 2 months ago

Issue Description

docker run commands result in an "unable to upgrade to tcp, received 409" error. podman run commands work fine.

podman info:

host:
  arch: arm64
  buildahVersion: 1.37.2
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - pids
  - rdma
  - misc
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.10-1.fc40.aarch64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.10, commit: '
  cpuUtilization:
    idlePercent: 99.46
    systemPercent: 0.26
    userPercent: 0.28
  cpus: 6
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: coreos
    version: "40"
  eventLogger: journald
  freeLocks: 2035
  hostname: localhost.localdomain
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 6.9.12-200.fc40.aarch64
  linkmode: dynamic
  logDriver: journald
  memFree: 3200524288
  memTotal: 3794071552
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.12.1-1.20240819115418474394.main.6.gc2cd0be.fc40.aarch64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.13.0-dev
    package: netavark-1.12.1-1.20240819170533312370.main.26.g4358fd3.fc40.aarch64
    path: /usr/libexec/podman/netavark
    version: netavark 1.13.0-dev
  ociRuntime:
    name: crun
    package: crun-1.16-1.20240813143753154884.main.16.g26c7687.fc40.aarch64
    path: /usr/bin/crun
    version: |-
      crun version UNKNOWN
      commit: 158b340ec38e187abee05cbf3f27b40be2b564d0
      rundir: /run/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20240726.g57a21d2-1.fc40.aarch64
    version: |
      pasta 0^20240726.g57a21d2-1.fc40.aarch64-pasta
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  rootlessNetworkCmd: pasta
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.2-2.fc40.aarch64
    version: |-
      slirp4netns version 1.2.2
      commit: 0ee2d87523e906518d34a6b423271e4826f71faf
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.5
  swapFree: 0
  swapTotal: 0
  uptime: 0h 8m 51.00s
  variant: v8
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /usr/share/containers/storage.conf
  containerStore:
    number: 13
    paused: 0
    running: 0
    stopped: 13
  graphDriverName: overlay
  graphOptions:
    overlay.imagestore: /usr/lib/containers/storage
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphRootAllocated: 99252940800
  graphRootUsed: 5610475520
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Supports shifting: "true"
    Supports volatile: "true"
    Using metacopy: "true"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 2
  runRoot: /run/containers/storage
  transientStore: false
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 5.2.2
  Built: 1724198400
  BuiltTime: Tue Aug 20 20:00:00 2024
  GitCommit: ""
  GoVersion: go1.22.6
  Os: linux
  OsArch: linux/arm64
  Version: 5.2.2

podman version

Client:       Podman Engine
Version:      5.2.2
API Version:  5.2.2
Go Version:   go1.23.0
Git Commit:   fcee48106a12dd531702d729d17f40f6e152027f
Built:        Wed Aug 21 13:43:11 2024
OS/Arch:      darwin/arm64

Server:       Podman Engine
Version:      5.2.2
API Version:  5.2.2
Go Version:   go1.22.6
Built:        Tue Aug 20 20:00:00 2024
OS/Arch:      linux/arm64

Steps to reproduce the issue

Steps to reproduce the issue

  1. docker run -it ubuntu -- bash

Describe the results you received

Unable to find image 'ubuntu:latest' locally
9f23a71f1e31: Download complete 
1a799365aa63: Download complete 
unable to upgrade to tcp, received 409

Describe the results you expected

I expected the container to run.

podman info output

If you are unable to run podman info for any reason, please provide the podman version, operating system and its version and the architecture you are running.

Podman in a container

No

Privileged Or Rootless

Privileged

Upstream Latest Release

No

Additional environment details

Additional information

No response

Luap99 commented 2 months ago

Please run podman machine ssh and check the server log with something like journalctl -u podman.service and see an error is logged there when you run the docker command

d-m commented 2 months ago

Here are the logs:

Sep 06 13:21:32 localhost.localdomain podman[1262]: 2024-09-06 13:21:32.455900512 -0400 EDT m=+190.791991175 volume create ab0ca3f3a6ed91dbe03ee7e4127cebfd89fa6058cd35be6f22d2f3096be76eb9
Sep 06 13:21:32 localhost.localdomain podman[1262]: 2024-09-06 13:21:32.457141111 -0400 EDT m=+190.793231816 volume create 74a9ec663cc2fc537241531cfb9b2ddfad70651f89e401a9bf44bc95355915b1
Sep 06 13:21:32 localhost.localdomain podman[1262]: 2024-09-06 13:21:32.45833846 -0400 EDT m=+190.794429164 container create 83ad206abf3454e30f16ca729a1ea01bb377d19a21ec40c307ac78f70f00ecbc (image=ubuntu:latest, name=practical_hawking)
Sep 06 13:21:32 localhost.localdomain podman[1262]: @ - - [06/Sep/2024:13:21:32 -0400] "POST /v1.41/containers/create HTTP/1.1" 201 88 "" "Docker-Client/27.1.1-rd (darwin)"
Sep 06 13:21:32 localhost.localdomain podman[2278]: [INFO  netavark::firewall] Using iptables firewall driver
Sep 06 13:21:32 localhost.localdomain podman[2278]: [INFO  netavark::network::netlink] Adding route (dest: 0.0.0.0/0 ,gw: 10.88.0.1, metric 100)
Sep 06 13:21:32 localhost.localdomain podman[1262]: time="2024-09-06T13:21:32-04:00" level=info msg="Running conmon under slice machine.slice and unitName libpod-conmon-83ad206abf3454e30f16ca729a1ea01bb377d19a21ec40c307ac78f70f00ecbc.scope"
Sep 06 13:21:32 localhost.localdomain podman[1262]: time="2024-09-06T13:21:32-04:00" level=info msg="Request Failed(Conflict): preparing container 83ad206abf3454e30f16ca729a1ea01bb377d19a21ec40c307ac78f70f00ecbc for attach: crun: open executable: Operation not permitted: OCI permission denied"
Sep 06 13:21:32 localhost.localdomain podman[1262]: @ - - [06/Sep/2024:13:21:32 -0400] "POST /v1.41/containers/83ad206abf3454e30f16ca729a1ea01bb377d19a21ec40c307ac78f70f00ecbc/attach?stderr=1&stdin=1&stdout=1&stream=1 HTTP/1.1" 409 228 "" "Docker-Client/27.1.1-rd (darwin)"
Luap99 commented 2 months ago

crun: open executable: Operation not permitted: OCI permission denied

This sounds like your image executable cannot be executed for some reason. Does this happen with all images?