podman inspect returning empty Port bindings when running container inside a rootless podman container.

mvmandapati commented 6 months ago

Issue Description

podman inspect returning empty Port bindings when running container inside a rootless podman container. Container starts fine and running podman ps shows the running status.

Running podman ps --all --format "{{.Names}} {{.Ports}} {{.Mounts}} {{.Status}}" shows running container with missing ports data.

[podman@8f7c7f2d9649 /]$ podman ps --all --format "{{.Names}} {{.Ports}} {{.Mounts}} {{.Status}}"
postgres  [/proc /var/lib/postgresql/data] Up 5 minutes

Steps to reproduce the issue

Start a rootless podman container. podman run --rm -it --security-opt label=disable --user podman quay.io/podman/stable bash
Start a postgres db container in detached mode. podman run -d --rm --name postgres -e POSTGRES_PASSWORD=changeit -p 5432:5432 docker.io/postgres:14

Run podman inspect and verify NetworkSettings.

[podman@8f7c7f2d9649 /]$ podman inspect postgres
[
 {
      .......
      "NetworkSettings": {
           "EndpointID": "",
           "Gateway": "",
           "IPAddress": "",
           "IPPrefixLen": 0,
           "IPv6Gateway": "",
           "GlobalIPv6Address": "",
           "GlobalIPv6PrefixLen": 0,
           "MacAddress": "",
           "Bridge": "",
           "SandboxID": "",
           "HairpinMode": false,
           "LinkLocalIPv6Address": "",
           "LinkLocalIPv6PrefixLen": 0,
           "Ports": {},
           "SandboxKey": "",
           "Networks": {
                "host": {
                     "EndpointID": "",
                     "Gateway": "",
                     "IPAddress": "",
                     "IPPrefixLen": 0,
                     "IPv6Gateway": "",
                     "GlobalIPv6Address": "",
                     "GlobalIPv6PrefixLen": 0,
                     "MacAddress": "",
                     "NetworkID": "host",
                     "DriverOpts": null,
                     "IPAMConfig": null,
                     "Links": null
                }
           }
      },
      .......
 }
]

Run podman port --all which also returns empty response.

Describe the results you received

NetworkSettings > Ports is empty

Describe the results you expected

Expected 5432/tcp port binding result.

podman info output

[podman@d03f68178d25 /]$ podman info
host:
  arch: amd64
  buildahVersion: 1.33.7
  cgroupControllers: []
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: conmon-2.1.10-1.fc39.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.10, commit: '
  cpuUtilization:
    idlePercent: 98.99
    systemPercent: 0.32
    userPercent: 0.69
  cpus: 12
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: container
    version: "39"
  eventLogger: file
  freeLocks: 2043
  hostname: d03f68178d25
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 1
      size: 999
    - container_id: 1000
      host_id: 1001
      size: 64535
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 1
      size: 999
    - container_id: 1000
      host_id: 1001
      size: 64535
  kernel: 5.15.146.1-microsoft-standard-WSL2
  linkmode: dynamic
  logDriver: k8s-file
  memFree: 19336491008
  memTotal: 25204871168
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.10.0-1.fc39.x86_64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.10.0
    package: netavark-1.10.3-1.fc39.x86_64
    path: /usr/libexec/podman/netavark
    version: netavark 1.10.3
  ociRuntime:
    name: crun
    package: crun-1.14.4-1.fc39.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.14.4
      commit: a220ca661ce078f2c37b38c92e66cf66c012d9c1
      rundir: /tmp/podman-run-1000/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20240326.g4988e2b-1.fc39.x86_64
    version: |
      pasta 0^20240326.g4988e2b-1.fc39.x86_64
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: false
    path: /tmp/podman-run-1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.2-1.fc39.x86_64
    version: |-
      slirp4netns version 1.2.2
      commit: 0ee2d87523e906518d34a6b423271e4826f71faf
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.3
  swapFree: 25769803776
  swapTotal: 25769803776
  uptime: 0h 55m 7.00s
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /home/podman/.config/containers/storage.conf
  containerStore:
    number: 4
    paused: 0
    running: 1
    stopped: 3
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/podman/.local/share/containers/storage
  graphRootAllocated: 1081101176832
  graphRootUsed: 101523914752
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 2
  runRoot: /tmp/containers-user-1000/containers
  transientStore: false
  volumePath: /home/podman/.local/share/containers/storage/volumes
version:
  APIVersion: 4.9.4
  Built: 1711445992
  BuiltTime: Tue Mar 26 09:39:52 2024
  GitCommit: ""
  GoVersion: go1.21.8
  Os: linux
  OsArch: linux/amd64
  Version: 4.9.4

Podman in a container

Yes

Privileged Or Rootless

Rootless

Upstream Latest Release

Yes

Additional environment details

Additional information

Additional information like issue happens only occasionally or issue happens with a particular architecture or on a particular setting

Luap99 commented 6 months ago

check containers.conf inside the image, https://github.com/containers/image_build/blob/main/podman/containers.conf

As you can see the default netns mode is set to host so there are no ports. Normally you should get warning like this Port mappings have been discarded as one of the Host, Container, Pod, and None network modes are in use. Seems like this doesn't work when the netns is set in containers.conf though.

mvmandapati commented 6 months ago

Thanks @Luap99.

We were trying to get OpenShift + Jenkins Agent Pods with Rootless Podman working to support CI builds that use testcontainers. Following the blog article https://www.redhat.com/sysadmin/podman-inside-container and trying to use Podman stable image.

If we cant use netns:host, I guess we can only run testcontainers in rootfull mode.

Luap99 commented 6 months ago

Well you can just run with --network bridge to overwrite the default but this might not work if you try to run in an unprivileged pod.

github-actions[bot] commented 5 months ago

A friendly reminder that this issue had no activity for 30 days.

mvmandapati commented 5 months ago

Thanks @Luap99. Managed to get test containers working inside a pod using your suggestions. This issue can be closed now.

containers / podman