search

containers / podman

Podman: A tool for managing OCI containers and pods.
https://podman.io
Apache License 2.0
23.45k stars 2.38k forks source link

rootless podman ERRO[0000] error joining network namespace for container #6800

Closed balamuruganravi closed 4 years ago

balamuruganravi commented 4 years ago

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description To manage containers after the system reboot I am trying setup systemd script for rootless podman containers. podman run command

podman run -d \
-p 9200:9200 \
-p 9300:9300 \
--name paperkraft \
-v $CONTAINER_NAME:/usr/share/cluster/data \
-e "ES_JAVA_OPTS=-Xms2g -Xmx2g" \
-e "path.data=/usr/share/cluster/data" \
-e "http.host=0.0.0.0" \
$CONTAINER_IMAGE

cat /etc/systemd/system/paperkraft.service

[Unit]
Description=paperkraft  podman container
Wants=syslog.service

[Service]
User=podman
Restart=always
ExecStartPre=/usr/bin/podman system migrate
ExecStart=/usr/bin/podman start -a paperkraft
ExecStop=/usr/bin/podman stop -t 10 paperkraft

[Install]
WantedBy=multi-user.target

Steps to reproduce the issue:

  1. Run the container

  2. systemctl daemon-reload

  3. systemctl enable paperkraft.service Created symlink from /etc/systemd/system/multi-user.target.wants/paperkraft.service to /etc/systemd/system/paperkraft.service.

Describe the results you received: After the system reboot container are not started and giving the below output for podman ps or podman ps -a

ERRO[0000] error joining network namespace for container 3164432a60af4b2320b404c44676edd87846b2ac7fcd5cc464a2f94184161ea0: error retrieving network namespace at /tmp/run-600000/netns/cni-1a003ffc-4779-8bd9-273e-19325859e8a1: unknown FS magic on "/tmp/run-600000/netns/cni-1a003ffc-4779-8bd9-273e-19325859e8a1": 58465342
ERRO[0000] unable to get container info: "container 3164432a60af4b2320b404c44676edd87846b2ac7fcd5cc464a2f94184161ea0 is not valid: container has already been removed"

Describe the results you expected: paperkraft container should be running state

Additional information you deem important (e.g. issue happens only occasionally): System reboot Output of podman version:

Version:            1.6.4
RemoteAPI Version:  1
Go Version:         go1.13.4
OS/Arch:            linux/amd64

Output of podman info --debug:

debug:
  compiler: gc
  git commit: ""
  go version: go1.13.4
  podman version: 1.6.4
host:
  BuildahVersion: 1.12.0-dev
  CgroupVersion: v1
  Conmon:
    package: conmon-2.0.6-1.module+el8.2.0+6368+cf16aa14.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.6, commit: 9adfe850ef954416ea5dd0438d428a60f2139473'
  Distribution:
    distribution: '"rhel"'
    version: "8.2"
  IDMappings:
    gidmap:
    - container_id: 0
      host_id: 600000
      size: 1
    - container_id: 1
      host_id: 666666
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 600000
      size: 1
    - container_id: 1
      host_id: 666666
      size: 65536
  MemFree: 9740709888
  MemTotal: 16496082944
  OCIRuntime:
    name: runc
    package: runc-1.0.0-65.rc10.module+el8.2.0+6368+cf16aa14.x86_64
    path: /usr/bin/runc
    version: 'runc version spec: 1.0.1-dev'
  SwapFree: 0
  SwapTotal: 0
  arch: amd64
  cpus: 4
  eventlogger: file
  hostname: ip-10-198-154-92.cloud.dev.net
  kernel: 4.18.0-193.1.2.el8_2.x86_64
  os: linux
  rootless: true
  slirp4netns:
    Executable: /usr/bin/slirp4netns
    Package: slirp4netns-0.4.2-3.git21fdece.module+el8.2.0+6368+cf16aa14.x86_64
    Version: |-
      slirp4netns version 0.4.2+dev
      commit: 21fdece2737dc24ffa3f01a341b8a6854f8b13b4
  uptime: 1h 35m 11.67s (Approximately 0.04 days)
registries:
  blocked:
  - all
  insecure: null
  search:
  - test.artifactory.global.com
  - artifactory.global.com
store:
  ConfigFile: /home/podman/.config/containers/storage.conf
  ContainerStore:
    number: 5
  GraphDriverName: overlay
  GraphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: fuse-overlayfs-0.7.2-5.module+el8.2.0+6368+cf16aa14.x86_64
      Version: |-
        fuse-overlayfs: version 0.7.2
        FUSE library version 3.2.1
        using FUSE kernel interface version 7.26
  GraphRoot: /home/podman/.local/share/containers/storage
  GraphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  ImageStore:
    number: 16
  RunRoot: /tmp/run-600000
  VolumePath: /home/podman/.local/share/containers/storage/volumes

Package info (e.g. output of rpm -q podman or apt list podman):

podman-1.6.4-11.module+el8.2.0+6368+cf16aa14.x86_64

Additional environment details (AWS, VirtualBox, physical, etc.): AWS & VM

mheon commented 4 years ago

This is most likely systemd killing Podman and removing the login session for the user in question. You need to run loginctl enable-linger for the user in question to ensure systemd does not assume the user has logged out and remove its temporary directory.

Also, we strongly recommend you use podman generate systemd to generate systemd units, I can see several issues with the example you posted (no KillMode=none, not using conmon PID files)

balamuruganravi commented 4 years ago

thanks for your response @mheon after running loginctl enable-linger podman command and system reboot. container process started able to load application URL. But podman ps command giving the same error. is anything I am doing wrong here

ERRO[0000] error joining network namespace for container 6d8c0684f5ebf6a7fccfd1addf71313d4c2411169c11d4ee2c54260386571a36: error retrieving network namespace at /tmp/run-600000/netns/cni-f6a5553c-0994-5d4f-17ef-2530f4300903: unknown FS magic on "/tmp/run-600000/netns/cni-f6a5553c-0994-5d4f-17ef-2530f4300903": 58465342
ERRO[0000] unable to get container info: "container 6d8c0684f5ebf6a7fccfd1addf71313d4c2411169c11d4ee2c54260386571a36 is not valid: container has already been removed"

Also the container auto start is not consistent if I do the reboot again. If I use podman generate systemd I can't share the systemd Unit files with other user and currently I can't do podman upgrade as well.

balamuruganravi commented 4 years ago

Hi @mheon I updated my podman run scripts to run as pod . and generated systemd unit file for pod . But still no success

  1. podman generate systemd --files --restart-policy=always --name kraft
  2. created folder mkdir -p ~/.config/systemd/user
  3. copied unit file to $HOME/.config/systemd/user
  4. systemctl --user enable pod-kraft.service but received the below error.

Failed to connect to bus: No such file or directory

podman run -dt \
--pod new:kraft \
-p 9200:9200 \
-p 9300:9300 \
--name paperkraft \
-v $CONTAINER_NAME:/usr/share/cluster/data \
-e "ES_JAVA_OPTS=-Xms2g -Xmx2g" \
-e "path.data=/usr/share/cluster/data" \
-e "http.host=0.0.0.0" \
$CONTAINER_IMAGE
balamuruganravi commented 4 years ago

can someone help me to resolve the above mentioned issue.

mheon commented 4 years ago

If that systemctl command is failing, you most likely do not have a systemd user session enabled/running (which is strange, because I was fairly certain RHEL shipped with them enabled). Might be caused if you tried to use systemctl --user as root?

balamuruganravi commented 4 years ago

I executed this command as podman user only 

podman@ip-xx-xx-xx-xx[tempods] $ systemctl --user enable pod-kraft.service
Failed to connect to bus: No such file or directory
cubadonis commented 4 years ago

In my case, this was an SELinux Issue. Try to create your container with

-v $CONTAINER_NAME:/usr/share/cluster/data:Z \ instead of -v $CONTAINER_NAME:/usr/share/cluster/data \

I executed this command as podman user only

podman@ip-xx-xx-xx-xx[tempods] $ systemctl --user enable pod-kraft.service
Failed to connect to bus: No such file or directory

check your user environment especially $XDG_RUNTIME_DIR. The easiest way to fix this is to login directly as user "podman" or try ssh podman@localhost

for further information about this issue take a look at https://github.com/systemd/systemd/issues/10574

github-actions[bot] commented 4 years ago

A friendly reminder that this issue had no activity for 30 days.

rhatdan commented 4 years ago

Is this still an issue?

balamuruganravi commented 4 years ago

Is this still an issue?

Sorry for the delay. Our platform team updated the podman installation recently. I am yet to verify it. Soon I will test and post my updates here.

jenschurchill commented 4 years ago

Any podman command I run, gives:

podman ps -a

ERRO[0000] error joining network namespace for container 470b76724bcff05db55806baeb376b9951e8a4db66412937ed827e3d18677e4a: error retrieving network namespace at /run/user/1000/netns/cni-8bf84c49-dcbd-bd4b-f954-a09d142834f0: failed to Statfs "/run/user/1000/netns/cni-8bf84c49-dcbd-bd4b-f954-a09d142834f0": no such file or directory

ERRO[0000] unable to get container info: "container 470b76724bcff05db55806baeb376b9951e8a4db66412937ed827e3d18677e4a is not valid: container has already been removed"

I have stopped and removed all containers, and run podman system prune --all Rebooted the system, and still get the above.

mheon commented 4 years ago

Does podman rm --force 470b76724bcff05db55806baeb376b9951e8a4db66412937ed827e3d18677e4a work?

jenschurchill commented 4 years ago

Yes, it appears to. Thank you, incredible response time!

mheon commented 4 years ago

Seems to be fixed, closing - please re-open if incorrect

opensourceally commented 3 years ago

Getting Similar error


**ERRO[0000] error joining network namespace for container 

9936ec84b05dcf89a1946b19e193139ed51e2727df77a5d7fbc54c13fb46c7bb: 
error retrieving network namespace at 
/tmp/run-1000/netns/cni-b0a12ed0-f7f3-9796-dc26-0207fcbcdf96: 

unknown FS magic on "/tmp/run-1000/netns/cni-b0a12ed0-f7f3-9796-dc26-0207fcbcdf96": ef53
Error: error joning network namespace of container 
9936ec84b05dcf89a1946b19e193139ed51e2727df77a5d7fbc54c13fb46c7bb: 
error retrieving network namespace at 
/tmp/run-1000/netns/cni-b0a12ed0-f7f3-9796-dc26-0207fcbcdf96: unknown FS magic on 
"/tmp/run-1000/netns/cni-b0a12ed0-f7f3-9796-dc26-0207fcbcdf96": ef53**

WSL2
Ubuntu 20
podman 2.1.1
balamuruganravi commented 3 years ago

@mheon could you please reopen this issue.

mheon commented 3 years ago

I'd recommend opening a new issue - this one has gotten very stale.

balamuruganravi commented 3 years ago

@mheon PTAL https://github.com/containers/podman/issues/7976

lededev commented 3 years ago

WSL2 / Alpine:edge / podman version 3.1.0 no systemd, no openrc

podman ps -a
ERRO[0000] error joining network namespace for container 3fd6086ec181198bf270b6a9e4f660cdaee4a991a8461c0c6045c21f8c6dbb79: error retrieving network namespace at /tmp/podman-run-1000/netns/cni-b61c831a-c43e-2a9d-b848-a27f55048da1: unknown FS magic on "/tmp/podman-run-1000/netns/cni-b61c831a-c43e-2a9d-b848-a27f55048da1": ef53
Error: error joining network namespace of container 3fd6086ec181198bf270b6a9e4f660cdaee4a991a8461c0c6045c21f8c6dbb79: error retrieving network namespace at /tmp/podman-run-1000/netns/cni-b61c831a-c43e-2a9d-b848-a27f55048da1: unknown FS magic on "/tmp/podman-run-1000/netns/cni-b61c831a-c43e-2a9d-b848-a27f55048da1": ef53

podman rm --force 3fd6086ec181198bf270b6a9e4f660cdaee4a991a8461c0c6045c21f8c6dbb79
ERRO[0000] error joining network namespace for container 3fd6086ec181198bf270b6a9e4f660cdaee4a991a8461c0c6045c21f8c6dbb79: error retrieving network namespace at /tmp/podman-run-1000/netns/cni-b61c831a-c43e-2a9d-b848-a27f55048da1: unknown FS magic on "/tmp/podman-run-1000/netns/cni-b61c831a-c43e-2a9d-b848-a27f55048da1": ef53
Error: error freeing lock for container 3fd6086ec181198bf270b6a9e4f660cdaee4a991a8461c0c6045c21f8c6dbb79: no such file or directory

How to resolve?

rhatdan commented 3 years ago

podman 3.1.1 Should resolve this, but has not been released yet. You are best to roll back to previous version 3.1.1 should be released early next week. Or build the package from the upstream.

alexcern commented 3 years ago

For rootless podman users on WSL2: as far as I can tell this happens after a reboot because podman needs to refresh its state after a reboot, but was unable to detect that the system has rebooted in WSL2.

Solution: rm -rf /tmp/run-$(id -u)/libpod/tmp (podman expects this folder to vanish after a reboot to detect the reboot)

Useful info about this in https://www.redhat.com/sysadmin/sudo-rootless-podman.

rhatdan commented 3 years ago

Since WSL2 does not have systemd or mount /tmp with tmpfs, then it breaks the assumption. Do you know if there is a similar way to trigger this behaviour in WSL2? IE Cause certain files to be deleted? Do we need to drop an init script?

alexcern commented 3 years ago

My understanding is there is no systemd/init.d at all; it seems we can run things at startup via the Windows scheduler (example) but that's not very convenient.

In my case I made do with assuming that /proc/1 being more recent than the temp folder indicated a reboot took place. I added this to my .bashrc:

function refresh_rootless_podman_after_reboot {
  local libpod_tmp="/tmp/run-$(id -u)/libpod/tmp"
  if [ /proc/1 -nt "${libpod_tmp}" ]; then
    rm -rf "${libpod_tmp}" 
  fi
}
refresh_rootless_podman_after_reboot

But a simpler solution could be to include the boot ID in the temp path so we always get a fresh folder after reboot, e.g export XDG_RUNTIME_DIR="/tmp/run-$(id -u)/$(cat /proc/sys/kernel/random/boot_id)"

benyaminl commented 2 years ago

@alexcern by anychance, is the folder change in 3.3.4, or it's depend on the distro? because the folder changed to /tmp/podman-run-(uid)/

jpbow commented 2 years ago

The below command worked for me (running Podman 3.4.2 on WSL 2):

rm -rf /tmp/podman-run-$(id -u)/libpod/tmp

der-eismann commented 2 years ago

I just encountered this issue with podman 4.0.2 on AlmaLinux 8, no WSL.

$ podman ps -a
ERRO[0000] Joining network namespace for container 96f881cd6f14891d79bdac918bd5dea8f9923d8282c6807b433ba189cec9ace2: error retrieving network namespace at /run/user/1001/netns/netns-e293186e-0dd0-326c-dbc9-2b263fcc0d1d: unknown FS magic on "/run/user/1001/netns/netns-e293186e-0dd0-326c-dbc9-2b263fcc0d1d": 1021994 
Error: error joining network namespace of container 96f881cd6f14891d79bdac918bd5dea8f9923d8282c6807b433ba189cec9ace2: error retrieving network namespace at /run/user/1001/netns/netns-e293186e-0dd0-326c-dbc9-2b263fcc0d1d: unknown FS magic on "/run/user/1001/netns/netns-e293186e-0dd0-326c-dbc9-2b263fcc0d1d": 1021994

$ podman rm --force 96f881cd6f14891d79bdac918bd5dea8f9923d8282c6807b433ba189cec9ace2
ERRO[0000] Joining network namespace for container 96f881cd6f14891d79bdac918bd5dea8f9923d8282c6807b433ba189cec9ace2: error retrieving network namespace at /run/user/1001/netns/netns-e293186e-0dd0-326c-dbc9-2b263fcc0d1d: unknown FS magic on "/run/user/1001/netns/netns-e293186e-0dd0-326c-dbc9-2b263fcc0d1d": 1021994 
ERRO[0000] container_linux.go:419: signaling init process caused: operation not permitted 
Error: cannot remove container 96f881cd6f14891d79bdac918bd5dea8f9923d8282c6807b433ba189cec9ace2 as it could not be stopped: error sending SIGKILL to container 96f881cd6f14891d79bdac918bd5dea8f9923d8282c6807b433ba189cec9ace2: operation not permitted

The containers were created by testcontainers over the Podman socket, not sure if that did anything special.

rhatdan commented 2 years ago

Could you update to podman 4.1 and see if this continues to happen. Please open a new issue rather then adding to a closed issue.

jangel97 commented 2 years ago

Hi,

We are using podman 4.1.1

[awx@exec2 ~]$ podman version
Client:       Podman Engine
Version:      4.1.1
API Version:  4.1.1
Go Version:   go1.17.7
Built:        Mon Jul 11 14:56:53 2022
OS/Arch:      linux/amd64

We are using Ansible Automation Platform, which relies on podman and I am still seeing the issue.

[awx@exec2 ~]$ podman ps
ERRO[0001] Joining network namespace for container 3aee535c82cd67a79a961a907522fdd375b65a5e462a11f532e033a8ac9b40fc: error retrieving network namespace at /run/user/1005/netns/netns-c3bef77c-a537-dafd-3369-cd567111feb8: failed to Statfs "/run/user/1005/netns/netns-c3bef77c-a537-dafd-3369-cd567111feb8": no such file or directory 
ERRO[0001] Joining network namespace for container 9d618926ff78856b7d18ee9c2fa3ab265061bbdf7626a690143d7ef4e9212fe0: error retrieving network namespace at /run/user/1005/netns/netns-6ee00e54-9900-2427-4da2-446c43870aa2: failed to Statfs "/run/user/1005/netns/netns-6ee00e54-9900-2427-4da2-446c43870aa2": no such file or directory 
ERRO[0001] Joining network namespace for container d445368385c2794b73b0c159cb836329149f6e0287d92647e0bc5f60601df52c: error retrieving network namespace at /run/user/1005/netns/netns-98671a72-c2c8-6080-08e9-fec055587df0: failed to Statfs "/run/user/1005/netns/netns-98671a72-c2c8-6080-08e9-fec055587df0": no such file or directory 
ERRO[0001] Joining network namespace for container e3ef7a74997bea6118bc2b2e8accc8ee2ab253ace6ce8b0a47c6c92d5c12f66f: error retrieving network namespace at /run/user/1005/netns/netns-e729af5a-54b1-c136-d833-4fcdf3d89cb8: failed to Statfs "/run/user/1005/netns/netns-e729af5a-54b1-c136-d833-4fcdf3d89cb8": no such file or directory 
CONTAINER ID  IMAGE                                               COMMAND               CREATED         STATUS             PORTS       NAMES
d1294beacfe2  hub/custom-iso-provisioning:latest  ssh-agent sh -c t...  12 minutes ago  Up 12 minutes ago              ansible_runner_106884

I do not see any directory /run/user/1005 in my system... It looks like the files are in /tmp/podman-run-1005/netns/. Also, none of the reported network namespaces can be found in the /tmp/podman-run-1005/netns/.

[awx@exec2 ~]$ ls /tmp/podman-run-1005/netns/
netns-0b35ca2d-be90-db24-7e97-571b1b5c5bb7  netns-9c14dcb0-650c-5b39-b299-059c75af78e1  netns-c7cdd98f-69e3-f087-cf9a-bf9e1fe42329
netns-26f03598-bce9-6bda-a002-76fd718187ce  netns-a3b618bd-a448-d165-2661-a01fff440f57  netns-fb365b3e-8dd9-9165-b27f-eb02e0c871a9

What is the impact of this error? Can it reduce production in our platform? So far, the errors are there but I do not think they are impacting our workloads.

yaroslavsadin commented 1 year ago

@rhatdan Getting this issue with RHEL9 and podman 4.0.2, is there any update/fix/workaround?

rhatdan commented 1 year ago

Did you enable the linger?

yaroslavsadin commented 1 year ago

@rhatdan Do you mean loginctl enable-linger $(id -un)?

rhatdan commented 1 year ago

Yes

yaroslavsadin commented 1 year ago

@rhatdan I'll try it out, thanks

rlenferink commented 1 year ago

@rhatdan, on my system (Rocky Linux 9.1 with Podman 4.2.0) this is still an issue. Lingering is enabled but containers are not started and the same error as described in this issue pops up:

[podman@server2 ~]$ loginctl user-status $(id -un)
podman (990)
           Since: Sat 2023-01-14 16:37:06 CET; 31s ago
           State: lingering
          Linger: yes
            Unit: user-990.slice
                  └─user@990.service
                    └─init.scope
                      ├─844 /usr/lib/systemd/systemd --user
                      └─846 "(sd-pam)"

[podman@server2 ~]$ podman ps
ERRO[0000] Joining network namespace for container 2c01eddb8ae6b41f455fb2e71885760b886073cfcd684080285d7032b87c6f64: error retrieving network namespace at /run/user/990/netns/netns-5c8ceec4-2678-8a95-a179-6a06aa4f56db: failed to Statfs "/run/user/990/netns/netns-5c8ceec4-2678-8a95-a179-6a06aa4f56db": no such file or directory 
ERRO[0000] Joining network namespace for container 350ea594aa7ddc839c9c6dd2409af8b02ce9b5ee66d51f15220d5d28acce993f: error retrieving network namespace at /run/user/990/netns/netns-2b1876a9-f55d-cd66-5caf-ac131fa5f7a6: failed to Statfs "/run/user/990/netns/netns-2b1876a9-f55d-cd66-5caf-ac131fa5f7a6": no such file or directory 
ERRO[0000] Joining network namespace for container 67476af881d80ba69cc5dd986895a51e903df155a7a721258de8a1713c3fffa3: error retrieving network namespace at /run/user/990/netns/netns-59834835-a5bd-9ce4-149b-e5d39aeb1f6a: failed to Statfs "/run/user/990/netns/netns-59834835-a5bd-9ce4-149b-e5d39aeb1f6a": no such file or directory 
ERRO[0000] Joining network namespace for container 72a55ef327eef91e9aed97c9ece53cda4b2d2e4b14a522a458090d0cf7eff6d9: error retrieving network namespace at /run/user/990/netns/netns-aa5a8b18-af6e-66ce-fa01-33140ae534d6: failed to Statfs "/run/user/990/netns/netns-aa5a8b18-af6e-66ce-fa01-33140ae534d6": no such file or directory 
ERRO[0000] Joining network namespace for container 79f863de695267f3027d835b4f78e7e418a23b8feeae14fd2889e833a60a1aa8: error retrieving network namespace at /run/user/990/netns/netns-05f00422-064f-49c7-0875-10f81401bf31: failed to Statfs "/run/user/990/netns/netns-05f00422-064f-49c7-0875-10f81401bf31": no such file or directory 
ERRO[0000] Joining network namespace for container 7d7a707ee255e4b04e13cd617d0b09d44dcd2d7aaf20a19d888392843ecdc67e: error retrieving network namespace at /run/user/990/netns/netns-b668c156-48f4-abdf-21ac-901206352a62: failed to Statfs "/run/user/990/netns/netns-b668c156-48f4-abdf-21ac-901206352a62": no such file or directory 
ERRO[0000] Joining network namespace for container 7e75f6b8b1d6e9fde848c43fde7bbade2330f7a6a44d52e1ae988e316045a13a: error retrieving network namespace at /run/user/990/netns/netns-022fff9c-6e50-785f-461b-bfb625952317: failed to Statfs "/run/user/990/netns/netns-022fff9c-6e50-785f-461b-bfb625952317": no such file or directory 
ERRO[0000] Joining network namespace for container 883568b6d14f81577ae34ea12fd38be28d1b8580ab355b6baf8621d4be2bb469: error retrieving network namespace at /run/user/990/netns/netns-73bae70b-6ac0-d546-22fe-44d261e45267: failed to Statfs "/run/user/990/netns/netns-73bae70b-6ac0-d546-22fe-44d261e45267": no such file or directory 
ERRO[0000] Joining network namespace for container b72a579b0868c014760cd76bfc8d52c35495bc35c2d171cac42a28508e89bd0a: error retrieving network namespace at /run/user/990/netns/netns-66d1bf2f-b8a1-61e8-345e-2bac9f4f8007: failed to Statfs "/run/user/990/netns/netns-66d1bf2f-b8a1-61e8-345e-2bac9f4f8007": no such file or directory 
ERRO[0000] Joining network namespace for container cc67ed6bc5cc3209d39b1d41234a2675d355442cd136d5e11727734b10b24bcf: error retrieving network namespace at /run/user/990/netns/netns-9b83d2a4-c5b7-e681-b6ea-0e3208e60e3d: failed to Statfs "/run/user/990/netns/netns-9b83d2a4-c5b7-e681-b6ea-0e3208e60e3d": no such file or directory 
CONTAINER ID  IMAGE       COMMAND     CREATED     STATUS      PORTS       NAMES

I started a whole bunch of containers using docker-compose and restart: always, however, nothing is started and the above errors are shown.

Any ideas?

rhatdan commented 1 year ago

Do you have podman-restart enabled?

rlenferink commented 1 year ago

Do you have podman-restart enabled?

No, I did not have it enabled (was not aware of it). Enabling the service and rebooting the ERRO[0000] Joining network namespace also disappeared (and the containers are started). Thanks @rhatdan !