Container stops working after some time

PapeThePope commented 1 year ago

Issue Description

Hey, im trying to host a NodeRed Instance and I got it working so far. Unfortunately I have the problem, that the container stops working after some time. (Most of the time in like 20 Minutes) There is no error in the log of the container and the only error messages I get are the ones when i stop the container. When I do this I get: ERRO[0000] Unable to clean up network for container 76ceb20ce38eed21c3e8cbc5f991454471ac491a650da79c6284e19d69607af7: "tearing down network namespace configuration for container 76ceb20ce38eed21c3e8cbc5f991454471ac491a650da79c6284e19d69607af7: 1 error occurred:\n\t* plugin type=\"bridge\" failed (delete): cni plugin bridge failed: running [/usr/sbin/iptables -t nat -D POSTROUTING -s xx.xx.xx.xx -j CNI-2332533c1d991ec660f2e299 -m comment --comment name: \"podman\" id: \"76ceb20ce38eed21c3e8cbc5f991454471ac491a650da79c6284e19d69607af7\" --wait]: exit status 2: iptables v1.8.4 (nf_tables): Chain 'CNI-2332533c1d991ec660f2e299' does not exist\nTry `iptables -h' or 'iptables --help' for more information.\n\n\n" 76

Important: I dont have access do direct internet. I only have mirrors for the package and docker repositories.

Steps to reproduce the issue

podman run -d -p 1880:1880 -v /home/nodedata:/data --name mynodered nodered/node-red
wait some minutes
stops working and gets an error message on shutdown

Describe the results you received

Container is stopping to work after some minutes

Describe the results you expected

Container should work non-stop

podman info output

host:
  arch: amd64
  buildahVersion: 1.29.0
  cgroupControllers:
  - cpuset
  - cpu
  - cpuacct
  - blkio
  - memory
  - devices
  - freezer
  - net_cls
  - perf_event
  - net_prio
  - hugetlb
  - pids
  - rdma
  cgroupManager: systemd
  cgroupVersion: v1
  conmon:
    package: conmon-2.1.6-1.module+el8.8.0+18098+9b44df5f.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.6, commit: 8c4ab5a095127ecc96ef8a9c885e0e1b14aeb11b'
  cpuUtilization:
    idlePercent: 98.54
    systemPercent: 0.65
    userPercent: 0.81
  cpus: 4
  distribution:
    distribution: '"rhel"'
    version: "8.8"
  eventLogger: file
  hostname: xxxx
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 4.18.0-425.13.1.el8_7.x86_64
  linkmode: dynamic
  logDriver: k8s-file
  memFree: 285347840
  memTotal: 3911933952
  networkBackend: cni
  ociRuntime:
    name: runc
    package: runc-1.1.4-1.module+el8.8.0+18060+3f21f2cc.x86_64
    path: /usr/bin/runc
    version: |-
      runc version 1.1.4
      spec: 1.0.2-dev
      go: go1.19.4
      libseccomp: 2.5.2
  os: linux
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_SYS_CHROOT,CAP_NET_RAW,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.0-2.module+el8.8.0+18060+3f21f2cc.x86_64
    version: |-
      slirp4netns version 1.2.0
      commit: 656041d45cfca7a4176f6b7eed9e4fe6c11e8383
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.2
  swapFree: 8587821056
  swapTotal: 8589930496
  uptime: 21h 27m 36.00s (Approximately 0.88 days)
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - mirrorrepo
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 1
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphRootAllocated: 10726932480
  graphRootUsed: 1281142784
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 1
  runRoot: /run/containers/storage
  transientStore: false
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 4.4.1
  Built: 1682529191
  BuiltTime: Wed Apr 26 19:13:11 2023
  GitCommit: ""
  GoVersion: go1.19.6
  Os: linux
  OsArch: linux/amd64
  Version: 4.4.1

Podman in a container

No

Privileged Or Rootless

Privileged

Upstream Latest Release

No

Additional environment details

No response

Additional information

Time between fails is not even. I didnt find anything on google or elsewhere :(

PapeThePope commented 1 year ago

Puppet has overwritten the iptables, so problem solved :D

Luap99 commented 1 year ago

Note that you can run podman network reload --all to restore network connectivity without restarting the container when other applications (e.g. firewall-cmd --reload) delete our iptables rules.

containers / podman