Container systemd service unit depends on NFS Mounting

zoumingzhe commented 1 month ago

Issue Description

I deployed my service using podman-compose and mounted an NFS directory (/mnt/data/download) for the container in Centos Stream9 (5.14.0-437.el9.x86_6). And create a systemd service unit using the podman generate systemd command.

After my deployment is complete, I can access files in the NFS directory within the container, but when the host restarts, the NFS directory (/mnt/data/download) in the container is always empty.

[root@files ~]# cat /etc/fstab 

#
# /etc/fstab
# Created by anaconda on Sun May 26 04:30:24 2024
#
# Accessible filesystems, by reference, are maintained under '/dev/disk/'.
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.
#
# After editing this file, run 'systemctl daemon-reload' to update systemd
# units generated from this file.
#
/dev/mapper/cs-root     /                       xfs     defaults        0 0
UUID=7f456e92-85ed-4666-90ae-52baafeaaa13 /boot                   xfs     defaults        0 0
/dev/mapper/cs-swap     none                    swap    defaults        0 0
UUID=a5d1467a-dbe5-4b3d-bcad-08046afaaeb5 /mnt auto nofail 0 0
pangu.stor:/mnt/server/Downloader /mnt/datas/download nfs defaults

Steps to reproduce the issue

Deploy a service using podman-compose and mount the nfs directory (/mnt/data/download).
Create a systemd service unit using the 'podman generate systemd' command.
Restart the host.

Describe the results you received

The container works normally, but the nfs directory (/mnt/data/download) is empty.

A manual restart is required to restore.

Describe the results you expected

After restarting the host, the container needs to be mounted by NFS and then started (especially when NFS servers are started simultaneously).

podman info output

[root@files ~]# podman version
Client:       Podman Engine
Version:      4.9.4-dev
API Version:  4.9.4-dev
Go Version:   go1.21.7 (Red Hat 1.21.7-1.el9)
Built:        Wed Mar 20 18:22:46 2024
OS/Arch:      linux/amd64
[root@files ~]#  podman info
host:
  arch: amd64
  buildahVersion: 1.33.5
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  - rdma
  - misc
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.10-2.el9.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.10, commit: d807bb8c1de3dc05fb66c77d2979a7f6903804bf'
  cpuUtilization:
    idlePercent: 99.07
    systemPercent: 0.53
    userPercent: 0.39
  cpus: 2
  databaseBackend: sqlite
  distribution:
    distribution: centos
    version: "9"
  eventLogger: journald
  freeLocks: 2033
  hostname: files.vm
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.14.0-437.el9.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 1107890176
  memTotal: 3497226240
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.9.0-1.el9.x86_64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.9.0
    package: netavark-1.10.3-1.el9.x86_64
    path: /usr/libexec/podman/netavark
    version: netavark 1.10.3
  ociRuntime:
    name: crun
    package: crun-1.14.4-1.el9.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.14.4
      commit: a220ca661ce078f2c37b38c92e66cf66c012d9c1
      rundir: /run/user/0/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20231204.gb86afe3-1.el9.x86_64
    version: |
      pasta 0^20231204.gb86afe3-1.el9.x86_64-pasta
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.3-1.el9.x86_64
    version: |-
      slirp4netns version 1.2.3
      commit: c22fde291bb35b354e6ca44d13be181c76a0a432
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.2
  swapFree: 1608220672
  swapTotal: 1719660544
  uptime: 181h 6m 5.00s (Approximately 7.54 days)
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.access.redhat.com
  - registry.redhat.io
  - docker.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 10
    paused: 0
    running: 10
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphRootAllocated: 14315159552
  graphRootUsed: 8587726848
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "true"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 4
  runRoot: /run/containers/storage
  transientStore: false
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 4.9.4-dev
  Built: 1710930166
  BuiltTime: Wed Mar 20 18:22:46 2024
  GitCommit: ""
  GoVersion: go1.21.7 (Red Hat 1.21.7-1.el9)
  Os: linux
  OsArch: linux/amd64
  Version: 4.9.4-dev

[root@files ~]# rpm -q podman
podman-5.0.0-1.el9.x86_64

Podman in a container

No

Privileged Or Rootless

None

Upstream Latest Release

Yes

Additional environment details

[root@files ~]# uname -a
Linux files.vm 5.14.0-437.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Apr 9 12:57:02 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Additional information

[root@files ~]# cat /etc/systemd/system/container-dir-browser-download.service 
# container-dir-browser-download.service
# autogenerated by Podman 4.9.4-dev
# Wed Jul 10 20:44:20 CST 2024

[Unit]
Description=Podman container-dir-browser-download.service
Documentation=man:podman-generate-systemd(1)
Wants=network-online.target
After=network-online.target
RequiresMountsFor=/run/containers/storage

[Service]
Environment=PODMAN_SYSTEMD_UNIT=%n
Restart=on-failure
TimeoutStopSec=70
ExecStart=/usr/bin/podman start dir-browser-download
ExecStop=/usr/bin/podman stop  \
        -t 10 dir-browser-download
ExecStopPost=/usr/bin/podman stop  \
        -t 10 dir-browser-download
PIDFile=/run/containers/storage/overlay-containers/d8dd4b7647af7838d43e341000b7d48b55d420b246fb49a05929d42f847eca6f/userdata/conmon.pid
Type=forking

[Install]
WantedBy=default.target

zoumingzhe commented 1 month ago

The system service unit seems to only rely on RequiresMountsFor=/run/containers/storage.

If possible, when executing podman generate systemd, the nfs directory can be automatically added to RequiresMountsFor. Otherwise, we can add a --requires-mounts option, just like --requires.

[root@files ~]# podman generate systemd --help
[DEPRECATED] Generate systemd units

Description:
  Generate systemd units for a pod or container.
  The generated units can later be controlled via systemctl(1).

DEPRECATED command:
It is recommended to use Quadlets for running containers and pods under systemd.

Please refer to podman-systemd.unit(5) for details.

Usage:
  podman generate systemd [options] {CONTAINER|POD}

Examples:
  podman generate systemd CTR
  podman generate systemd --new --time 10 CTR
  podman generate systemd --files --name POD

Options:
      --after stringArray         Add dependencies order to the generated unit file
      --container-prefix string   Systemd unit name prefix for containers (default "container")
  -e, --env stringArray           Set environment variables to the systemd unit files
  -f, --files                     Generate .service files instead of printing to stdout
      --format string             Print the created units in specified format (json)
  -n, --name                      Use container/pod names instead of IDs
      --new                       Create a new container or pod instead of starting an existing one
      --no-header                 Skip header generation
      --pod-prefix string         Systemd unit name prefix for pods (default "pod")
      --requires stringArray      Similar to wants, but declares stronger requirement dependencies
      --restart-policy string     Systemd restart-policy (default "on-failure")
      --restart-sec uint          Systemd restart-sec
      --separator string          Systemd unit name separator between name/id and prefix (default "-")
      --start-timeout uint        Start timeout override
      --stop-timeout uint         Stop timeout override (default 10)
      --template                  Make it a template file and use %i and %I specifiers. Working only for containers
      --wants stringArray         Add (weak) requirement dependencies to the generated unit file

[root@files ~]#

rhatdan commented 1 month ago

Could you look at using quadlets rather the podman systemd generate, then you can configure the quadlet file the way you want from a systemd point of view.

zoumingzhe commented 1 month ago

Could you look at using quadlets rather the podman systemd generate, then you can configure the quadlet file the way you want from a systemd point of view.

I am currently not considering using quadlet.

giuseppe commented 1 month ago

I don't think it is realistic to expect we can expose all the systemd configuration from the podman CLI. If you need a custom configuration, just edit the file

zoumingzhe commented 1 month ago

I don't think it is realistic to expect we can expose all the systemd configuration from the podman CLI. If you need a custom configuration, just edit the file

I think podman generate systemd needs to provide necessary options instead of directly editing the file.

giuseppe commented 1 month ago

"podman generate systemd" is a deprecated command. We are not going to add new flags to a command that will likely be dropped in a future release

containers / podman