containers / podman

Podman: A tool for managing OCI containers and pods.
https://podman.io
Apache License 2.0
22.08k stars 2.28k forks source link

podman generate systemd doesn't put "-d" / "--detach" when "-p53:53/udp" is specified without a space #22238

Open dmotte opened 2 months ago

dmotte commented 2 months ago

Issue Description

When I create a container with podman create ... -p53:53/udp ... and then I run podman generate systemd ..., the -d flag is not put in the generated file. But if I use podman create ... -p 53:53/udp ... (note the additional space between -p and 53) instead, then the -d flag is there.

As a side effect, this issue also causes a problem with logs when running a container as a systemd service unit: each log line is written twice in journald: the first time by the podman run ... command's standard output (stdout) and the second time by the journald log driver.

For more details and a better explanation of the problem, see the steps below to reproduce the issue.

Steps to reproduce the issue

  1. podman rm -f coredns; podman create --name=coredns -p53:53/udp docker.io/coredns/coredns; podman generate systemd -n --new coredns > coredns-01.ini (UDP port without the space)
  2. podman rm -f coredns; podman create --name=coredns -p 53:53/udp docker.io/coredns/coredns; podman generate systemd -n --new coredns > coredns-02.ini (UDP port with the space)

Describe the results you received

The generated files look like this:

coredns-01.ini:

# container-coredns.service
# autogenerated by Podman 4.3.1
# Mon Apr  1 23:46:09 UTC 2024

[Unit]
Description=Podman container-coredns.service
Documentation=man:podman-generate-systemd(1)
Wants=network-online.target
After=network-online.target
RequiresMountsFor=%t/containers

[Service]
Environment=PODMAN_SYSTEMD_UNIT=%n
Restart=on-failure
TimeoutStopSec=70
ExecStartPre=/bin/rm \
        -f %t/%n.ctr-id
ExecStart=/usr/bin/podman run \
        --cidfile=%t/%n.ctr-id \
        --cgroups=no-conmon \
        --rm \
        --sdnotify=conmon \
        --replace \
        --name=coredns \
        -p53:53/udp docker.io/coredns/coredns
ExecStop=/usr/bin/podman stop \
        --ignore -t 10 \
        --cidfile=%t/%n.ctr-id
ExecStopPost=/usr/bin/podman rm \
        -f \
        --ignore -t 10 \
        --cidfile=%t/%n.ctr-id
Type=notify
NotifyAccess=all

[Install]
WantedBy=default.target

coredns-02.ini:

# container-coredns.service
# autogenerated by Podman 4.3.1
# Mon Apr  1 23:46:22 UTC 2024

[Unit]
Description=Podman container-coredns.service
Documentation=man:podman-generate-systemd(1)
Wants=network-online.target
After=network-online.target
RequiresMountsFor=%t/containers

[Service]
Environment=PODMAN_SYSTEMD_UNIT=%n
Restart=on-failure
TimeoutStopSec=70
ExecStartPre=/bin/rm \
        -f %t/%n.ctr-id
ExecStart=/usr/bin/podman run \
        --cidfile=%t/%n.ctr-id \
        --cgroups=no-conmon \
        --rm \
        --sdnotify=conmon \
        -d \
        --replace \
        --name=coredns \
        -p 53:53/udp docker.io/coredns/coredns
ExecStop=/usr/bin/podman stop \
        --ignore -t 10 \
        --cidfile=%t/%n.ctr-id
ExecStopPost=/usr/bin/podman rm \
        -f \
        --ignore -t 10 \
        --cidfile=%t/%n.ctr-id
Type=notify
NotifyAccess=all

[Install]
WantedBy=default.target

Differences (with diff coredns-01.ini coredns-02.ini):

3c3
< # Mon Apr  1 23:46:09 UTC 2024
---
> # Mon Apr  1 23:46:22 UTC 2024
22a23
>       -d \
25c26
<       -p53:53/udp docker.io/coredns/coredns
---
>       -p 53:53/udp docker.io/coredns/coredns

As you can see, the -d flag is present only in coredns-02.ini, even if the two generated should be equivalent

Describe the results you expected

The generated files should be equivalent.

podman info output

host:
  arch: amd64
  buildahVersion: 1.28.2
  cgroupControllers:
  - cpu
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon_2.1.6+ds1-1_amd64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.6, commit: unknown'
  cpuUtilization:
    idlePercent: 99.05
    systemPercent: 0.32
    userPercent: 0.63
  cpus: 4
  distribution:
    codename: bookworm
    distribution: debian
    version: "12"
  eventLogger: journald
  hostname: podmanbox01
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 6.1.0-9-amd64
  linkmode: dynamic
  logDriver: journald
  memFree: 1446547456
  memTotal: 4105310208
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun_1.8.1-1+deb12u1_amd64
    path: /usr/bin/crun
    version: |-
      crun version 1.8.1
      commit: f8a096be060b22ccd3d5f3ebe44108517fbf6c30
      rundir: /run/user/1000/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  remoteSocket:
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns_1.2.0-1_amd64
    version: |-
      slirp4netns version 1.2.0
      commit: 656041d45cfca7a4176f6b7eed9e4fe6c11e8383
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.4
  swapFree: 0
  swapTotal: 0
  uptime: 1h 52m 42.00s (Approximately 0.04 days)
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries: {}
store:
  configFile: /home/vagrant/.config/containers/storage.conf
  containerStore:
    number: 5
    paused: 0
    running: 1
    stopped: 4
  graphDriverName: vfs
  graphOptions: {}
  graphRoot: /home/vagrant/.local/share/containers/storage
  graphRootAllocated: 126629462016
  graphRootUsed: 2919960576
  graphStatus: {}
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 6
  runRoot: /run/user/1000/containers
  volumePath: /home/vagrant/.local/share/containers/storage/volumes
version:
  APIVersion: 4.3.1
  Built: 0
  BuiltTime: Thu Jan  1 00:00:00 1970
  GitCommit: ""
  GoVersion: go1.19.8
  Os: linux
  OsArch: linux/amd64
  Version: 4.3.1

Podman in a container

No

Privileged Or Rootless

Rootless

Upstream Latest Release

No

Additional environment details

I'm running Debian 12 (bookworm) and I installed Podman using apt-get update; apt-get install -y podman podman-compose

More specifically, I'm running a https://github.com/dmotte/vagrant-podmanbox Vagrant box (Virtual Machine), version v2023.07.18.2319, but it doesn't matter. It's still the official Podman version from the official Debian 12 apt repos, and it's up-to-date. See the dpkg -s podman output below

vagrant@podmanbox01:~$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
vagrant@podmanbox01:~$ podman version
Client:       Podman Engine
Version:      4.3.1
API Version:  4.3.1
Go Version:   go1.19.8
Built:        Thu Jan  1 00:00:00 1970
OS/Arch:      linux/amd64
vagrant@podmanbox01:~$ dpkg -s podman
Package: podman
Status: install ok installed
Priority: optional
Section: admin
Installed-Size: 35951
Maintainer: Debian Go Packaging Team <pkg-go-maintainers@lists.alioth.debian.org>
Architecture: amd64
Source: libpod (4.3.1+ds1-8)
Version: 4.3.1+ds1-8+b1
Depends: libc6 (>= 2.34), libdevmapper1.02.1 (>= 2:1.02.97), libgpgme11 (>= 1.4.1), libseccomp2 (>= 2.5.0), libsubid4 (>= 1:4.11.1), conmon (>= 2.0.18~), golang-github-containers-common, crun | runc (>= 1.0.0~rc92~)
Recommends: buildah (>= 1.28), dbus-user-session, fuse-overlayfs (>= 1.0.0~), slirp4netns (>= 0.4.1~), catatonit | tini | dumb-init, uidmap
Suggests: containers-storage, docker-compose, iptables
Breaks: buildah (<< 1.10.1-6), fuse-overlayfs (<< 0.7.1), slirp4netns (<< 0.4.1)
Conffiles:
 /etc/cni/net.d/87-podman-bridge.conflist a87c090f17c5274af878e7106e969b60
 /etc/containers/libpod.conf ceec5a77b5f6a56d212eeed7b707d322
Description: engine to run OCI-based containers in Pods
 Podman is an engine for running OCI-based containers in Pods.
 Podman provides a CLI interface for managing Pods, Containers, and
 Container Images.
 .
 At a high level, the scope of libpod and podman is the following:
  * Support multiple image formats including the OCI and Docker image
    formats.
  * Support for multiple means to download images including trust & image
    verification.
  * Container image management (managing image layers, overlay filesystems,
    etc).
  * Full management of container lifecycle.
  * Support for pods to manage groups of containers together.
  * Resource isolation of containers and pods.
  * Support for a Docker-compatible CLI interface through Podman.
 .
 Podman is a daemon-less alternative to Docker.
Built-Using: containerd (= 1.6.20~ds1-1), docker-registry (= 2.8.2+ds1-1), docker.io (= 20.10.24+dfsg1-1), golang-1.19 (= 1.19.8-2), golang-dbus (= 5.1.0-1), golang-fsnotify (= 1.6.0-2), golang-ginkgo (= 1.16.5-3), golang-g
ithub-acarl005-stripansi (= 0.0~git20180116.5a71ef0-3), golang-github-appc-cni (= 1.1.2-1), golang-github-blang-semver (= 4.0.0-1), golang-github-buger-goterm (= 0.0+git20181115.c206103-3), golang-github-cespare-xxhash (= 2
.1.1-2), golang-github-checkpoint-restore-go-criu (= 5.3.0-2), golang-github-chzyer-readline (= 1.4.39.g2972be2-3), golang-github-cilium-ebpf (= 0.9.1-1), golang-github-containerd-stargz-snapshotter (= 0.12.0-2), golang-git
hub-containernetworking-plugins (= 1.1.1+ds1-3), golang-github-containers-buildah (= 1.28.2+ds1-3), golang-github-containers-common (= 0.50.1+ds1-4), golang-github-containers-image (= 5.23.1-4), golang-github-containers-oci
crypt (= 1.0.3-1), golang-github-containers-psgo (= 1.7.1+ds1-1), golang-github-containers-storage (= 1.43.0+ds1-8), golang-github-coreos-bbolt (= 1.3.6-2), golang-github-coreos-go-systemd (= 22.3.2-1), golang-github-cyphar
-filepath-securejoin (= 0.2.3-1), golang-github-davecgh-go-spew (= 1.1.1-3), golang-github-disiqueira-gotree (= 3.0.2-2), golang-github-docker-docker-credential-helpers (= 0.6.4+ds1-1), golang-github-docker-go-connections (
= 0.4.0-4), golang-github-docker-go-units (= 0.4.0-4), golang-github-docker-libtrust (= 0.0~git20150526.0.9cbd2a1-3.1), golang-github-fsouza-go-dockerclient (= 1.8.1-1), golang-github-fullsailor-pkcs7 (= 0.0~git20210826.33d
0574-2), golang-github-ghodss-yaml (= 1.0.0+git20220118.d8423dc-2), golang-github-golang-protobuf-1-3 (= 1.3.5-4), golang-github-google-go-intervals (= 0.0.2-2), golang-github-google-gofuzz (= 1.2.0-1), golang-github-google
-shlex (= 0.0~git20191202.e7afc7f-1), golang-github-google-uuid (= 1.3.0-1), golang-github-gorilla-handlers (= 1.5.1-3), golang-github-gorilla-mux (= 1.8.0-1), golang-github-gorilla-schema (= 1.2.0-2), golang-github-hashico
rp-errwrap (= 1.1.0-1), golang-github-hashicorp-go-multierror (= 1.1.1-2), golang-github-jinzhu-copier (= 0.3.2-2), golang-github-json-iterator-go (= 1.1.12-1), golang-github-juju-ansiterm (= 1.0.0-1), golang-github-klauspo
st-compress (= 1.15.12+ds1-3), golang-github-klauspost-pgzip (= 1.2.5-2), golang-github-kr-fs (= 0.1.0-2), golang-github-lunixbochs-vtclean (= 1.0.0-1), golang-github-manifoldco-promptui (= 0.8.0-2), golang-github-mattn-go-
colorable (= 0.1.13-1), golang-github-mattn-go-isatty (= 0.0.17-1), golang-github-mattn-go-runewidth (= 0.0.14-1), golang-github-mattn-go-shellwords (= 1.0.10-2), golang-github-moby-sys (= 0.0~git20220606.416188a-1), golang
-github-moby-term (= 0.0~git20221120.abb1982-1), golang-github-modern-go-concurrent (= 1.0.3-1.1), golang-github-modern-go-reflect2 (= 1.0.2-2), golang-github-morikuni-aec (= 1.0.0-3), golang-github-nxadm-tail (= 1.4.5+ds1-
5), golang-github-opencontainers-go-digest (= 1.0.0-2), golang-github-opencontainers-image-spec (= 1.1.0~rc2-3), golang-github-opencontainers-runtime-tools (= 0.9.0+git20220423.g0105384-2), golang-github-opencontainers-seli
nux (= 1.10.0+ds1-1), golang-github-opencontainers-specs (= 1.0.2.118.g5cfc4c3-1), golang-github-openshift-imagebuilder (= 1.2.3+ds1-2), golang-github-pkg-errors (= 0.9.1-2), golang-github-pkg-sftp (= 1.13.5-2), golang-gith
ub-pmezard-go-difflib (= 1.0.0-3), golang-github-proglottis-gpgme (= 0.1.1-2), golang-github-rivo-uniseg (= 0.4.2-1), golang-github-spf13-cobra (= 1.6.1-1), golang-github-spf13-pflag (= 1.0.6~git20210604-d5e0c0615ace-1), go
lang-github-sylabs-sif (= 2.8.3-1), golang-github-ulikunitz-xz (= 0.5.6-2), golang-github-vbatts-tar-split (= 0.11.2+ds1-1), golang-github-vbauerster-mpb (= 7.3.2-1), golang-github-vishvananda-netlink (= 1.1.0.125.gf243826-
4), golang-github-vishvananda-netns (= 0.0~git20211101.5004558-1), golang-github-vividcortex-ewma (= 1.1.1-2), golang-github-xeipuuv-gojsonpointer (= 0.0~git20190905.02993c4-3), golang-github-xeipuuv-gojsonreference (= 0.0~
git20180127.bd5ef7b-3), golang-github-xeipuuv-gojsonschema (= 1.2.0-3), golang-go-patricia (= 2.3.1-1), golang-go-zfs (= 3.0.0-1), golang-go.crypto (= 1:0.4.0-1), golang-gocapability-dev (= 0.0+git20200815.42c35b4-2), golan
g-gogoprotobuf (= 1.3.2-3), golang-golang-x-net (= 1:0.7.0+dfsg-1), golang-golang-x-sync (= 0.1.0-1), golang-golang-x-sys (= 0.3.0-1), golang-golang-x-term (= 0.3.0-1), golang-golang-x-text (= 0.7.0-1), golang-golang-x-xerr
ors (= 0.0~git20200804.5ec99f8-1), golang-gomega (= 1.10.3-1), golang-google-genproto (= 0.0~git20200413.b5235f6-3), golang-google-grpc (= 1.33.3-2), golang-google-protobuf (= 1.28.1-3), golang-gopkg-inf.v0 (= 0.9.1-2), gol
ang-gopkg-square-go-jose.v2 (= 2.6.0-2), golang-gopkg-tomb.v1 (= 0.0~git20141024.0.dd63297-8), golang-gopkg-yaml.v3 (= 3.0.1-3), golang-k8s-sigs-yaml (= 1.3.0-1), golang-logrus (= 1.9.0-1), golang-toml (= 1.2.0-2), golang-y
aml.v2 (= 2.4.0-4), rootlesskit (= 1.1.0-1), runc (= 1.1.5+ds1-1)
Homepage: https://github.com/containers/podman

Additional information

Even if I didn't test this issue with the latest Podman release, I think the problem could be around here:

https://github.com/containers/podman/blob/814b7b003cc630bf6ab188274706c383f9fb9915/pkg/systemd/generate/containers.go#L410

And this part didn't change, so I suspect it's still present in the latest version.

dmotte commented 2 months ago

Update: I have a possible hint on what's going on here: IMHO, it could be possible that, for some reason, -p53:53/udp (without the space) is interpreted as -p -5 -3 -: -5 -3 -/ -u -d -p, which contains -d, while -p 53:53/udp (with the space) couldn't be interpreted like that, because it's composed of two separate arguments.

Anyway, I checked also with --publish=53:53/udp and this last one works fine too.

Luap99 commented 2 months ago

Note that podman generate is deprecated in favour of quadlet so it is unlikely that we will fix it. Also note that quadlet was only added v4.4 so it is not supported by your older version. However if somebody else want to submit a PR to fix it we will consider it

eriksjolund commented 2 months ago

This issue reminds me of an old issue

in the sense that both issues concern option command-line parsing for the command podman generate systemd (I'm don't know if the actual bugs causing the issues are related though)

eriksjolund commented 2 months ago

Regarding

Describe the results you expected The generated files should be equivalent.

~I would have expected the podman generate systemd ... command to fail when given the option -p53:53/udp~

~Shouldn't there always be a space between a short option and its value?~

update

I tried both the commands

podman run -p8080:80 -ti docker.io/library/nginx

docker run -p8080:80 -ti docker.io/library/nginx

and noticed that both commands succeeded. I am bit surprised that the commands didn't fail, because I thought there always needs to be a space between a short option and its value. Aparantly it's not needed when using Podman and Docker. I learned something new today.

Luap99 commented 2 months ago

-p53:53/udp -p 53:53/udp and -p=53:53/udp should all be equivalent.

The issue is that podman generate systemd depends on the cli and re parses that the cli to extract certain options because we must modify it slightly. However the issues results here because we parse it with a incomplete flag definition and without strict error enforcing. The parser simply has no idea what do to as it missing a lot of context.

Something like -it can mean -i -t but it could also mean -i with the value t, the only reason that works is because the flag are defined normaly so the parser knowns that both are bool options and do not take an arg. In the generate systemd case -p is not defined as such the parser has no idea that the option accepts and arg and it seems it defaults to an bool so it seems to assume each char is a bool option which then results in d in udp being assumed as flag not a value.

Given we we recommend quadlet now (bugs like these are why quadlet is much nicer) and the fact that there is a simple workaround (never use the short form syntax without space) I think a wontfix is appropriate but I am also fine to keep it open in case someone else wants toi contribuate a simple bug fix, however this is no way limted to -p, and other shorthand flag can trigger the same thing, e.g. -unobody

dmotte commented 2 months ago

@eriksjolund thanks for sharing!

Well, in fact, if it's a short option (single letter, such as -p), usually you can also write it as a single argument (without the space) and it should work :) This behavior comes from the getopt function (POSIX standard). See e.g. https://superuser.com/questions/1652953/linux-program-argument-with-vs-without-space

Luap99 commented 2 months ago

@eriksjolund thanks for sharing!

Well, in fact, if it's a short option (single letter, such as -p), usually you can also write it as a single argument (without the space) and it should work :) This behavior comes from the getopt function (POSIX standard). See e.g. https://superuser.com/questions/1652953/linux-program-argument-with-vs-without-space

Yes this syntax is supported by all the applications using getopt, the -p=53:53/udp however is not supported by getopt and a special extension by go parser we use https://github.com/spf13/pflag/

github-actions[bot] commented 1 month ago

A friendly reminder that this issue had no activity for 30 days.