search

coreos / coreos-assembler

Tooling container to assemble CoreOS-like systems
https://coreos.github.io/coreos-assembler/
Apache License 2.0
336 stars 165 forks source link

kargs handling regressions? #2430

Open miabbott opened 2 years ago

miabbott commented 2 years ago

Using quay.io/coreos-assembler/coreos-assembler:latest (maps to commit 350c1d93bc3343c8f523d642664bccd8dc0424c9), when I do coreos-assembler run --kargs foo=bar --kargs bar=foo, only the last karg is being appended to the kernel cmdline.

$ coreos-assembler run --kargs foo=bar --kargs bar=foo
COREOS_ASSEMBLER_CONTAINER_RUNTIME_ARGS=--name cosa-again
BASH_FUNC_coreos-assembler%%=() {  env | grep --color=auto COREOS_ASSEMBLER;
 chrt --idle 0 podman run --rm -ti --security-opt label=disable --privileged --uidmap=1000:0:1 --uidmap=0:1:1000 --uidmap 1001:1001:64536 -v ${PWD}:/srv/ --device /dev/kvm --device /dev/fuse --tmpfs /tmp -v /var/tmp:/var/tmp --name coreos-assembler ${COREOS_ASSEMBLER_CONFIG_GIT:+-v $COREOS_ASSEMBLER_CONFIG_GIT:/srv/;
+ chrt --idle 0 podman run --rm -ti --security-opt label=disable --privileged --uidmap=1000:0:1 --uidmap=0:1:1000 --uidmap 1001:1001:64536 -v /var/home/miabbott/openshift-os:/srv/ --device /dev/kvm --device /dev/fuse --tmpfs /tmp -v /var/tmp:/var/tmp --name coreos-assembler --name cosa-again quay.io/coreos-assemblero
[EVENT | QEMU guest is ready for SSH] [ [0;32m  OK   [0m] Started Network Manager Script Dispatcher Service.

Red Hat Enterprise Linux CoreOS 49.84.202109102026-0
  Part of OpenShift 4.9, RHCOS is a Kubernetes native operating system
  managed by the Machine Config Operator (`clusteroperator/machine-config`).

WARNING: Direct SSH access to machines is not recommended; instead,
make configuration changes via `machineconfig` objects:
  https://docs.openshift.com/container-platform/4.9/architecture/architecture-rhcos.html

---
Last login: Tue Sep 14 14:34:40 2021
[core@cosa-devsh ~]$ cat /proc/cmdline | more
BOOT_IMAGE=(hd0,gpt3)/ostree/rhcos-e0dce123559987d6a71c679e6357caebe104c0ed2778092a601fdf6c79f4dc0c/vmlinuz-4.18.0-305.17.1.el8_4.x86_64 random.trust_cpu=on console=tty0 console=ttyS0,115200n8 ignition.platform.id=qemu ignition.firstboot ostree=/ostree/boot.1/rhcos/e0dce123559987d6a71c679e6357caebe104c0ed2778092a601f
df6c79f4dc0c/0 bar=foo

Additionally, trying to combine multiple kargs into a single string causes cosa to choke:

$ coreos-assembler run --kargs "foo=bar bar=foo"
COREOS_ASSEMBLER_CONTAINER_RUNTIME_ARGS=--name cosa-again
BASH_FUNC_coreos-assembler%%=() {  env | grep --color=auto COREOS_ASSEMBLER;
 chrt --idle 0 podman run --rm -ti --security-opt label=disable --privileged --uidmap=1000:0:1 --uidmap=0:1:1000 --uidmap 1001:1001:64536 -v ${PWD}:/srv/ --device /dev/kvm --device /dev/fuse --tmpfs /tmp -v /var/tmp:/var/tmp --name coreos-assembler ${COREOS_ASSEMBLER_CONFIG_GIT:+-v $COREOS_ASSEMBLER_CONFIG_GIT:/srv/src/config/:ro} ${COREOS_ASSEMBLER_GIT:+-v $COREOS_ASSEMBLER_GIT/src/:/usr/lib/coreos-assembler/:ro} ${COREOS_ASSEMBLER_CONTAINER_RUNTIME_ARGS} ${COREOS_ASSEMBLER_CONTAINER:-quay.io/coreos-assembler/coreos-assembler:latest} $@;
+ chrt --idle 0 podman run --rm -ti --security-opt label=disable --privileged --uidmap=1000:0:1 --uidmap=0:1:1000 --uidmap 1001:1001:64536 -v /var/home/miabbott/openshift-os:/srv/ --device /dev/kvm --device /dev/fuse --tmpfs /tmp -v /var/tmp:/var/tmp --name coreos-assembler --name cosa-again quay.io/coreos-assembler/coreos-assembler:latest run --kargs foo=bar bar=foo
Error: Unhandled non-option argument passed for qemu: bar=foo
2021-09-14T14:35:39Z cli: Unhandled non-option argument passed for qemu: bar=foo
+ rc=1
+ set +x

These look like regressions based on the idea of the implementation here:

https://github.com/coreos/coreos-assembler/pull/1263#issuecomment-601426449

...and the actual implementation in https://github.com/coreos/coreos-assembler/pull/1265

miabbott commented 2 years ago

@mike-nguyen points out that multiple kargs can be passed as a string if you are already in the coreos-assembler shell

$ coreos-assembler shell
COREOS_ASSEMBLER_CONTAINER_RUNTIME_ARGS=--name cosa-again
BASH_FUNC_coreos-assembler%%=() {  env | grep --color=auto COREOS_ASSEMBLER;
 chrt --idle 0 podman run --rm -ti --security-opt label=disable --privileged --uidmap=1000:0:1 --uidmap=0:1:1000 --uidmap 1001:1001:64536 -v ${PWD}:/srv/ --device /dev/kvm --device /dev/fuse --tmpfs /tmp -v /var/tmp:/var/tmp --name coreos-assembler ${COREOS_ASSEMBLER_CONFIG_GIT:+-v $COREOS_ASSEMBLER_CONFIG_GIT:/srv/src/config/:ro} ${COREOS_ASSEMBLER_GIT:+-v $COREOS_ASSEMBLER_GIT/src/:/usr/lib/coreos-assembler/:ro} ${COREOS_ASSEMBLER_CONTAINER_RUNTIME_ARGS} ${COREOS_ASSEMBLER_CONTAINER:-quay.io/coreos-assembler/coreos-assembler:latest} $@;
+ chrt --idle 0 podman run --rm -ti --security-opt label=disable --privileged --uidmap=1000:0:1 --uidmap=0:1:1000 --uidmap 1001:1001:64536 -v /var/home/miabbott/openshift-os:/srv/ --device /dev/kvm --device /dev/fuse --tmpfs /tmp -v /var/tmp:/var/tmp --name coreos-assembler --name cosa-again quay.io/coreos-assembler/coreos-assembler:latest shell
[coreos-assembler]$ coreos-assembler run --kargs "foo=bar bar=foo"
[EVENT | QEMU guest is ready for SSH] [ [0;32m  OK   [0m] Started Generate SSH keys snippet f…via console-login-helper-messages.
Red Hat Enterprise Linux CoreOS 49.84.202109102026-0
  Part of OpenShift 4.9, RHCOS is a Kubernetes native operating system
  managed by the Machine Config Operator (`clusteroperator/machine-config`).

WARNING: Direct SSH access to machines is not recommended; instead,
make configuration changes via `machineconfig` objects:
  https://docs.openshift.com/container-platform/4.9/architecture/architecture-rhcos.html

---
[core@cosa-devsh ~]$ cat /proc/cmdline | more
BOOT_IMAGE=(hd0,gpt3)/ostree/rhcos-e0dce123559987d6a71c679e6357caebe104c0ed2778092a601fdf6c79f4dc0c/vmlinuz-4.18.0-305.17.1.el8_4.x86_64 random.trust_cpu=on console=tty0 console=ttyS0,115200n8 ignition.platform.id=qemu ignition.firstboot ostree=/ostree/boot.1/rhcos/e0dce123559987d6a71c679e6357caebe104c0ed2778092a601f
df6c79f4dc0c/0 foo=bar bar=foo
[core@cosa-devsh ~]$

However, multiple uses of --karg don't appear to be honored

$ coreos-assembler shell
COREOS_ASSEMBLER_CONTAINER_RUNTIME_ARGS=--name cosa-again
BASH_FUNC_coreos-assembler%%=() {  env | grep --color=auto COREOS_ASSEMBLER;
 chrt --idle 0 podman run --rm -ti --security-opt label=disable --privileged --uidmap=1000:0:1 --uidmap=0:1:1000 --uidmap 1001:1001:64536 -v ${PWD}:/srv/ --device /dev/kvm --device /dev/fuse --tmpfs /tmp -v /var/tmp:/var/tmp --name coreos-assembler ${COREOS_ASSEMBLER_CONFIG_GIT:+-v $COREOS_ASSEMBLER_CONFIG_GIT:/srv/src/config/:ro} ${COREOS_ASSEMBLER_GIT:+-v $COREOS_ASSEMBLER_GIT/src/:/usr/lib/coreos-assembler/:ro} ${COREOS_ASSEMBLER_CONTAINER_RUNTIME_ARGS} ${COREOS_ASSEMBLER_CONTAINER:-quay.io/coreos-assembler/coreos-assembler:latest} $@;
+ chrt --idle 0 podman run --rm -ti --security-opt label=disable --privileged --uidmap=1000:0:1 --uidmap=0:1:1000 --uidmap 1001:1001:64536 -v /var/home/miabbott/openshift-os:/srv/ --device /dev/kvm --device /dev/fuse --tmpfs /tmp -v /var/tmp:/var/tmp --name coreos-assembler --name cosa-again quay.io/coreos-assembler/coreos-assembler:latest shell
[coreos-assembler]$ coreos-assembler run --kargs foo=bar --kargs bar=foo
[EVENT | QEMU guest is ready for SSH] [ [0;32m  OK   [0m] Started Generate SSH keys snippet f…via console-login-helper-messages.
Red Hat Enterprise Linux CoreOS 49.84.202109102026-0
  Part of OpenShift 4.9, RHCOS is a Kubernetes native operating system
  managed by the Machine Config Operator (`clusteroperator/machine-config`).

WARNING: Direct SSH access to machines is not recommended; instead,
make configuration changes via `machineconfig` objects:
  https://docs.openshift.com/container-platform/4.9/architecture/architecture-rhcos.html

---
[core@cosa-devsh ~]$ cat /proc/cmdline 
BOOT_IMAGE=(hd0,gpt3)/ostree/rhcos-e0dce123559987d6a71c679e6357caebe104c0ed2778092a601fdf6c79f4dc0c/vmlinuz-4.18.0-305.17.1.el8_4.x86_64 random.trust_cpu=on console=tty0 console=ttyS0,115200n8 ignition.platform.id=qemu ignition.firstboot ostree=/ostree/boot.1/rhcos/e0dce123559987d6a71c679e6357caebe104c0ed2778092a601fdf6c79f4dc0c/0 bar=foo
[core@cosa-devsh ~]$
jlebon commented 2 years ago

What does which cosa say? From the output above, it looks like you lost the quotes around "$@" in the alias definition, so it's splitting on the space within the quoted --kargs arg.

For the multiple --kargs part, I opened https://github.com/coreos/coreos-assembler/pull/2431.