"can not mix '--config' with arguments" common issues about v1beta4 UpgradeConfiguration

[SataQiu]

When I try to upgrade the cluster using kubeadm upgrade apply/plan --config upgrade.yaml (master branch latest code), many flags can not be mixed but are not defined in the config file.

For example, an error will occur when you execute this command kubeadm upgrade apply v1.30.0 --config upgrade.yaml --feature-gates EtcdLearnerMode=true

can not mix '--config' with arguments [feature-gates]
To see the stack trace of this error execute with --v=5 or higher

Other commands and flags may have similar problems that must be checked carefully.

I create this issue to discuss a solution.

There are two ways to handle this:

1. Add these parameters to the corresponding config struct (UpgradeConfiguration)
2. Allow mixed use of these flags with --config

what do you think？@neolit123 @pacoxu

We'd better fix this issue before the new version is released to reduce the problems introduced by v1beta4

Similar with: https://github.com/kubernetes/kubernetes/pull/125779

[neolit123]

i think we need to drop support if any flag that performs a cluster reconfiguration on upgrade. --feature-gates us such a flag. we should deprecate it. the correct way to change fgs is to edit the config map. --yes was recently allowed to mix; we shouldn't add it in config.

what other flags are problematic?

edit: checked https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-upgrade/ and i think we have config coverage for most flags, some flags were discussed as not desired for config 🤔

[neolit123]

cc @calvin0327

[SataQiu]

i think we need to drop support if any flag that performs a cluster reconfiguration on upgrade. --feature-gates us such a flag

Ok, I will send a PR later to deprecate --feature-gates flag for kubeadm upgrade apply/plan

what other flags are problematic?

Maybe not completely, but I found these problems:

# kubeadm upgrade plan --config upgrade.yaml --allow-missing-template-keys xx --output xx --show-managed-fields xx
can not mix '--config' with arguments [allow-missing-template-keys、output、show-managed-fields]

[neolit123]

Maybe not completely, but I found these problems:

# kubeadm upgrade plan --config upgrade.yaml
can not mix '--config' with arguments [all

if we are just passing --config but it complains about some options that we didn't pass, that seems like a bug. please send a PR for that too if possible.

one step in the right direction would be to enable --config for upgrade in kinder workflows. i will take this action item.

[SataQiu]

if we are just passing --config but it complains about some options that we didn't pass, that seems like a bug. please send a PR for that too if possible.

Sorry for the misleading. These flags were passed with --config in my test. I think we should also allow these flags mixed with --config, but don't deprecate them, right? @neolit123

[neolit123]

if we are just passing --config but it complains about some options that we didn't pass, that seems like a bug. please send a PR for that too if possible.

Sorry for the misleading. These flags were passed with --config in my test. I think we should also allow these flags mixed with --config, but don't deprecate them, right? @neolit123

i think these flags come from our usage of the kubectl cmd lib for structured output. --output is our flag. these should be allowed with config.

[tmurakam]

In the Kubespray project, we use the following flags for kubeadm upgrade but they can't be mixed with --config option.

• --allow-experimental-upgrades
• --certificate-renewal
• --etcd-upgrade
• --force
• --yes

How to use these options with --config? It seems that the v1beta3 config does not support the options. Without this, it is hard to support upgrading k8s 1.29 to 1.30 by kubespray.

[neolit123]

In the Kubespray project, we use the following flags for kubeadm upgrade but they can't be mixed with --config option.

• --allow-experimental-upgrades
• --certificate-renewal
• --etcd-upgrade
• --force
• --yes

How to use these options with --config? It seems that the v1beta3 config does not support the options. Without this, it is hard to support upgrading k8s 1.29 to 1.30 by kubespray.

the usage of v1beta3.InitConfiguration|ClusterConfiguration with --config on upgrade has been long deprecated and there was a warning printed to the user for ~10 releases. it's a non-supported and also dangerous form of reconfguration.

it's just not possible to use --config and mix these options, which is the expected behavior. --yes does not exist in v1beta4 so mixing it with --config should be ok, in 1.31.

you should drop your usage of --config on upgrade to 1.30 and 1.31 for non-v1beta4 APIs and use similar actions to reconfigure a kubeadm cluster: https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-reconfigure/

[tmurakam]

@neolit123 Thank you for your answer. I understood.