Canary metadata removes existing pod metadata on role switch

[huynhsontung]

Checklist:

• [x] I've included steps to reproduce the bug.
• [x] I've included the version of argo rollouts.

Describe the bug

When using the canary deployment strategy, I want to add extra labels to canary pods using only canaryMetadata. If I have the same label in the pod template, when the ReplicaSet switches from canary to stable, that label is removed instead of restored to whatever I define in the pod template.

A workaround is to set the same label in both canaryMetadata and stableMetadata.

To Reproduce

Say I have the following Rollout config:

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: example-rollout
spec:
  template:
    metadata:
      labels:
        role: stable
    [...]
  strategy:
    canary:
      canaryMetadata:
        labels:
          role: canary
    [...]

When canary deployment takes place, pods will have role=canary label as expected. But when those pods get promoted to stable, the role label is removed, instead of restored to role=stable.

The workaround would be to also set stableMetadata.labels.role: stable.

Expected behavior

Canary metadata should not delete labels defined in the pod template upon promotion. In fact, why have a separate stableMetadata at all when the pod template metadata is effectively "stable" metadata?

Version

v1.5.1

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritize the issues with the most 👍.

[meeech]

FWIW This behaviour made sense to me? Pod Template happens first. Then the override labels. I wouldn't expect it to restore the base one that got overwritten.

[huynhsontung]

I guess my assumption was canaryMetadata is a merge over the metadata in the template. So when the pods are promoted, I expect the template metadata to be used as is. In reality, all canary metadata is marked as ephemeral (regardless of whether it is in the template or not). When role switch occurs, these labels are removed. It's the removal part that caught me off guard.

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity.