kubectl diff fails to detect deletions made in manifest after kubectl create

[hlastras]

What happened: When using kubectl create to deploy resources and subsequently modifying the original manifest (specifically deleting an environment variable), kubectl diff does not detect the deletion, although it correctly detects changes (e.g., updating an environment variable value). This behavior contrasts with kubectl apply, where such deletions are detected.

What you expected to happen: I expected kubectl diff to show all differences between the current state of the cluster and the state defined by the modified manifest, including deletions of elements such as environment variables.

How to reproduce it (as minimally and precisely as possible):

cat << EOF > deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: busybox-deployment
  labels:
    app: busybox
spec:
  replicas: 0
  selector:
    matchLabels:
      app: busybox
  template:
    metadata:
      labels:
        app: busybox
    spec:
      containers:
      - name: busybox
        image: busybox
        env:
        - name: TESTENV1
          value: "var1"
        - name: TESTENV2
          value: "var2"
EOF

kubectl create -f deployment.yaml

# Apply some changes on the file (deleting an envvar and updating another)
sed -i '/- name: TESTENV1/{N;d;}' deployment.yaml
sed -i 's/value: "var2"/value: "foo"/' deployment.yaml

kubectl diff -f deployment.yaml

Output:

@@ -4,7 +4,7 @@
   annotations:
     deployment.kubernetes.io/revision: "1"
   creationTimestamp: "2024-04-22T21:04:20Z"
-  generation: 1
+  generation: 2
   labels:
     app: busybox
   name: busybox-deployment
@@ -34,7 +34,7 @@
         - name: TESTENV1
           value: var1
         - name: TESTENV2
-          value: var2
+          value: foo
         image: busybox
         imagePullPolicy: Always
         name: busybox

Expected output:

@@ -6,7 +6,7 @@
   annotations:
     deployment.kubernetes.io/revision: "1"
   creationTimestamp: "2024-04-22T21:04:28Z"
-  generation: 1
+  generation: 2
   labels:
     app: busybox
   name: busybox-deployment
@@ -33,10 +33,8 @@
     spec:
       containers:
       - env:
-        - name: TESTENV1
-          value: var1
         - name: TESTENV2
-          value: var2
+          value: foo
         image: busybox
         imagePullPolicy: Always
         name: busybox

If on the above script I change kubectl create by kubectl apply, the output is as expected.

Anything else we need to know?: This behavior has been consistently reproducible across different environments, suggesting it might be inherent to the current implementation of kubectl diff rather than an environment-specific issue. Apologies in advance if this is the expected behavior, but I could not find any clear documentation explaining this.

Environment:

• Kubernetes client and server versions (use kubectl version):
  • AWS EKS cluster version 1.27 with kubectl version 1.27
  • Kind cluster version 1.27 with kubectl version 1.27
  • Kind cluster version 1.29 with kubectl version 1.29
• Cloud provider or hardware configuration: AWS EKS, Kind clusters
• OS: MacOS Monterey 12.6

[mpuckett159]

/triage accepted

[ardaguclu]

I tested this with the same resource and could have reproduced it when I use create command to create the resources. However, if I follow the same steps with the only difference by using apply instead of create, it worked;

cat << EOF > deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: busybox-deployment
  labels:
    app: busybox
spec:
  replicas: 0
  selector:
    matchLabels:
      app: busybox
  template:
    metadata:
      labels:
        app: busybox
    spec:
      containers:
      - name: busybox
        image: busybox
        env:
        - name: TESTENV1
          value: "var1"
        - name: TESTENV2
          value: "var2"
EOF

kubectl apply -f deployment.yaml

# Apply some changes on the file (deleting an envvar and updating another)
sed -i '/- name: TESTENV1/{N;d;}' deployment.yaml
sed -i 's/value: "var2"/value: "foo"/' deployment.yaml

kubectl diff -f deployment.yaml

 creationTimestamp: "2024-04-26T10:13:54Z"
-  generation: 1
+  generation: 2
   labels:
     app: busybox
   name: busybox-deployment
@@ -33,10 +33,8 @@
     spec:
       containers:
       - env:
-        - name: TESTENV1
-          value: var1
         - name: TESTENV2
-          value: var2
+          value: foo
         image: busybox
         imagePullPolicy: Always
         name: busybox

@hlastras As a sig-cli maintainers, we always recommend to use apply command rather than create. Apply is declarative way of managing resources and always should be the preferred approach.

I'll not close this issue because apparently there is mismatch between create and diff commands (diff command works more aligning with apply) in case someone would want to spend time on it. /unassign

[Coen90]

Hi @ardaguclu I would like to work on this issue.

[Coen90]

/assign