horizontalpodautoscalers conflicts system generated annotations

[universam1]

Issue that we see, that Kapp seems to be clashing with system generated annotations.

Error:

Operation cannot be fulfilled on horizontalpodautoscalers.autoscaling "some-service": the object has been modified; please apply your changes to the latest version and try again (reason: Conflict)

Potential reason - Kapp detects system generated annotations as diffs and might detect a conflict:

- annotations:                     
-    autoscaling.alpha.kubernetes.io/conditions: '[{"type":"AbleToScale","status":"True","lastTransitionTime":"2019-12-09T21:20:38Z","reason":"ScaleDownStabilized","message":"recent 
-      recommendations were higher than current one, applying the highest recent recommendation"},{"type":"ScalingActive","status":"True","lastTransitionTime":"2020-11-25T11:13:01Z","reason":"ValidMetricFound","message":"the 
-      HPA was able to successfully calculate a replica count from cpu resource utilization 
-      (percentage of request)"},{"type":"ScalingLimited","status":"True","lastTransitionTime":"2020-11-26T08:31:08Z","reason":"TooFewReplicas","message":"the 
-      desired replica count is less than the minimum replica count"}]' 
-    autoscaling.alpha.kubernetes.io/current-metrics: '[{"type":"Resource","resource":{"name":"cpu","currentAverageUtilization":0,"currentAverageValue":"2m"}}]'

Solution: Kapp should ignore these fields

[loewenstein]

Could you achieve what you want using kapp configuration like rebase rules?

https://github.com/vmware-tanzu/carvel-kapp/blob/develop/docs/config.md

[universam1]

Probably yes, but it would require changing over dozens of repositories and training of even more developers. Can’t this be generally improved?

[cppforlife]

@universam1 could you provide minimal config to repro this scenario?

[universam1]

probably related to #213