search

argoproj / gitops-engine

Democratizing GitOps
https://pkg.go.dev/github.com/argoproj/gitops-engine?tab=subdirectories
Apache License 2.0
1.69k stars 259 forks source link

Outdated static `pkg/utils/kube/scheme/parser.go` causing `field not declared in schema` errors in ArgoCD #558

Closed r0bobo closed 4 months ago

r0bobo commented 11 months ago

I've had two cases of sync errors in ArgoCD:

For CSIDriver:

Failed to compare desired state to live state: failed to perform pre-diff normalization: error building typed results: error creating typedLive: .spec.seLinuxMount: field not declared in schema

For Pod with native sidecar containers and the field .spec.initContainers.restartPolicy:

Failed to compare desired state to live state: failed to perform pre-diff normalization: error building typed results: error creating typedConfig: errors: .spec.template.spec.initContainers[name="helper"].restart: field not declared in schema .spec.template.spec.initContainers[name="podman"].restart: field not declared in schema

After digging around in the code I think the problem might be that the static inlined schema here is outdated and is missing those fields.

Here is a related ArgoCD issue for the CSIDriver problem.

Kubernetes version: v1.28.4 with the SidecarContainers feature gate enabled ArgoCD version: v2.9.3

adberger commented 10 months ago

Any updates on this?

seLinuxMount is in the spec since kubernetes v1.25!

see:

wpi86 commented 8 months ago

Hi, have similar issue, but with Jobs "error building typed results: error creating typedLive: errors: .spec.podReplacementPolicy: field not declared in schema." JobPodReplacementPolicy featuregate now in beta and enabled by default. eks 1.29 + argo v2.10.0+2175939

Typositoire commented 8 months ago

I'm also currently hitting this with Kyverno latest chart and multiple fields, the only way to make the chart compute without having Failed to compare desired state to live state: failed to calculate diff: error calculating structured merge diff: error building typed value from live resource: XYZ: field not declared in schema is to ignore them like so.

ignoreDifferences:
  - group: kyverno.io
    kind: ClusterPolicy
    jqPathExpressions:
      - .status.validatingadmissionpolicy
      - .status.autogen
      - '.spec.rules[].skipBackgroundRequests'
      - .spec.admission
kamelj commented 5 months ago

Hello @wpi86 I have the same issue here, What did you do regarding your error "errors: .spec.podReplacementPolicy: field not declared in schema"?

Hi, have similar issue, but with Jobs "error building typed results: error creating typedLive: errors: .spec.podReplacementPolicy: field not declared in schema." JobPodReplacementPolicy featuregate now in beta and enabled by default. eks 1.29 + argo v2.10.0+2175939

wpi86 commented 5 months ago

Hello @wpi86 I have the same issue here, What did you do regarding your error "errors: .spec.podReplacementPolicy: field not declared in schema"?

Hi, have similar issue, but with Jobs "error building typed results: error creating typedLive: errors: .spec.podReplacementPolicy: field not declared in schema." JobPodReplacementPolicy featuregate now in beta and enabled by default. eks 1.29 + argo v2.10.0+2175939

Hi, currently we keep our jobs as argo hooks, in this case argo doesn't try to validate them on sync and it works

martijnvdp commented 5 months ago

think i got a similar issue with the latest gatekeeper release i got this error even the field is in the scheme k8s 1.28+

helm:

kind: ValidatingWebhookConfiguration
  {{- if ge (int .Capabilities.KubeVersion.Minor) 28 }}
  matchConditions: {{ toYaml .Values.validatingWebhookMatchConditions | nindent 4 }}
  {{- end }}

Failed to compare desired state to live state: failed to perform pre-diff normalization: error building typed results: error creating typedConfig: .webhooks[name="validation.gatekeeper.sh"].matchConditions: field not declared in schema

eks 1.28 argo v2.10.1+a79e0ea

update could fix it by removing

    resource.customizations.ignoreDifferences.all: |
        managedFieldsManagers:
        - kube-controller-manager

from my argocd configmap

ptr1120 commented 4 months ago

I also have a problem with a simple job that breaks the complete deployment.

Sync operation to 3a93a6cc5f459e30b53659ffcd0b8fc9e5df2acb failed: ComparisonError: Failed to compare desired state to live state: failed to perform pre-diff normalization: error building typed results: error creating typedLive: errors:
  .spec.podReplacementPolicy: field not declared in schema
  .status.terminating: field not declared in schema

I tried to fix it with

  ignoreDifferences:
    - kind: Job
      jqPathExpressions:
        - .spec.podReplacementPolicy
        - .status.terminating

without success.

Any idea?

wpi86 commented 4 months ago

I also have a problem with a simple job that breaks the complete deployment.

Sync operation to 3a93a6cc5f459e30b53659ffcd0b8fc9e5df2acb failed: ComparisonError: Failed to compare desired state to live state: failed to perform pre-diff normalization: error building typed results: error creating typedLive: errors:
  .spec.podReplacementPolicy: field not declared in schema
  .status.terminating: field not declared in schema

I tried to fix it with

  ignoreDifferences:
    - kind: Job
      jqPathExpressions:
        - .spec.podReplacementPolicy
        - .status.terminating

without success.

Any idea?

Argo has released a new version that knows this field, the update solved our problem

ptr1120 commented 4 months ago

@wpi86 thank you, but updating argocd to v2.12.0-rc1 for me does not solve the problem with the job, maybe it's additionally due to using server-side-apply (https://github.com/argoproj/argo-cd/issues/17358)

Failed to compare desired state to live state: failed to calculate diff: error calculating structured merge diff: error building typed value from live resource: .spec.podReplacementPolicy: field not declared in schema
ptr1120 commented 4 months ago

The only difference I notice is ignoring field managers in argocd-cm produces 2 errors (spec.podReplacementPolicy, status.terminating):

Failed to compare desired state to live state: failed to perform pre-diff normalization: error building typed results: error creating typedLive: errors: .status.terminating: field not declared in schema .spec.podReplacementPolicy: field not declared in schema

Removing the filed managers ignore rules from argocd-cd produces 1 error (podReplacementPolicy):

Failed to compare desired state to live state: failed to calculate diff: error calculating structured merge diff: error building typed value from live resource: .spec.podReplacementPolicy: field not declared in schema
stongo commented 4 months ago

I'm having the same problem with other fields. Another workaround is to disable validation completely in the sync policy.

syncOptions:
- Validate=false
crenshaw-dev commented 4 months ago

I've updated the static schema to 0.29. Please try Argo CD 2.12.0-rc3. https://github.com/argoproj/gitops-engine/pull/588