Closed annabarnes1138 closed 3 years ago
I am having the same issue and tried to workaround it without success.
First thing i did was trying to ignore-missing-schemas in the kubeval command scripts/validate.sh
kubeval ${file} **--ignore-missing-schemas** --strict --additional-schema-locations=file:///tmp/flux-crd-schemas
After doing that I am able to progress yet having the next issue.
Run ./scripts/validate.sh
INFO - Downloading Flux OpenAPI schemas
INFO - Validating ./clusters/staging/flux-system/gotk-components.yaml
INFO - Validating ./clusters/staging/flux-system/kustomization.yaml
INFO - Validating ./clusters/staging/flux-system/gotk-sync.yaml
INFO - Validating ./clusters/staging/infrastructure.yaml
INFO - Validating ./clusters/staging/apps.yaml
INFO - Validating ./clusters/production/infrastructure.yaml
INFO - Validating ./clusters/production/apps.yaml
INFO - Validating ./.github/workflows/test.yaml
INFO - Validating ./.github/workflows/e2e.yaml
INFO - Validating ./infrastructure/kustomization.yaml
INFO - Validating ./infrastructure/sources/kustomization.yaml
INFO - Validating ./infrastructure/sources/podinfo.yaml
INFO - Validating ./apps/base/podinfo/release.yaml
INFO - Validating ./apps/base/podinfo/kustomization.yaml
INFO - Validating ./apps/base/podinfo/namespace.yaml
INFO - Validating ./apps/staging/kustomization.yaml
INFO - Validating ./apps/staging/podinfo-values.yaml
INFO - Validating ./apps/production/kustomization.yaml
INFO - Validating ./apps/production/podinfo-values.yaml
INFO - Validating clusters
WARN - Set to ignore missing schemas
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid Namespace (flux-system)
WARN - ./clusters/staging/flux-system/gotk-components.yaml containing a CustomResourceDefinition (alerts.notification.toolkit.fluxcd.io) was not validated against a schema
WARN - ./clusters/staging/flux-system/gotk-components.yaml containing a CustomResourceDefinition (buckets.source.toolkit.fluxcd.io) was not validated against a schema
WARN - ./clusters/staging/flux-system/gotk-components.yaml containing a CustomResourceDefinition (gitrepositories.source.toolkit.fluxcd.io) was not validated against a schema
WARN - ./clusters/staging/flux-system/gotk-components.yaml containing a CustomResourceDefinition (helmcharts.source.toolkit.fluxcd.io) was not validated against a schema
WARN - ./clusters/staging/flux-system/gotk-components.yaml containing a CustomResourceDefinition (helmreleases.helm.toolkit.fluxcd.io) was not validated against a schema
WARN - ./clusters/staging/flux-system/gotk-components.yaml containing a CustomResourceDefinition (helmrepositories.source.toolkit.fluxcd.io) was not validated against a schema
WARN - ./clusters/staging/flux-system/gotk-components.yaml containing a CustomResourceDefinition (kustomizations.kustomize.toolkit.fluxcd.io) was not validated against a schema
WARN - ./clusters/staging/flux-system/gotk-components.yaml containing a CustomResourceDefinition (providers.notification.toolkit.fluxcd.io) was not validated against a schema
WARN - ./clusters/staging/flux-system/gotk-components.yaml containing a CustomResourceDefinition (receivers.notification.toolkit.fluxcd.io) was not validated against a schema
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid ServiceAccount (flux-system.helm-controller)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid ServiceAccount (flux-system.kustomize-controller)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid ServiceAccount (flux-system.notification-controller)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid ServiceAccount (flux-system.source-controller)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid ClusterRole (crd-controller-flux-system)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid ClusterRoleBinding (cluster-reconciler-flux-system)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid ClusterRoleBinding (crd-controller-flux-system)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid Service (flux-system.notification-controller)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid Service (flux-system.source-controller)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid Service (flux-system.webhook-receiver)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid Deployment (flux-system.helm-controller)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid Deployment (flux-system.kustomize-controller)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid Deployment (flux-system.notification-controller)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid Deployment (flux-system.source-controller)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid NetworkPolicy (flux-system.allow-scraping)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid NetworkPolicy (flux-system.allow-webhooks)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid NetworkPolicy (flux-system.deny-ingress)
WARN - Set to ignore missing schemas
ERR - ./clusters/staging/flux-system/kustomization.yaml: Missing 'metadata' key
Error: Process completed with exit code 1.
As you can see it detects missing metadata in the kustomization.yaml...
I then added the a metadata field in the kustomization.yaml
metadata:
name: flux
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
metadata:
name: flux
resources:
- gotk-components.yaml
- gotk-sync.yaml
After doing this I've got and exit 1 and the same error from the begining ;).
PASS - ./clusters/production/apps.yaml contains a valid Kustomization (flux-system.apps)
INFO - Validating kustomize overlays
INFO - Validating kustomization ./clusters/staging/flux-system/
ERR - stdin: Failed initializing schema https://kubernetesjsonschema.dev/master-standalone-strict/customresourcedefinition-apiextensions-v1.json: Could not read schema from HTTP, response status is 404 Not Found
Failed initializing schema file:///tmp/flux-crd-schemas/master-standalone-strict/customresourcedefinition-apiextensions-v1.json: open /tmp/flux-crd-schemas/master-standalone-strict/customresourcedefinition-apiextensions-v1.json: no such file or directory
Error: Process completed with exit code 1.
Then I tried downloading the missing schema
curl -sL -o /tmp/flux-crd-schemas/master-standalone-strict/customresourcedefinition-apiextensions-v1.json https://raw.githubusercontent.com/instrumenta/kubernetes-json-schema/master/master-local/customresourcedefinition-apiextensions-v1.json
I also had to add the following base dependency of the schema.
curl -sL -o /tmp/flux-crd-schemas/master-standalone-strict/_definitions.json https://raw.githubusercontent.com/instrumenta/kubernetes-json-schema/master/master-local/_definitions.json
Then I had an exit 1 with no more details.
WARN - ./clusters/staging/flux-system/gotk-components.yaml contains an invalid CustomResourceDefinition (helmreleases.helm.toolkit.fluxcd.io) - metadata.creationTimestamp: Invalid type. Expected: string, given: null
WARN - ./clusters/staging/flux-system/gotk-components.yaml contains an invalid CustomResourceDefinition (helmrepositories.source.toolkit.fluxcd.io) - metadata.creationTimestamp: Invalid type. Expected: string, given: null
WARN - ./clusters/staging/flux-system/gotk-components.yaml contains an invalid CustomResourceDefinition (kustomizations.kustomize.toolkit.fluxcd.io) - metadata.creationTimestamp: Invalid type. Expected: string, given: null
WARN - ./clusters/staging/flux-system/gotk-components.yaml contains an invalid CustomResourceDefinition (providers.notification.toolkit.fluxcd.io) - metadata.creationTimestamp: Invalid type. Expected: string, given: null
WARN - ./clusters/staging/flux-system/gotk-components.yaml contains an invalid CustomResourceDefinition (receivers.notification.toolkit.fluxcd.io) - metadata.creationTimestamp: Invalid type. Expected: string, given: null
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid ServiceAccount (flux-system.helm-controller)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid ServiceAccount (flux-system.kustomize-controller)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid ServiceAccount (flux-system.notification-controller)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid ServiceAccount (flux-system.source-controller)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid ClusterRole (crd-controller-flux-system)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid ClusterRoleBinding (cluster-reconciler-flux-system)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid ClusterRoleBinding (crd-controller-flux-system)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid Service (flux-system.notification-controller)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid Service (flux-system.source-controller)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid Service (flux-system.webhook-receiver)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid Deployment (flux-system.helm-controller)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid Deployment (flux-system.kustomize-controller)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid Deployment (flux-system.notification-controller)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid Deployment (flux-system.source-controller)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid NetworkPolicy (flux-system.allow-scraping)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid NetworkPolicy (flux-system.allow-webhooks)
PASS - ./clusters/staging/flux-system/gotk-components.yaml contains a valid NetworkPolicy (flux-system.deny-ingress)
Error: Process completed with exit code 1.
After that i removed the --ignore-missing-schemas from the kubeval command in scripts/validate.sh
as the dependencies should be already in place.
kubeval ${file} --strict --additional-schema-locations=file:///tmp/flux-crd-schemas
I got the same result. There weren't schema missing errors yet the exit code 1.
There were some warnings in the gotk-components.yaml
file flux bootstrap
pushes into clusters/staging/flux-system
.
These are the warnings i got.
WARN - ./clusters/staging/flux-system/gotk-components.yaml contains an invalid CustomResourceDefinition (alerts.notification.toolkit.fluxcd.io) - metadata.creationTimestamp: Invalid type. Expected: string, given: null
WARN - ./clusters/staging/flux-system/gotk-components.yaml contains an invalid CustomResourceDefinition (buckets.source.toolkit.fluxcd.io) - metadata.creationTimestamp: Invalid type. Expected: string, given: null
WARN - ./clusters/staging/flux-system/gotk-components.yaml contains an invalid CustomResourceDefinition (gitrepositories.source.toolkit.fluxcd.io) - metadata.creationTimestamp: Invalid type. Expected: string, given: null
WARN - ./clusters/staging/flux-system/gotk-components.yaml contains an invalid CustomResourceDefinition (helmcharts.source.toolkit.fluxcd.io) - metadata.creationTimestamp: Invalid type. Expected: string, given: null
WARN - ./clusters/staging/flux-system/gotk-components.yaml contains an invalid CustomResourceDefinition (helmreleases.helm.toolkit.fluxcd.io) - metadata.creationTimestamp: Invalid type. Expected: string, given: null
WARN - ./clusters/staging/flux-system/gotk-components.yaml contains an invalid CustomResourceDefinition (helmrepositories.source.toolkit.fluxcd.io) - metadata.creationTimestamp: Invalid type. Expected: string, given: null
WARN - ./clusters/staging/flux-system/gotk-components.yaml contains an invalid CustomResourceDefinition (kustomizations.kustomize.toolkit.fluxcd.io) - metadata.creationTimestamp: Invalid type. Expected: string, given: null
WARN - ./clusters/staging/flux-system/gotk-components.yaml contains an invalid CustomResourceDefinition (providers.notification.toolkit.fluxcd.io) - metadata.creationTimestamp: Invalid type. Expected: string, given: null
WARN - ./clusters/staging/flux-system/gotk-components.yaml contains an invalid CustomResourceDefinition (receivers.notification.toolkit.fluxcd.io) - metadata.creationTimestamp: Invalid type. Expected: string, given: null
I bootstrapped an existing k8s cluster with the staging config. After it committed the flux-system files the test workflow started failing. Here is the pertinent part of the log
It seems to be failing at line 45:
kubeval ${file} --strict --additional-schema-locations=file:///tmp/flux-crd-schemas
Any suggestions?