Cluster updates

[danisla]

• Update the Selkies CRD to use API v1.
• Use CNRM GKE addon instead of manual install.
• Improve custom domain and load balancer support.

[videlanicolas]

I checked out this branch and re-deployed my infrastructure. It seems apiextensions.k8s.io/v1 is not getting picked up:

Step #1 - "deploy-manifests": Installing CRDs
Step #1 - "deploy-manifests": Applying deployment.
Step #1 - "deploy-manifests": Getting access to cluster "broker-us-west1" in "us-west1".
Step #1 - "deploy-manifests": Configuration files to be used: [{kind: CustomResourceDefinition, name: istiocontrolplanes.install.istio.io}]
Step #1 - "deploy-manifests": Applying configuration files to cluster.
Step #1 - "deploy-manifests": error: unable to recognize "STDIN": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
Step #1 - "deploy-manifests": Error: failed to apply deployment: failed to apply CustomResourceDefinition configuration file with name "istiocontrolplanes.install.istio.io" to cluster: failed to apply config from string: command to apply kubernetes config from string to cluster failed: exit status 1
Finished Step #1 - "deploy-manifests"
ERROR
ERROR: build step 1 "gcr.io/REDACTED/broker-installer" failed: step exited with non-zero status: 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

BUILD FAILURE: Build step failure: build step 1 "gcr.io/REDACTED/broker-installer" failed: step exited with non-zero status: 1
ERROR: (gcloud.builds.submit) build 760aef79-1cd9-4ff7-a718-5ec795db05ad completed with status "FAILURE"

I tried to manually apply the .yaml file:

$ gke-deploy apply --project ${PROJECT_ID} --cluster broker-us-west1 --location us-west1 --filename base/pod-broker/crd.yaml
Applying deployment.
Getting access to cluster "broker-us-west1" in "us-west1".
Configuration files to be used: [{kind: CustomResourceDefinition, name: brokerappconfigs.gcp.solutions} {kind: CustomResourceDefinition, name: brokerappuserconfigs.gcp.solutions}]
Applying configuration files to cluster.
The CustomResourceDefinition "brokerappconfigs.gcp.solutions" is invalid:
* spec.validation.openAPIV3Schema.properties[spec].properties[nodeTiers].items.properties[resources].properties[limits].properties[cpu].type: Required value: must not be empty for specified object fields
* spec.validation.openAPIV3Schema.properties[spec].properties[nodeTiers].items.properties[resources].properties[requests].properties[cpu].type: Required value: must not be empty for specified object fields
Error: failed to apply deployment: failed to apply CustomResourceDefinition configuration file with name "brokerappconfigs.gcp.solutions" to cluster: failed to apply config from string: command to apply kubernetes config from string to cluster failed: exit status 1

It seems you need to specify type for openAPIV3Schema.properties[spec].properties[nodeTiers].items.properties[resources].properties[requests].properties[cpu]Probably type: object on both cases? I used that and it worked.

[videlanicolas]

/opt/istio-operator/deploy/crds/istio_v1alpha2_istiocontrolplane_crd.yaml also depends on apiextensions.k8s.io/v1beta1, so we should update to the latest istio.

[danisla]

@videlanicolas thanks, I think the CRD schema should be now.

Working on updating istio now, it will be in the next PR.