aws / aws-app-mesh-controller-for-k8s

A controller to help manage App Mesh resources for a Kubernetes cluster.
Apache License 2.0
183 stars 110 forks source link

Move appmesh-controller CRDs to structural schema #472

Open CrawX opened 3 years ago

CrawX commented 3 years ago

Is your feature request related to a problem? When using the CRDs defined by the appmesh-controller (such as VirtualNode) with the terraform kubernetes-alpha provider, the provider displays the following warning:

kubernetes_manifest.virtualnode: Creating...
╷
│ Warning: This custom resource does not have an associated OpenAPI schema.
│
│   with kubernetes_manifest.virtualnode,
│   on configserver.tf line 247, in resource "kubernetes_manifest" "virtualnode":
│  247: resource "kubernetes_manifest" "virtualnode" {
│
│ We could not find an OpenAPI schema for this custom resource. Updates to this resource will cause a forced replacement.

My definition looks like this:

resource "kubernetes_manifest" "virtualnode" {
  provider = kubernetes-alpha

  manifest = {
    apiVersion = "appmesh.k8s.aws/v1beta2"
    kind = "VirtualNode"
    metadata = {
      name = local.app_name
      namespace = local.namespace
    }
    ...
  }
}

When using other CRDs (such as Certificate from cert-manager), this warning does not surface.

I tried to narrow it down and it seems like the CRDs by cert-manager are defined with version apiextensions.k8s.io/v1 while those of the appmesh.controller are defined with version apiextensions.k8s.io/v1beta1. According to these docs, apiextensions.k8s.io/v1beta1 is used for non-structural schemas.

My analysis might be way off here, this is just something that I noticed while trying to debug it. I have no idea how complex this adjustment would be, if someone can point me in the right direction I might start hacking around with it.

Describe the solution you'd like The appmesh-controller CRDs should use structural CRDs so kubernetes-alpha can fully support them which enables app mesh configuration from terraform scripts.

Describe alternatives you've considered banzaiclouds terraform k8s provider does seem to work better but can't handle changes in the definitions very well. It complains about "awsName" being read-only even though its not being changed. I'd prefer to use the kubernetes-alpha provider (which has better support for waiting on resources and should eventually reach GA at some point...)

Shreya027 commented 2 years ago

/assign Shreya027