[RFE] Add webhook customization field to cluster_agent_deployment_customization

[Pursu1tOfHapp1ness]

Is your feature request related to a problem? Please describe.

If you want to customize cluster_agent_deployment_customization in rancher2_cluster resource, you can use these 3 sections:

• append_tolerations
• override_affinity
• override_resource_requirements

But, when you apply needed changes, you can't change anything related to rancher-webhook functionality which is automatically installed by this process with default values. This provides strange feeling that you can change fleet-agent and cattle-cluster-agent, but there is not way how to change anything related to rancher-webhook.

An example with conditions: I want to register existed EKS cluster (a.k.a import it) and place registration process functionality that will be created by rancher2_cluster resource to specific NG with taints. And I also don't have any untainted or unlabeled nodes in the cluster. The TF code will be:

resource "rancher2_cluster" "test" {
  provider = rancher2.admin

  name        = lower(data.aws_eks_cluster.test.name)
  eks_config_v2 {
    cloud_credential_id = data.rancher2_cloud_credential.rancher.id
    name                = data.aws_eks_cluster.test.name
    region              = var.region
    imported            = true
  }
  cluster_agent_deployment_customization {
    append_tolerations {
      key      = "Key"
      value    = "Value"
      operator = "Equal"
    }
    override_affinity = <<EOF
{
  "nodeAffinity": {
    "requiredDuringSchedulingIgnoredDuringExecution": {
      "nodeSelectorTerms": [{
        "matchExpressions": [{
          "key": "Key",
          "operator": "In",
          "values": [
            "Value"
          ]
        }]
      }]
    }
  }
}
EOF
    override_resource_requirements {
      cpu_limit      = "250m"
      cpu_request    = "250m"
      memory_limit   = "1024Mi"
      memory_request = "1024Mi"
    }
  }

  fleet_agent_deployment_customization {
    append_tolerations {
      key      = "Key"
      value    = "Value"
      operator = "Equal"
    }
    override_affinity = <<EOF
{
  "nodeAffinity": {
    "requiredDuringSchedulingIgnoredDuringExecution": {
      "nodeSelectorTerms": [{
        "matchExpressions": [{
          "key": "Key",
          "operator": "In",
          "values": [
            "Value"
          ]
        }]
      }]
    }
  }
}
EOF
    override_resource_requirements {
      cpu_limit      = "250m"
      cpu_request    = "125m"
      memory_limit   = "512Mi"
      memory_request = "256Mi"
    }
  }

  depends_on = [
    data.rancher2_cloud_credential.rancher.id
  ]
}

I will see:

So, to make sure, that everything is finished appropriately, I need to patch manually tolerations and nodeSelector. I guess this shouldn't be done in this way and whole process should be finished through rancher2_cluster functionality.

Describe the solution you'd like

To add to the cluster_agent_deployment_customization:

cluster_agent_deployment_customization {
    override_affinity = ....
    append_tolerations {......}
    override_resource_requirements {....}

    webhook {     <================ New section
        append_tolerations {
           key      = "Key"
           value    = "Value"
           operator = "Operator"
           effect = "Effect"
         }
        override_node_selectors {
          "Key" = "Value"
        }
    }
}

Describe alternatives you've considered

Currently, I don't find any information and any good automated way how to do this and don't create wheel of hell in TF code. Please, mention to me, if you know something more or I am wrong about registration process through rancher2_cluster.

[hameno]

Any update here?