Add details for flags/settings that will migrate over to MKE 4 for Custom Kube Config

The purpose of this issue ticket is to improve the writeup for the migration for Custom Kube Config. Either this writeup can go in https://mirantis.github.io/mke-docs/docs/migrate-from-mke-3/ or https://mirantis.github.io/mke-docs/docs/configuration/kubernetes/. The development epic is based on https://mirantis.jira.com/browse/BOP-515.

The following should migrate over from MKE 3 to 4 with respect to Custom Kube Config (confirmed as per the testing effort):

CustomKubeAPIServerFlags
CustomKubeSchedulerFlags
CustomKubeControllerManagerFlags
MKE 3 Node/Worker Profiles
MKE 3 Manager Settings (there are about 20~30 toml values which were coded for)

MKE 3 Toml Snippets relevant for points 1 to 3:

  custom_kube_api_server_flags = [
      "--enable-admission-plugins=PodSecurity"
  ]
  custom_kube_scheduler_flags = [
    "--authentication-token-webhook-cache-ttl=20s",
    "--authorization-webhook-cache-authorized-ttl=20s",
    "--v=2",
    "--tls-min-version=VersionTLS13"
  ]
  custom_kube_controller_manager_flags = [
    "--attach-detach-reconcile-sync-period=1m30s",
    "--authentication-token-webhook-cache-ttl=20s",
    "--cluster-name=my_mke"
  ]

All possible options and values for this section can be found in: https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/ https://kubernetes.io/docs/reference/command-line-tools-reference/kube-scheduler/ https://kubernetes.io/docs/reference/command-line-tools-reference/kube-scheduler/

MKE 3 Toml Snippet relevant for point 4: https://docs.mirantis.com/mke/3.7/ops/administer-cluster/custom-kubelet-profiles/add-kubelet-node-profiles.html

MKE 3 Toml Snippet relevant for point 5 (see https://docs.mirantis.com/mke/3.7/ops/administer-cluster/configure-an-mke-cluster/configuration-options.html): etcd_storage_quota = "100Mi" kube_api_server_profiling_enabled = true kube_controller_manager_profiling_enabled = true kube_scheduler_profiling_enabled = true kube_scheduler_bind_to_all = true cipher_suites_for_kube_api_server = ["TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"] cipher_suites_for_kubelet = ["TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"] cipher_suites_for_etcd_server = ["TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"] kube_api_server_auditing = true kube_api_server_audit_log_maxage = 30 kube_api_server_audit_log_maxbackup = 10 kube_api_server_audit_log_maxsize = 100 kube_api_server_request_timeout = "30s" kube_manager_terminated_pod_gc_threshold = 100 k8s_event_rate_limit = 5 k8s_always_pull_images_ac_enabled = true unauthenticated_http2_dos_mitigation = true cipher_suites_for_kubelet = "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256" kubelet_max_pods = 110 kubelet_pods_per_core = 10 kubelet_data_root = "/var/lib/kubelet" windows_kubelet_data_root = "C:\var\lib\kubelet" kubelet_event_record_qps = 5 custom_kubelet_flags = ["--cgroup-driver=systemd", "--rotate-certificates=true"] custom_kubelet_flags_profiles = { "high-performance" = "--cpu-manager-policy=static", "default" = "--memory-manager-policy=none" } and cipher_suites_for_kubelet = "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256" kubelet_max_pods = 110 kubelet_pods_per_core = 10 kubelet_data_root = "/var/lib/kubelet" windows_kubelet_data_root = "C:\var\lib\kubelet" kubelet_event_record_qps = 5 custom_kubelet_flags = ["--cgroup-driver=systemd", "--rotate-certificates=true"] custom_kubelet_flags_profiles = { "high-performance" = "--cpu-manager-policy=static", "default" = "--memory-manager-policy=none" }

Point 5 confirmed using: https://github.com/MirantisContainers/mke/pull/270, https://github.com/MirantisContainers/mke/pull/273, https://github.com/Mirantis/orca/blob/master/types/node_config.go, and https://docs.mirantis.com/mke/3.7/ops/administer-cluster/configure-an-mke-cluster/configuration-options.html

Mirantis / mke-docs

Add details for flags/settings that will migrate over to MKE 4 for Custom Kube Config #195