gardener-robot-ci-2
commented
10 months ago nt status).
```other operator github.com/gardener/etcd-druid #575 @aaronfern
etcd-custom-image updates from `v3.4.13-bootstrap-9` to `v3.4.13-bootstrap-10`The `shootstate-extensions` and `shootstate-secret` controllers have been dropped. The `gardenlet`'s component config file should be updated to no longer specify related configuration (`.controllers.{shootSecret,shootStateSync}`).Following dependencies are updated:
Go - 1.20.3
client-go - v0.26.2
controller-runtime - v0.14.5
gomega - v1.27.1
zap - v1.24.0
gardener/gardener v1.69.0
k8s (api and apimachinery) - v0.26.2Test-machinery integration tests are now using upstream K8s e2e test images such as `registry.k8s.io/e2e-test-images/busybox`, `registry.k8s.io/e2e-test-images/agnhost` instead Gardener images such as `eu.gcr.io/gardener-project/3rd/busybox`, `eu.gcr.io/gardener-project/3rd/alpine` and others.Since `Namespace`s are no longer deleted (and forcefully finalized after some grace period), the `shoot.gardener.cloud/cleanup-namespaces-finalize-grace-period-seconds` annotation does no longer have any effect. Relevant Kubernetes resources are still cleaned up (see [this document](https://github.com/gardener/gardener/blob/master/docs/usage/shoot_cleanup.md)) for more information.Gardener sets [`minDomains`](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/#spread-constraint-definition) for shoot system components to the number of zones configured in the system component worker pool(s).
⚠️ It is strongly recommended to place at least one worker node per availability zone for system component workers in order to ensure hitch-free rolling updates and scheduling of pods. You may need to adjust the `maximum: <number-of-zones>` values of your system component worker pool(s).
This configuration only takes effect for clusters which enabled feature gate `MinDomainsInPodTopologySpread` (enabled by default as of Kubernetes `v1.27`).The shoot namespace in seeds is redeployed during the shoot migration flow to update the zones in use.`nginx-ingress-controller-seed` image is updated to `v1.7.1` for `1.24.x+` seeds.When deploying this version of `gardener-operator`, make sure that you update your `Garden` resources with the new `.spec.virtualCluster.gardener.clusterIdentity` field. If you already have a `gardener-apiserver` deployment, make sure that the value matches the `--cluster-identity` flag of the current `gardener-apiserver` deployment.A bug is fixed that rendered the "CPU usage" panel of the "VPN" Plutono dashboard blank.The `--node-monitor-grace-period` flag of `kube-controller-manager` is now defaulted to `40s` for Shoot clusters using Kubernetes version `1.27` and higher.The `Shoot` maintenance controller now updates the CRI of worker pools from `docker` to `containerd` when force-upgrading from Kubernetes `v1.22` to `v1.23`.Introduced `delta-snapshot-retention-period` CLI flag to extend the configurable retention period for delta snapshots in `etcd-backup-restore`, enhancing flexibility for backup retention.The plutono dashboards are now verified as part of `make check`.Update gardener/gardener to 1.77.1.An issue has been fixed which was causing a broken `ControlPlaneHealthy` condition report for `Shoot`s when the `MachineControllerManagerDeployment` feature gate gets enabled until their next reconciliation.If `gardenlet` is responsible for a managed `Seed`, it will delete all `ShootState` resources for its `Shoot`s that are not currently in migration. See also [GEP-22](https://github.com/gardener/gardener/blob/master/docs/proposals/22-improved-usage-of-shootstate-api.md) for further details about the motivation.The certificate chains served by `kube-apiserver`s does now include the CA certificates used to sign their server certificates.A bug causing the managedseed controller to error if the controller restarts and the seed secret is already deleted is now fixed.Add CVE categorization for etcd-backup-restore.Backup-restore waits for its etcd to be ready before attempting to update peerUrlGardener base image is updated to `gcr.io/distroless/static-debian12:nonroot`.It is now possible to reference `Secret`s containing kubeconfigs for admission plugins in `Shoot`s. The referenced `Secret` must be referenced in`.spec.resources` as well as in `.spec.kubernetes.kubeAPIServer.admissionPlugins[].kubeconfigSecretName`.gardener-apiserver now exposes a new `core.gardener.cloud/v1beta1.InternalSecret` API, see the [documentation](https://github.com/gardener/gardener/blob/master/docs/concepts/apiserver.md#internalsecrets) for more information.Gardener Scheduler's Minimal Distance strategy can take scheduling decisions based on region distances configured by operators. This especially improves the allocation for shoots of providers regions for which the standard Levenshtein distance is inappropriate. Please see `docs/concepts/scheduler.md` for more information.github.com/gardener/etcd-druid #714 @aaronfern
Alpine image used in init containers is now part of the IMAGEVECTOR_OVERWRITESeveral low timeouts (30s) that were introduced in v1.71.0 for several steps are now reverted as in some cases the Network/ControlPlane reconciliation cannot succeed for 30s.Added an example for `AdminKubeconfigRequest` via the Python Kubernetes client.Gardener can now support clusters with Kubernetes version 1.28. In order to allow creation/update of 1.28 clusters you will have to update the version of your provider extension(s) to a version that supports 1.28 as well. Please consult the respective releases and notes in the provider extension's repository.The `pkg/utils/gardener.IntStrPtrFromInt` function has been renamed to `IntStrPtrFromInt32` since `intstr.FromInt` is deprecated.It is required to have `ControllerRegistrations`s for Kinds `ControlPlane`, `Infrastructure` and `Worker` with the same types used for seeds (`seed.spec.provider.type`). This is already the case if seeds and shoots share the same cloud provider. The seed reconciliation flow waits for the associated `ControllerInstallation` to be ready before continuing rolling out seed system components. It allows Gardener provider extensions to ship components that not only act on shoot control-plane but also on seed system components.Bump builder image golang from `1.20.2` to `1.20.4`The GA-ed feature gates `SeedChange` and `CopyEtcdBackupsDuringControlPlaneMigration` have been removed.Gardener now deploys the [cluster-autoscaler](https://github.com/gardener/autoscaler) earlier in the shoot reconciliation flow without checking if the worker pools are ready.Improves client recreate during cluster reconcile.Etcd-druid will now deploy distroless `etcd-wrapper` and `etcd-backup-restore` images. Please refer to [etcd-wrapper](https://github.com/gardener/etcd-wrapper) for more information.The kind clusters are now unified to use `garden.local.gardener.cloud` DNS name in the containerd config when configuring registry mirror hostnames. Previously, to access the pull through registry cache some kind clusters were configured to use `garden.local.gardener.cloud`, others - the Node name of the control plane Node.The `.spec.kubernetes.kubeAPIServer.serviceAccountConfig.acceptedIssuers` field of the `Shoot` spec no longer allows duplicate values.Base image on `telegraf` and `tune2fs` is upgraded from 3.17.2 to 3.18.0`leader-election-resource-lock` flag is dropped and the leader-election resource-lock is hard coded to leases.Resolved an issue where the Custodian Controller was not updating the `Replicas` field in the `etcd` status to reflect the `CurrentReplicas` from the StatefulSet status. This fix ensures consistent behavior with the `etcd` Controller in Druid.The `kube-apiserver` no longer mounts root CA bundles from the underlying host.Add new grafana dashboard of seed deployment replicas Eliminated `Role` helm charts and converted into Golang component with added unit tests.Prevent nil pointer exceptions on shoot deletion in `gardenlet` when seed namespace is gone.The `gardener-operator` does now also manage `kube-state-metrics`.Gardener now uses 3072 bit RSA keys in order to generate TLS certificates.Updated golang container image build version to 1.20.4The following dependencies are updated:
- `k8s.io/*` : `v0.26.4` -> `v0.27.5`
- `sigs.k8s.io/controller-runtime`: `v0.14.6` -> `v0.15.2`Etcd-backup-restore now uses the user home directory to create files.When scaling from single-node to multi-node etcd cluster, Etcd Druid will now first ensure that any change to the peer URL (e.g TLS enablement) is seen by the existing etcd process running within the etcd member pod. Once that is confirmed then it will scale up the Etcd StatefulSet and add relevant annotations.A `generate-admin-kubeconf.sh` script which can be used to generate an admin kubeconfig for a local shoot cluster was added in the `hack/usage` directory.The garbage collection controller now also considers managed resources when deciding if secrets/configmaps should be garbage collected.Operators can now use the annotation `gardener.cloud/operation=rotate-observability-credentials` on the `garden` resource to rotate the observability credentials. Update `vertical-pod-autoscaler` to `v0.14.0`.Add a learner with backoff in case of scale-up feature is triggered.Updated go to 1.20.7Add new flag `metrics-scrape-wait-duration` for compaction controller to set a wait duration at the end of every compaction job, to allow for metrics to be scraped by a Prometheus instance.Gardener-based e2e test for the event-logger.`gardener-resource-manager`'s `system-components-config` webhook no longer adds the toleration for the `ToBeDeletedByClusterAutoscaler` taint to system components in shoot clusters. The `ToBeDeletedByClusterAutoscaler` taint is maintained by the `cluster-autoscaler`. This was breaking `cluster-autoscaler`'s drain mechanism when scaling down an under-utilized node. It was causing just evicted system components from to be deleted node to be scheduled again on the to be deleted node.Multiple expanders for `cluster-autoscaler` can now be specified in the `Shoot` API via the `.spec.kubernetes.clusterAutoscaler.expander` field.When `Shoot`s were updated from non high-availability to `zone` high-availability, it could happen that the control-plane was scheduled to two instead of three zones. This issue is relevant for cloud providers with an inconsistent zone naming (`Azure` is currently the only candidate to our knowledge).
Existing shoots with the before mentioned problem must be fixed manually be operators if required. An automatic move of `etcd`s and their volumes is not part of this fix due to availability reasons.A new field `.spec.virtualCluster.dns.domains` was added to the `Garden` API. This field allows to expose the `kube-apiserver` of the virtual cluster via multiple domains. Earlier, the API only accepted one domain name via `.spec.virtualCluster.dns.domain`.
⚠️ With this change `.spec.virtualCluster.dns.domain` is deprecated and will be removed in the next release. Please update your `Garden` resource to the new `.spec.virtualCluster.dns.domains` field by removing the existing domain configuration from `dns.domain` and add it as the first entry of `dns.domains`.When Seed's `spec.settings.ownerChecks.enabled=false` gardenlet is now able to delete the owner DNSRecord for a Shoot stuck in deletion where the kube-apiserver Deployment is missing but the Infrastructure is present and cannot be deleted for some reason (infrastructure dependency, invalid credentials).Etcd druid will now not support `policy/v1beta1` for `PodDisruptionBudget`s and will only use `policy/v1` for `PodDisruptionBudget`sGo version is updated to 1.20.6.Gardener autoscaler now backs-off early from a node-group (i.e. machinedeployment) in case of `ResourceExhausted` error. Refer docs at `https://github.com/gardener/autoscaler/blob/machine-controller-manager-provider/cluster-autoscaler/FAQ.md#when-does-autoscaler-backs-off-early-from-a-node-group` for details.The feature gates `FullNetworkPolicies` and `HAControlPlanes` have been promoted to GA and are now locked to "unconditionally enabled".The `core/v1alpha1` API version is dropped. Before upgrading to this version, make sure that there are no resources in the etcd stored in the `core/v1alpha1` API version. Otherwise, the gardener-apiserver@v1.72.0 will fail to start.`Shoot`s allow to optionally configure a specific scheduler via `.spec.schedulerName`. The `default-scheduler` is used in case non is configured. Please note, that `Shoot`s will remain `Pending` in case a scheduler name is configured but an adequate scheduler is not available in the landscape.The gardenlet and the gardener-operator will now use the new `service.kubernetes.io/topology-mode=auto` annotation when enabling topology-aware routing for a Service when the Kubernetes version of the runtime cluster is >= 1.27. In Kubernetes 1.27, the `service.kubernetes.io/topology-aware-hints=auto` annotation is deprecated in favor of the newly introduced `service.kubernetes.io/topology-mode=auto`Annotations in `seed.spec.settings.loadBalancerServices.annotations` are now applied to the Nginx-Ingress load balancer service in the seed cluster.A bug has been fixed which caused `ServiceAccount`s related to garden access secrets for extensions to leak in the seed namespace in the garden cluster after uninstallation of said extensions.Prevent fluent-bit-to-vali plugin panic when Cluster is updated and its Shoot has no lastOperation setGardener now supports seed clusters with Kubernetes versions up to `v1.26`.`uncachedObjects` under pkg/client/kubernetes/options.go is now removed from Config struct which is used to set options for new ClientSets. Now the uncached objects can be directly set under `clientOptions.Cache.DisableFor` field.The field `.spec.secretRef` in the `Seed` API has been deprecated and will be removed in a future release of Gardener.A bug where MCM removed a machine other than the one , CA wanted , is resolved.The deprecated feature gate `APIServerSNI` has been removed.Etcd snapshot compaction jobs will now be named `<etcd-name>-compactor` for better readability for human operators.Methods `SkipIf` and `DoIf` for `TaskFn` have been dropped. A new field `SkipIf` is introduced in `Task`, If set to true the task will be skipped and will also not be reported by the progress reporter.Bump alpine base version for Docker build to `3.18.2`.Deactivate leader election, health and readiness checks when running `make *-debug.`A regression was fixed that prevented deletions for shoot clusters which were created with a wrong configuration (e.g. with an unavailable domain name).Revendors the bbolt from `v1.3.6` to `v1.3.7`Support for `nip.io` shoot domains is discontinued.Introduces a skaffold local development pipeline to fluent-bit-vali-pluginFixes for `make check` target Allow the kubelet configuration to define swap behaviour {LimitedSwap / UnlimitedSwap} for k8s >= 1.22Updated cluster-proportional-autoscaler to v1.8.8A bug has been fixed which was causing the garbage collector in `gardener-resource-manager` to wrongfully collect `Secret`s related to `ManagedResource`s when the source and the target cluster are equal.Added a LeaderElectionID to the controller options, allowing to run multiple instances of HVPA with leader election when `--leader-elect=true` is passed as commandline argThere is now a new script (`hack/check-skaffold-deps-for-binary.sh`) that can be used by gardener extensions to validate their skaffold ko dependencies.Webhooks remediator sets the timeoutSeonds to 3 seconds for webhook affecting lease resources in `kube-system` namespace only if there is no objectSelector provided in webhook.An issue causing panic in the health check for extension, when the health check result is empty, is fixed.A guideline for developers regarding [`TODO` statements](https://github.com/gardener/gardener/blob/docs/master/development/process.md#todo-statements) has been introduced.Add `Care` reconciler to `Garden` controller in `gardener-operator`. Fix an issue, where DNS lookups for non-existing pods of a StatefulSet yielded one of the existing pods even when it should not have. The following images are updated:
- registry.k8s.io/metrics-server/metrics-server: v0.6.3 -> v0.6.4
- registry.k8s.io/cpa/cluster-proportional-autoscaler: v1.8.8 -> v1.8.9
- registry.k8s.io/coredns/coredns: v1.10.0 -> v1.10.1
- quay.io/prometheus/blackbox-exporter: v0.23.0 -> v0.24.0
- quay.io/prometheus/node-exporter: v1.5.0 -> v1.6.1
- ghcr.io/credativ/plutono: v7.5.22 -> v7.5.23
- ghcr.io/prometheus-operator/prometheus-config-reloader: v0.61.1 -> v0.67.1
- registry.k8s.io/dns/k8s-dns-node-cache: 1.22.20 -> 1.22.23The worker count for the [NetworkPolicy controller](https://github.com/gardener/gardener/blob/master/docs/concepts/resource-manager.md#networkpolicy-controller) in GRM was increased to `20`. This is necessary to create and update `NetworkPolicies` in time, esp. on larger seed clusters.A new make target is introduced to add license headers.The `Worker` state reconciler has been dropped, i.e., updated provider extensions will no longer populate the machine state to the `.status.state` field of `Worker` resources. For a few releases, `gardenlet` will no longer persist any still existing data in the `.status.state` field of `Worker` resources during a control plane migration of a `Shoot`, and it will set `.status.state` to `nil` after a successful reconciliation or restore operation.The `gardener-apiserver` now drops expired `Kubernetes` and `MachineImage` versions from `Cloudprofile`s during creation.Bumped up the custom image version to v3.4.13-bootstrap-11⚠️ The deprecated field `.spec.settings.ownerChecks` has been removed from the Seed API. Please check your `Seed`s and remove any usage before upgrading to this Gardener version.Status of `garden` now includes the `ObservabilityComponentsHealthy` condition which show the health of observability components in the garden runtime-cluster.The `charts/images.yaml` file was moved to `imagevector/images.yaml`.`gardener-operator` now takes over management of `gardener-metrics-exporter`.Fixes a bug in backup-restore which falsely detects scale-up scenario incase of rolling update of statefulset.The following image is updated:
- `quay.io/prometheus/prometheus`: `v2.43.1` -> `v2.47.0`The `virtual-garden-kube-apiserver` service (for the `virtual-garden` cluster) was switched from type `LoadBalancer` to `ClusterIP`. Please make sure to migrate all DNS records from the `virtual-garden-kube-apiserver` to the `istio-ingressgateway` endpoint before upgrading to this Gardener version.Custodian controller no longer watches leases owned by the etcd resources, thus reducing frequency of etcd status updates and now honouring `custodian-sync-period` value.A bug preventing `prometheus` ingress to use `wildcard-certificate` is fixed.`gardenlet` no longer reports the `Bootstrapped` condition on `Seed`s. Instead, it now reports the progress in `.status.lastOperation`, similar to how it's done for `Shoot`s.`pkg/utils/chart` does now support embedded charts. The already deprecated methods in the `ChartApplier` and `ChartRenderer` will be removed in a few releases, so extensions should adapt to embedded charts.`UseEtcdWrapper` feature gate has been introduced to allow users to opt for the new [etcd-wrapper](https://github.com/gardener/etcd-wrapper) image.Backupbucket/backupentry controllers: watch secret metadata only`maintenance-controller` now disables `PodSecurityPolicy` admission controller when forcefully upgrading the Kubernetes version of a `Shoot` to `v1.25`. It also ensures maximum workers of each for group is greater or equal to its number of zone for forceful upgrades to `v1.27`.A bug preventing `plutono` ingress to use `wildcard-certificate` is fixed.The deprecated `allow-to-seed-apiserver` `NetworkPolicy` is no longer available in garden or seed clusters. Use `allow-to-runtime-apiserver` instead.`kubectl get garden` now features additional printer column `Observability` providing information about the Observability components of the runtime cluster.The `core/v1alpha1` API version is dropped. Make sure that you don't use the `core/v1alpha1` API version in your machinery.Update local-setup to `kind@v0.18.0`.The deprecated `identity` value is no longer passed when `ControllerInstallation` Helm charts are deployed.A bug that prevented referencing `ConfigMap`s in `.spec.resources` in `Shoot`s has been fixed.The `gardener.cloud/timestamp` annotation is now formatted as `time.RFC3339Nano`.Using internal API versions in `providerConfig` fields is no longer permitted (deprecated since more than `2y`). Ensure that you always use a versioned API.Block public access for S3 buckets created by integration tests.Update alpine base image version to 3.18.4.Shoot addon `nginx-ingress-controller` image is updated to `v1.8.0` for Kubernetes `v1.24+` clusters, to `v1.6.4` for Kubernetes `v1.23` clusters, and to `v1.4.0` for Kubernetes `v1.22` clusters.`machinepriority.machine.sapcloud.io` annotation on machine is now reset to 3 by autoscaler if the corresponding node doesn't have `ToBeDeletedByClusterAutoscaler` taintFunctions `controllerutils.GetAndCreateOrMergePatch`, `controllerutils.GetAndCreateOrStrategicMergePatch`, `controllerutils.CreateOrGetAndMergePatch` and `controllerutils.CreateOrGetAndStrategicMergePatch` were incompatibly changed and now accept a `controllerutils.PatchOption` instead of `client.MergeFromOption`.
If your controllers use one of these functions with `client.MergeFromOption`, you should update it to `controllerutils.PatchOption`.
The `controllerutils.PatchOption` can hold two options today:
- `client.MergeFromOption` which is passed to the underlying patch function.
- `controllerutils.SkipEmptyPatch` which prevents sending empty patches (`{}`).* More categories are added to label a release note for a PR on DWD.
* Release notifications would now be sent to `gardener-dwd` channel (private) on releases.The flags which went out-of-support in MCM v0.49.0 have been cleaned up from MCM deployment yaml.Bump builder image golang from `1.20.4` to `1.20.6` The unused `github.com/gardener/gardener/pkg/controllerutils/predicate.IsBeingMigratedPredicate`, `github.com/gardener/gardener/pkg/controllerutils/predicate.IsObjectBeingMigrated` and `github.com/gardener/gardener/pkg/utils/gardener.IsObjectBeingMigrated` funcs are now removed.Usage of the deprecated injection mechanisms in controller-runtime (like `InjectScheme`, `InjectLogger`, `InjectConfig`, `InjectClient`, `InjectCache` etc) as well as package `extensions/pkg/controller/common` are dropped in a preparation to upgrade to the next version where injection is removed entirely. With this, `Inject*` functions on controllers, predicates, actuators, delegates, and friends are not called anymore. When upgrading the `gardener/gardener` dependency to this version, all injection implementations need to be removed. As a replacement, you can get the needed clients and similar from the manager during initialisation of the component.Gardener can now support clusters with Kubernetes version 1.28. Extension developers have to prepare individual extensions as well to work with 1.28.Provider extensions must now pass the `cluster.Cluster` object for the garden cluster to the `genericactuator.NewActuator` function. See [this](https://github.com/gardener/gardener/blob/8d2f116aa606e5181cd430e5063dd798629bdc78/cmd/gardener-extension-provider-local/app/app.go#L228-L246) for an example how to create such a `cluster.Cluster` object.Before upgrading to this Gardener versions, you must make sure that the `Service`s of all registered provider extensions serving webhooks for the shoot cluster are annotated with `networking.resources.gardener.cloud/from-all-webhook-targets-allowed-ports=[{"protocol":"TCP","port":<port>}]`, `networking.resources.gardener.cloud/namespace-selectors=[{"matchLabels":{"gardener.cloud/role":"shoot"}}]`, and `networking.resources.gardener.cloud/pod-label-selector-namespace-alias=extensions`.The `lastUpdateTime` of extension conditions is no longer considered. Ensure that all registered extensions populate the `lastHeartbeatTime` field instead.Probes will not be created for shoots with no workers.⚠️ Gardener does no longer support garden, seed, or shoot clusters with Kubernetes versions < 1.24. Make sure to upgrade all existing clusters before upgrading to this Gardener version.Update alpine base image components to 3.18.3.When performing control plane migration with `provider-local`, the full migration and restoration logic implemented in the extensions library (generic `Worker` actuator) is now executed (previously, it was skipped). This improves the accuracy of the e2e tests for control plane migration.On deletion, the generic `ControlPlane` actuator will now redeploy the cloud config chart to allow provider extensions update the content with the most up-to-date information.The `register-kind2-env` and `tear-down-kind2-env` will no longer try to deploy and delete the `seed-local` `Secret`. This fixes an issue where `tear-down-kind2-env` would hang as it deletes and then waits for the `seed-local` `Secret` to be deleted which can not happen as long as the `local` `Seed` which uses it still exists. File ownership for `var/etcd/data` will be changed to non-root user (65532).`hack/generate.sh` has been renamed to `hack/generate-sequential.sh`.Gardenlet can now set feature gates for `etcd-druid`. They can be specified via the gardenlet configuration `GardenletConfiguration.EtcdConfig.FeatureGates`Bump g/g version to remove stale client-go dependencyEtcd-related secrets will now be mounted onto the `/var/` directory instead of `/root/`.Update golang 1.20.4 -> 1.21.3An issue causing several tasks from the Shoot reconciliation flow to fail with transient errors of type `duplicate filename in registry` is now fixed.Adapt vpa-updater QPS limits such that it doesn't get throttled on large clustersFollowing dependency has been updated:-
- github.com/gardener/etcd-druid v0.18.1 -> v0.18.4A new alpha feature gate named `MachineControllerManagerDeployment` has been introduced in `gardenlet`. Only enable it when all registered provider extensions in your landscape support this feature.`gardener-operator` is now managing the `gardener-resource-manager` instance as part of the virtual garden cluster control plane. It provides a `TokenRequest` API-based kubeconfig for `gardener-operator` to access the virtual garden cluster. The static token kubeconfig is now unconditionally disabled.The `gardener-operator` now enables full `NetworkPolicy` protection for the garden cluster. In case your garden cluster is a seed at the same time, make sure to keep the values of the `FullNetworkPoliciesInRuntimeCluster` feature gate in sync for both `gardener-operator` and `gardenlet`.`gardener-operator` now refuses to start if operators attempt to downgrade or skip minor Gardener versions. Please see [this document](https://github.com/gardener/gardener/blob/master/docs/deployment/version_skew_policy.md) for more information.If the `kubeletCSRApprover` controller is enabled, it is now mandatory to specify the namespace in the source cluster in which the `Machine` resources reside via `.controllers.kubeletCSRApprover.machineNamespace`.Now git revision and commit ids are properly propagated through build variables. These are showed in the fluent-bit plugin logs during start.Shoot fields `.spec.dns.providers[].domains` and `.spec.dns.providers[].zones` are now deprecated and expected to be removed in version `v1.87`. Please use the extensions' configuration to configure providers with this ability.Druid now exposes metrics related to snapshot compaction, on default port 8080. Please expose the desired metrics port via the etcd-druid service to allow metrics to be scraped by a Prometheus instance.Annotation `alpha.featuregates.shoot.gardener.cloud/node-local-dns` is deprecated and will be removed in future releases. Use field `.spec.systemComponents.nodeLocalDNS.enabled` in `Shoot` instead. Switching on node-local-dns via shoot specification will roll the nodes even if node-local-dns was enabled beforehand via annotation.An issue has been fixed that prevented setting the `UnauthenticatedHTTP2DOSMitigation` feature gate.Update etcd-custom-image to `v3.4.26-2`.`kubectl get garden` now features additional printer columns providing more information about the substantial configuration values and statuses.`gardener-operator` is now managing the Gardener control plane components (`gardener-{apiserver,admission-controller,controller-manager,scheduler}`).`gardener-operator` maintains the two most recent `generic-token-kubeconfig` secrets in the runtime-cluster. In addition the latest secret name is published to the `garden` resource in `.metadata.annotations[generic-token-kubeconfig.secret.gardener.cloud/name]`. Third-party components referring to this secret should check this annotation value after a credentials or CA rotation for the virtual-garden cluster took place.Concurrent empty machines bulk deletion can now be configured for `cluster-autoscaler` via the field `.spec.kubernetes.clusterAutoscaler.maxEmptyBulkDelete` in the `Shoot` API .A new feature gate named `ContainerdRegistryHostsDir` is introduced to gardenlet. When enabled, the `/etc/containerd/certs.d` directory is created on the Node and containerd is configured to look up for registries/mirrors configuration in this directory (if there is any configuration applied). In future, the [registry-cache extension](https://github.com/gardener/gardener-extension-registry-cache/) will add such registries/mirrors configuration under this directory (via OperatingSystemConfig mutation).Makefile targets have changed: Introduced gardener-setup, gardener-restore, gardener-local-mcm-up, non-gardener-setup, non-gardener-restore, non-gardener-local-mcm-up. Users can also directly use the scripts which are used by these makefile targets.It is no longer possible to configure `.spec.virtualCluster.kubernetes.kubeAPIServer.authorization` in the `Garden` API.Introduce flag `metrics-scrape-wait-duration` to `etcdbrctl compact` command, that specifies a wait duration at the end of a snapshot compaction, to allow Prometheus to scrape metrics related to compaction before the `etcdbrctl` process exits.The deprecated `.spec.virtualCluster.dns.domain` field has been dropped from the `Garden` API. Make use of `.spec.virtualCluster.dns.domains`.The `MachineControllerManagerDeployment` has been promoted to beta and is now enabled by default. Make sure that all registered provider extensions support this feature gate before upgrading to this version of Gardener.Shoot node network and seed pod network need to be disjoint. This will be checked during scheduling of a shoot cluster, i.e. during initial admission or on control-plane migration.The `VerticalPodAutoscaler` resources for `kube-proxy`s is no longer recreated when the Kubernetes patch version of the `Shoot` or the respective worker pools is updated. This ensures updated `kube-proxy`s keep the same CPU/memory resource requirements as before the patch version update. In order to put this change into effect, all existing `VerticalPodAutoscaler`s for `kube-proxy`s are getting recreated.The static token kubeconfig can no longer be enabled for Shoot clusters using Kubernetes version `1.27` and higher.Update golang 1.19.5 -> 1.20.4Kubernetes feature gate `UnauthenticatedHTTP2DOSMitigation` is considered valid for versions >= `1.25`.The `Deploying Shoot namespace in Seed` step was slightly improved. Earlier it failed at some occasions when it tried to read zone information for volumes that have not been created yet. This was a transient error that dissolved in subsequent reconcile runs.A bug which was causing race conditions to occur during reconciliation of extension resources was fixed. 
gardener-robot
Release Notes: