gardener-robot-ci-2
commented
1 year ago webhook as it is now included in Gardener Resource Manager.
```feature operator github.com/gardener/gardener #7931 @rfranzke
`gardener-operator` is now managing the `kube-controller-manager` instance as part of the virtual garden cluster control plane.`kubectl proxy` now works as expected in the local development setup in conjunction with highly available vpnThe following Golang dependencies have been updated:
- `k8s.io/*` from `v0.28.2` to `v0.28.3`
- `sigs.k8s.io/controller-runtime` from `v0.16.2` to `v0.16.3`The `kube-apiserver` no longer mounts root CA bundles from the underlying host.Eliminated `RoleBinding` helm charts and converted into Golang component with added unit tests.Gardener uses an `InternalSecret` per Shoot for syncing the client CA to the project namespace in the garden cluster (named `<shoot-name>.ca-client`). The `shoots/adminkubeconfig` subresource signs short-lived client certificates by retrieving the CA from the `InternalSecret`.Stability of the ssh tunnel in the local extension setup should improve due to better failure handling.Deprecated annotation `alpha.featuregates.shoot.gardener.cloud/node-local-dns-force-tcp-to-{cluster-dns, upstream-dns}` is removed. Use field `.spec.systemComponents.nodeLocalDNS.{forceTCPToClusterDNS, forceTCPToUpstreamDNS}` in `Shoot` instead.`nginx-ingress-controller` image is updated to `v1.9.1`.Alpine image used in init containers is now part of the IMAGEVECTOR_OVERWRITEFix verification.The `VerticalPodAutoscaler` resources for `kube-proxy`s is no longer recreated when the Kubernetes patch version of the `Shoot` or the respective worker pools is updated. This ensures updated `kube-proxy`s keep the same CPU/memory resource requirements as before the patch version update. In order to put this change into effect, all existing `VerticalPodAutoscaler`s for `kube-proxy`s are getting recreated.gardenlet: A regression preventing the alertmanager in the garden namespace from sending email notifications is now fixed.An issue causing deletion of a legacy (wrongly configured) Shoot cluster to be denied because of network ranges overlapping with the default VPN network is now fixed.`gardener-operator` now runs a new controller which protects `Secret`s and `ConfigMap`s with a finalizer in case they are referenced in `Garden` resources.Deprecated annotation `alpha.featuregates.shoot.gardener.cloud/node-local-dns` is removed. Use field `.spec.systemComponents.nodeLocalDNS.enabled` in `Shoot` instead. Switching on node-local-dns via shoot specification will roll the nodes even if node-local-dns was enabled beforehand via annotation.The `pkg/operation/botanist/component/*` resources have been moved to `pkg/component/*`.Update `k8s.io/client-go` from v0.17.0 to v0.26.2Now git revision and commit ids are properly propagated through build variables. These are showed in the fluent-bit plugin logs during start.The `.spec.kubernetes.kubeAPIServer.serviceAccountConfig.acceptedIssuers` field of the `Shoot` spec no longer allows duplicate values.When the `ShootForceDeletion` featuregate in the apiserver is turned on, users will be able to force-delete the Shoot. You **MUST** ensure that all the resources created in the IaaS account are cleaned up to prevent orphaned resources. Gardener will **NOT** delete any resources in the Shoot cloud-provider account. See [Shoot Force Deletion](https://github.com/gardener/gardener/blob/master/docs/usage/shoot_operations.md#force-deletion) for more details.The following images are updated:
- `registry.k8s.io/kube-state-metrics/kube-state-metrics`: `v2.5.0` -> `v2.8.2`⚠️ Gardener does no longer support garden, seed, or shoot clusters with Kubernetes versions < 1.22. Make sure to upgrade all existing clusters before upgrading to this Gardener version.An issue has been fixed which might have caused the deletion of `Shoot` clusters to stuck when a namespace was forcefully removed before all relevant resources have been cleaned up.The kind clusters are now unified to use `garden.local.gardener.cloud` DNS name in the containerd config when configuring registry mirror hostnames. Previously, to access the pull through registry cache some kind clusters were configured to use `garden.local.gardener.cloud`, others - the Node name of the control plane Node.Update `vertical-pod-autoscaler` to `v0.14.0`.The feature gates `FullNetworkPolicies` and `HAControlPlanes` have been promoted to GA and are now locked to "unconditionally enabled".An issue causing several tasks from the Shoot reconciliation flow to fail with transient errors of type `duplicate filename in registry` is now fixed.Added `errorCode` field in the `LastOperation` struct. This should be implemented only for the `CreateMachine` call in the `triggerCreationFlow`. This field will be utilized by Cluster autoscaler to do early backoff Annotation `alpha.featuregates.shoot.gardener.cloud/node-local-dns` is deprecated and will be removed in future releases. Use field `.spec.systemComponents.nodeLocalDNS.enabled` in `Shoot` instead. Switching on node-local-dns via shoot specification will roll the nodes even if node-local-dns was enabled beforehand via annotation.The project vendors the latest released gardener version - v1.73.0`operator` now deletes `ManagedResources` deployed to the virtual-garden before deleting `virtual-garden-kube-apiserver`.The `gardener-operator` does now also manage `kube-state-metrics`.APIServer validation allows updating to expired Kubernetes and machine image versions.The deprecated `.spec.virtualCluster.dns.domain` field has been dropped from the `Garden` API. Make use of `.spec.virtualCluster.dns.domains`.Update alpine base image components to 3.18.3.During the `Migrate` phase of a control plane migration of a `Shoot`, the state is now only persisted after all extension resources have been migrated. Consequently, make sure that you have added all state to the `.status.state` field of the respective extension object when running `Migrate()`.`gardener-operator` now refuses to start if operators attempt to downgrade or skip minor Gardener versions. Please see [this document](https://github.com/gardener/gardener/blob/master/docs/deployment/version_skew_policy.md) for more information.The `WorkerlessShoots` feature gate has been promoted to beta and is now turned on by default. Before deploying this Gardener version, make sure that all your registered extensions support this feature gate.`gardener-operator` now deploys `Istio` components into the garden runtime cluster.Add memory and cpu limits (maxAllowed) to Prometheus (H)VPAs.The following image is updated:
- quay.io/prometheus/prometheus: v2.41.0 -> v2.43.1Shoot fields `.spec.dns.providers[].domains` and `.spec.dns.providers[].zones` are now deprecated and expected to be removed in version `v1.87`. Please use the extensions' configuration to configure providers with this ability.It is now possible to specify custom linux kernel settings per worker pool for `Shoot`s via `.spec.provider.workers[].sysctls`, which may override Gardener default values.update client-go version and exclude the old one in go.modAnnotations in `seed.spec.settings.loadBalancerServices.annotations` are now applied to the Nginx-Ingress load balancer service in the seed cluster.A bug causing the managedseed controller to error if the controller restarts and the seed secret is already deleted is now fixed.Remove unneeded Monitor function from iptables implementation Operators can now use the annotation `gardener.cloud/operation=rotate-observability-credentials` on the `garden` resource to rotate the observability credentials. A bug has been fixed which caused `ServiceAccount`s related to garden access secrets for extensions to leak in the seed namespace in the garden cluster after uninstallation of said extensions.The `networking.resources.gardener.cloud/from-policy-pod-label-selector` and `networking.resources.gardener.cloud/from-policy-allowed-ports` annotations are now deprecated and will be removed in the future. Use `networking.resources.gardener.cloud/from-<pod-label-selector>-allowed-ports=<ports>` instead.Gardener denies setting `Shoot.Spec.ControlPlane.HighAvailability.FailureTolerance.Type` if shoot is hibernated.Use admission v1 instead of v1beta1 for apiserver-proxy webhook.A bug in the local development environment has been fixed which prevented admission of Gardener resources by extension webhooks.Updated golang container image build version to 1.20.4Kubernetes feature gate `UnauthenticatedHTTP2DOSMitigation` is considered valid for versions >= `1.25`.The testmachinery tests now use `AdminKubeconfig` of the `Shoot`s of `ManagedSeed`s to create seed client.All default images are now present in `images.yaml`Functions `controllerutils.GetAndCreateOrMergePatch`, `controllerutils.GetAndCreateOrStrategicMergePatch`, `controllerutils.CreateOrGetAndMergePatch` and `controllerutils.CreateOrGetAndStrategicMergePatch` were incompatibly changed and now accept a `controllerutils.PatchOption` instead of `client.MergeFromOption`.
If your controllers use one of these functions with `client.MergeFromOption`, you should update it to `controllerutils.PatchOption`.
The `controllerutils.PatchOption` can hold two options today:
- `client.MergeFromOption` which is passed to the underlying patch function.
- `controllerutils.SkipEmptyPatch` which prevents sending empty patches (`{}`).Bump `k8s.io/*` deps to v0.27.2Initial implementation for `Refresh()` method of `CloudProvider` interface done`AllMembersReady` condition has now been fixed to eventually show the correct overall readiness of an etcd cluster.The following image is updated:
- `quay.io/prometheus/alertmanager`: `v0.24.0` -> `v0.26.0``github.com/gardener/gardener/pkg/utils/gardener.ShootAccessSecret` was renamed to `AccessSecret`.The `.status.lastOperation` in `core.gardener.cloud/v1beta1.Seed` and `operator.gardener.cloud/v1alpha1.Garden` resources is now only updated each `5s` during a reconciliation. Previously, it was updated immediately when a task was finished.A bug has been fixed which was allowing users to specify an extension of the same type in `.spec.extensions[].type` more than once in the `Shoot` API.Run `make ci-e2e-kind` to run the e2e tests on local machineThe `shootstate-extensions` and `shootstate-secret` controllers have been dropped. The `gardenlet`'s component config file should be updated to no longer specify related configuration (`.controllers.{shootSecret,shootStateSync}`).It is now possible to trigger gardenlet kubeconfig renewal for unmanaged `Seed`s by annotating them with `gardener.cloud/operation=renew-kubeconfig`. This was already supported for `ManagedSeed`s only.Gardener now uses 3072 bit RSA keys in order to generate TLS certificates.The `Shoot` maintenance controller now updates the CRI of worker pools from `docker` to `containerd` when force-upgrading from Kubernetes `v1.22` to `v1.23`.Added check to ensure that the scale up annotation is removed from the etcd statefulset only when scale-up succeedsThe `ResourcesProgressing` condition appearing in the status of `ManagedResource`s now checks for non-terminated `Pod`s before reporting `status=False`.The deprecated `ChartRenderer.Render` and `ChartApplier.{Apply,Delete}` methods have been dropped. Use `ChartRendere.RenderEmbeddedFS` and `ChartApplier.{Apply,Delete}FromEmbeddedFS` instead.Feature gates have been introduced in etcd-druid, and can be specified using CLI flag `--feature-gate`.gardener-apiserver now exposes a new `core.gardener.cloud/v1beta1.InternalSecret` API, see the [documentation](https://github.com/gardener/gardener/blob/master/docs/concepts/apiserver.md#internalsecrets) for more information.The deprecated `core.gardener.cloud/apiserver-exposure` label and handling has been dropped.It is possible to delete a Shoot even if `shoot.gardener.cloud/ignore` annotation is set to true.It is possible now to create a workerless shoot cluster when the `WorkerlessShoots` feature gate in the `gardener-apiserver` is enabled. Please see [this document](https://github.com/gardener/gardener/blob/master/docs/usage/shoot_workerless.md) for more details.Shoot addon `nginx-ingress-controller` image is updated to `v1.3.0` for `v1.22+` shoots.Fixed a bug that caused HVPA reconciliation to fail with `expected pointer, but got v2beta1.MetricSpec type` when the HPA spec had changed.Vali is now updated to version v2.2.6The following mapper funcs from the extension library no longer accept a `context.Context` arg - `ClusterToContainerResourceMapper`, `ClusterToControlPlaneMapper`, `ClusterToDNSRecordMapper`, `ClusterToExtensionMapper`, `ClusterToInfrastructureMapper`, `ClusterToNetworkMapper`, `ClusterToWorkerMapper` and `ClusterToObjectMapper`. The `context.Context` arg was redundant and not used.Etcd-related secrets will now be mounted onto the `/var/` directory instead of `/root/`.The `check-apidiff` check was changed to only report incompatible and critical changes which need inspection from the developer's side.Add Prometheus alert for pending seed podsThe `hack/check-docforge.sh` script is now removed. The repo based manifest are removed in favor of a centrally managed manifests. See https://github.com/gardener/documentation/issues/431. The manifests are now maintained centrally in https://github.com/gardener/documentation/tree/master/.docforge.Block public access for S3 buckets created by integration tests.A new optional constraint `CRDsWithProblematicConversionWebhooks` is introduced in the `Shoot` status. This constraint indicates that there is at least one CRD in the cluster which has multiple stored versions and a conversion webhook configured, which could break the reconciliation flow of a `Shoot` in some cases.A new field `errorCodeCheckFunc` is introduced in the generic `Worker` actuator. This should be set to parse the Gardener error codes from the error returned in `Worker` reconciliation.Add an alert for VPNHAShootNoPods when shoot in HA (high availability) mode.HVPA supports k8s versions >= 1.25 by switching to `k8s.io/autoscaling/v2` when necessary for all API calls.Backup-restore waits for its etcd to be ready before attempting to update peerUrlProvider extensions should be adapted such that they no longer perform health checks specific to the `machine-controller-manager` deployment or the machines/nodes. In the future, `gardenlet` will take over performing these checks. Please see https://github.com/gardener/gardener/pull/8019 for an example how `provider-local` was adapted and replicate it for your provider extensions.⚠️ Gardener does no longer support garden, seed, or shoot clusters with Kubernetes versions < 1.24. Make sure to upgrade all existing clusters before upgrading to this Gardener version.Etcd druid will now not support `policy/v1beta1` for `PodDisruptionBudget`s and will only use `policy/v1` for `PodDisruptionBudget`s`gardener-operator` now renews garden access secrets and the gardenlet kubeconfig on all `Seed`s during CA/service account signing key credentials rotation.A bug preventing `prometheus` ingress to use `wildcard-certificate` is fixed.Validation has been added for `spec.kubernetes.kubeAPIServer.runtimeConfig` field in the Shoot API. Disabling APIs marked as "Required" by gardener is not permitted.Gardener sets [`minDomains`](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/#spread-constraint-definition) for shoot system components to the number of zones configured in the system component worker pool(s).
⚠️ It is strongly recommended to place at least one worker node per availability zone for system component workers in order to ensure hitch-free rolling updates and scheduling of pods. You may need to adjust the `maximum: <number-of-zones>` values of your system component worker pool(s).
This configuration only takes effect for clusters which enabled feature gate `MinDomainsInPodTopologySpread` (enabled by default as of Kubernetes `v1.27`).An issue has been fixed that prevented setting the `UnauthenticatedHTTP2DOSMitigation` feature gate.Applying Gardener resources server-side has caused the `the server is currently unable to handle the request` error which is now fixed.Etcd-backup-restore now uses the user home directory to create files.A bug that prevented finalizers from being added to referenced `Secret`s or `ConfigMap`s in `.spec.resources` in `Shoot`s has been fixed.Status of `garden` now includes the `ObservabilityComponentsHealthy` condition which show the health of observability components in the garden runtime-cluster.With this release the obervability compoents are updated to the latest release versions. Plutono is now at v2.5.25 and Vali is now at v2.2.9`gardenlet` will no longer respect `ConfigMap`s labeled with `extensions.gardener.cloud/configuration=logging`. The way to deploy a new filter or parser configuration is to create `ClusterFilter`s or `ClusterParser`s custom resources in the seed cluster.Change port of ssh reverse tunnel to 443Bump builder image golang from `1.20.4` to `1.20.6` Removed dead metrics code and refactored the remaining metrics codeIf the `kubeletCSRApprover` controller is enabled, it is now mandatory to specify the namespace in the source cluster in which the `Machine` resources reside via `.controllers.kubeletCSRApprover.machineNamespace`.Add CVE categorization for etcd-druid.Added an example for `AdminKubeconfigRequest` via the Python Kubernetes client.The kind cluster used in local setup does now use the new way in containerd to configure registry mirrors.:warning: `etcd.Status.ClusterSize`, `etcd.Status.ServiceName`, `etcd.Status.UpdatedReplicas` have been marked as deprecated and users should refrain from depending on these fields.It is now possible to provide namespace selectors for additional namespaces which should be covered by the `NetworkPolicy` controllers of `gardener-operator` or `gardenlet`. The selectors must be provided via their component configs. Please consult [this document](https://github.com/gardener/gardener/blob/master/docs/usage/network_policies.md#additional-namespace-coverage-in-gardenseed-cluster) for further insights.The `extensions/pkg/controller/operatingsystemconfig/oscommon` package is deprecated and will be removed as soon as the `UseGardenerNodeAgent` feature gate has been promoted to GA. OS extension developers should start adapting to this new feature, see [documentation](https://github.com/gardener/gardener/blob/master/docs/extensions/operatingsystemconfig.md#what-needs-to-be-implemented-to-support-a-new-operating-system) and [example](https://github.com/gardener/gardener/tree/master/pkg/provider-local/controller/operatingsystemconfig) based on `provider-local`.Following dependencies are updated:
Go - 1.20.3
client-go - v0.26.2
controller-runtime - v0.14.5
gomega - v1.27.1
zap - v1.24.0
gardener/gardener v1.69.0
k8s (api and apimachinery) - v0.26.2`gardener-operator` is now managing the `gardener-resource-manager` instance as part of the virtual garden cluster control plane. It provides a `TokenRequest` API-based kubeconfig for `gardener-operator` to access the virtual garden cluster. The static token kubeconfig is now unconditionally disabled.An issue has been fixed for highly-available `Shoot`s whose `etcd` clusters didn't get ready in the `Completing` phase of a CA credentials rotation.The following images are updated:
- `eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler`: `v1.21.5` -> `v1.21.6` (for Kubernetes `1.21`)
- `eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler`: `v1.22.5` -> `v1.22.6` (for Kubernetes `1.22`)
- `eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler`: `v1.23.3` -> `v1.23.4` (for Kubernetes `1.23`)
- `eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler`: `v1.24.2` -> `v1.24.3` (for Kubernetes `1.24`)
- `eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler`: `v1.25.2` -> `v1.25.3` (for Kubernetes `1.24`)
- `eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler`: `v1.26.1` -> `v1.26.2` (for Kubernetes `1.26`)The `alpha.kube-apiserver.scaling.shoot.gardener.cloud/class` annotation on `Shoot`s has no effect anymore and should be removed.On deletion, the generic `ControlPlane` actuator will now redeploy the cloud config chart to allow provider extensions update the content with the most up-to-date information.Gardener refined the scope of the problematic webhook matcher for `endpoint` objects. Earlier, shoot clusters were assigned a constraint reporting a problem with a `failurePolocy: Fail` webhook acting on these objects. Now, only `endpoint`s in the `kube-system` and `defaults` namespaces are considered for this check.`gardener-operator` is now managing the Gardener control plane components (`gardener-{apiserver,admission-controller,controller-manager,scheduler}`).Revendors the bbolt from `v1.3.6` to `v1.3.7`Since `Namespace`s are no longer deleted (and forcefully finalized after some grace period), the `shoot.gardener.cloud/cleanup-namespaces-finalize-grace-period-seconds` annotation does no longer have any effect. Relevant Kubernetes resources are still cleaned up (see [this document](https://github.com/gardener/gardener/blob/master/docs/usage/shoot_cleanup.md)) for more information.Backup-restore waits for its etcd to be ready before attempting to update peerUrl`extensions.gardener.cloud/v1alpha1.ControlPlane` is now deployed after `kube-apiserver` in the Shoot reconciliation flow.It is now possible to enable disabled APIs for workerless shoot clusters via `spec.kubernetes.kubeAPIServer.runtimeConfig`.While scaling up a non-HA etcd cluster to HA skipping the scale-up checks for first member of etcd cluster as first member can never be a part of scale-up scenarios.When `Shoot`s were updated from non high-availability to `zone` high-availability, it could happen that the control-plane was scheduled to two instead of three zones. This issue is relevant for cloud providers with an inconsistent zone naming (`Azure` is currently the only candidate to our knowledge).
Existing shoots with the before mentioned problem must be fixed manually be operators if required. An automatic move of `etcd`s and their volumes is not part of this fix due to availability reasons.An issue has been fixed which was causing a broken `ControlPlaneHealthy` condition report for `Shoot`s when the `MachineControllerManagerDeployment` feature gate gets enabled until their next reconciliation.For Shoot clusters using Kubernetes version `1.27` and higher, the `.spec.kubernetes.kubeControllerManager.podEvictionTimeout` field has no effect anymore since the backing `--pod-eviction-timeout` CLI flag has been removed.Configuring multiple `reserve-excess-capacity` deployments on `Seed`s is supported now by specifying `.spec.settings.excessCapacityReservation.configs`.The `MachineControllerManagerDeployment` has been promoted to beta and is now enabled by default. Make sure that all registered provider extensions support this feature gate before upgrading to this version of Gardener.`gardener-resource-manager`'s `system-components-config` webhook no longer adds the toleration for the `ToBeDeletedByClusterAutoscaler` taint to system components in shoot clusters. The `ToBeDeletedByClusterAutoscaler` taint is maintained by the `cluster-autoscaler`. This was breaking `cluster-autoscaler`'s drain mechanism when scaling down an under-utilized node. It was causing just evicted system components from to be deleted node to be scheduled again on the to be deleted node.The no longer required `--gardenlet-manages-mcm` option has been removed. All code in provider extensions related to management/deployment of `machine-controller-manager` should be removed.Add new flag `metrics-scrape-wait-duration` for compaction controller to set a wait duration at the end of every compaction job, to allow for metrics to be scraped by a Prometheus instance.A new controller in `gardenlet` for periodically backing up the `ShootState` for `Shoot`s has been introduced. This controller is only activated when `gardenlet` is responsible for an unmanaged `Seed` (i.e., one not backed by a `ManagedSeed` object). By default, backups are taken roughly each `6h`.`fluent-operator` is now installed in the `garden` namespace of seed clusters and will take care of the entire lifecycle of the `fluent-bit` `DaemonSet`.Extensions that wish to be scraped by the `seed-prometheus` must annotate their pods with `prometheus.io/scrape=true` along with `prometheus.io/name=<name>`. See https://github.com/gardener/gardener/blob/master/docs/monitoring/README.md#seed-prometheus for more details.Gardener now reports `node`s for which the `checksum/cloud-config-data` hasn't been populated yet. This could point towards an error on the node and that not all Gardener related configuration happened successfully.Update golang 1.19.5 -> 1.20.4The logging components: vali and valitail are now updated to v2.2.8.The `virtual-garden-kube-apiserver` service (for the `virtual-garden` cluster) was switched from type `LoadBalancer` to `ClusterIP`. Please make sure to migrate all DNS records from the `virtual-garden-kube-apiserver` to the `istio-ingressgateway` endpoint before upgrading to this Gardener version.CloudProfiles allow configuring update strategies {patch, minor, major} for machine images that affect update behavior during auto and force update.The GA-ed feature gates `HAControlPlanes` and `FullNetworkPoliciesInRuntimeCluster` have been removed.Webhooks remediator now sets the timeoutSeonds to 3 seconds for webhook affecting lease resources in `kube-system` namespace.gardener-apiserver: The kubelet version constraint validation is now fixed to also cover the Shoot K8s version update. Previously it was possible to update the Shoot K8s version to a new minor version when the Shoot has a worker pool with machine image version which kubeletVersionConstraint does not match the new K8s version.Introduces a skaffold local development pipeline to fluent-bit-vali-pluginRemoved `service.beta.kubernetes.io/aws-load-balancer-type: nlb` annotation from istio-ingressgateway service template. Set this annotation in seed configuration. Note: Changing load balancer type creates a new one, old one requires manual clean-up.Update golang 1.20.4 -> 1.21.3Base alpine image upgraded from `3.15.7` to `3.15.8`Provider extensions should be adapted such that they only inject their provider-specific `machine-controller-manager` sidecar container into the `machine-controller-manager` deployment instead of managing the full deployment themselves. In the future, `gardenlet` will take over managing it. Please see https://github.com/gardener/gardener/pull/8019 for an example how `provider-local` was adapted and replicate it for your provider extensions.Etcd-druid will now deploy distroless `etcd-wrapper` and `etcd-backup-restore` images. Please refer to [etcd-wrapper](https://github.com/gardener/etcd-wrapper) for more information.The `pkg/utils/gardener.IntStrPtrFromInt` function has been renamed to `IntStrPtrFromInt32` since `intstr.FromInt` is deprecated.Extensions have to implement the `ForceDelete` function in the actuator with the logic of forcefully deleting all the resources deployed by them.The following image is updated:
- quay.io/brancz/kube-rbac-proxy: v0.14.0 -> v0.14.2unit tests framework introduced to test implemented methods of `Cloudprovider` and `Nodegroup` interfaceAn issue has been fixed which caused CoreDNS to not rewrite CNAME values in DNS answers.A new feature gate named `ContainerdRegistryHostsDir` is introduced to gardenlet. When enabled, the `/etc/containerd/certs.d` directory is created on the Node and containerd is configured to look up for registries/mirrors configuration in this directory (if there is any configuration applied). In future, the [registry-cache extension](https://github.com/gardener/gardener-extension-registry-cache/) will add such registries/mirrors configuration under this directory (via OperatingSystemConfig mutation).`gardener-operator` no longer reports the `Reconciled` condition. Instead, it now reports the progress in `.status.lastOperation`, similar to how it's done for `Shoot`s.A bug causing `EtcdCopyBackupsTask` jobs to fail to create temp snapshot directory while using distroless etcd-backup-restore image `v0.25.x` has been resolved.The worker count for the [NetworkPolicy controller](https://github.com/gardener/gardener/blob/master/docs/concepts/resource-manager.md#networkpolicy-controller) in GRM was increased to `20`. This is necessary to create and update `NetworkPolicies` in time, esp. on larger seed clusters.The regression is now fixed and the control plane logs shall be visible in the Plutono dashboards.The `register-kind2-env` and `tear-down-kind2-env` will no longer try to deploy and delete the `seed-local` `Secret`. This fixes an issue where `tear-down-kind2-env` would hang as it deletes and then waits for the `seed-local` `Secret` to be deleted which can not happen as long as the `local` `Seed` which uses it still exists. It is now easier to annotate `Service`s related to extensions serving webhook handlers that must be reached by `kube-apiserver`s running in separate namespaces such that the respective network traffic gets allowed. Please refer to [this guide](https://github.com/gardener/gardener/blob/master/docs/usage/network_policies.md#webhook-servers) for all information. Extensions serving shoot webhook should make use of this new approach - the old functionality deploying dedicated `NetworkPolicy`s is deprecated and will be removed in the future.Prevent nil pointer exceptions on shoot deletion in `gardenlet` when seed namespace is gone.gardenlet: A regression causing metering related recording rules for the aggregate-prometheus not to be applied is now fixed.A configuration issue that resulted in a relatively slow startup and termination of the vali pods is fixed.Probes will not be created for shoots with no workers.The deprecated `extensions/pkg/controller/worker.{Options,ApplyMachineResources{ForConfig}}` symbols have been dropped since `gardenlet` takes over management of the `machine.gardener.cloud/v1alpha1` API CRDs since `gardener/gardener@v1.73`.The `alpha.featuregates.shoot.gardener.cloud/apiserver-sni-pod-injector` annotation has been dropped and is no longer available for `Shoot`s. It should be removed from all existing `Shoot` resources.The target cache for `gardener-resource-manager` is now unconditionally enabled, leading to faster reconciliations and less network I/O.Base alpine image for etcd-custom-image upgraded from `3.15.7` to `3.15.8`The skaffold version is updated from v2.7.0 to v2.8.0.The `.{source,target}ClientConnection.namespace` field has been renamed to `namespaces` and now takes a list of namespaces. The `.targetClientConnection.disableCachedClient` field has been removed.A bug where MCM removed a machine other than the one , CA wanted , is resolved.Test-machinery integration tests are now using upstream K8s e2e test images such as `registry.k8s.io/e2e-test-images/busybox`, `registry.k8s.io/e2e-test-images/agnhost` instead Gardener images such as `eu.gcr.io/gardener-project/3rd/busybox`, `eu.gcr.io/gardener-project/3rd/alpine` and others.When the Kubernetes control plane version is at least `v1.28`, it is now possible to set the worker pool Kubernetes version to be at most three versions behind the control plane version. Earlier, only a skew of at most two versions was allowed. Find more details [here](https://kubernetes.io/blog/2023/08/15/kubernetes-v1-28-release/#changes-to-supported-skew-between-control-plane-and-node-versions).Introduce DEP-04 [EtcdMember Custom Resource](https://github.com/gardener/etcd-druid/blob/master/docs/proposals/04-etcd-member-custom-resource.md).`Shoot`s allow to optionally configure a specific scheduler via `.spec.schedulerName`. The `default-scheduler` is used in case non is configured. Please note, that `Shoot`s will remain `Pending` in case a scheduler name is configured but an adequate scheduler is not available in the landscape.Grafana and Loki are replaced with the fork of their last Apache 2.0 licensed releases: Plutono and Vali, that will continue to receive security updates.Extensions running on seed clusters can get access to the garden cluster by using the injected kubeconfig specified by the `GARDEN_KUBECONFIG` environment variable. You can read about the details in this [doc](https://github.com/gardener/gardener/blob/master/docs/extensions/garden-api-access.md).A bug has been fixed which was causing the garbage collector in `gardener-resource-manager` to wrongfully collect `Secret`s related to `ManagedResource`s when the source and the target cluster are equal.Configure the value for the flag `metrics-scrape-wait-duration` for compaction controller to set a wait duration at the end of every compaction job, to allow for metrics to be scraped by a Prometheus instance.Bump alpine base version for Docker build to `3.18.2`.The `terraformer` library will now skip deletion of the Terraformer pod when the request context has been canceled. This change aims to prevent inconsistencies in Terraform state by attempting to allow uninterrupted execution of healthy Terraformer pods.A bug causing the gardenlet to panic when a ETCD encryption key rotation operation is triggered for a hibernated Shoot is now fixed. Now, triggering ETCD encryption key rotation or ServiceAccount signing key rotation is forbidden when the Shoot is in waking up phase.Introduce flag `metrics-scrape-wait-duration` to `etcdbrctl compact` command, that specifies a wait duration at the end of a snapshot compaction, to allow Prometheus to scrape metrics related to compaction before the `etcdbrctl` process exits.Update alpine base image version to 3.18.3.File ownership for `var/etcd/data` will be changed to non-root user (65532).
gardener-robot
Release Notes: