⬆️ Upgrade the `go.etcd.io/etcd` dependency to `v3.4.34`

[renormalize]

What this PR does / why we need it:

• Upgrade the etcd version to v3.4.34.
• Use the Restoration API provided by etcd to trigger a restore instead of relying on copied code for restoration.
• Upgrade the etcd version to v3.4.34 in the integration tests.

Which issue(s) this PR fixes: Fixes #792

Special notes for your reviewer:

Only commits beb9cbcdd41084638530fa29d48ff48e9102a792 and b849aaac6b0ba9200fb3671b5d3a1b3b74a10a8f have code changes which are relevant.

cc @seshachalam-yv

Release note:

`etcd-backup-restore` now uses etcd version `v3.4.34` for the embedded etcd during restoration.

`etcd-backup-restore` now uses the `Restore` API exposed by `go.etcd.io/etcd/clientv3/snapshot` to perform restoration of the etcd data directory, moving away from performing restoration of the data directory manually.

[seshachalam-yv]

It appears that we are currently referencing etcd version 3.4.13 in the following files:

• .ci/integration_test
• .ci/performance_regression_test
• chart/etcd-backup-restore/values.yaml
• docs/development/testing_and_dependencies.md

Could you please update these references to etcd version 3.4.34?

[renormalize]

Thanks for your review @seshachalam-yv!

Regarding your comments:

• The version mentioned here .ci/integration_test is just used for the echo, which I missed. Thanks for pointing it out. The tests already were running with v3.4.34 thankfully due to the change in this line.

• The following will have their "etcd" images, i.e. etcd-custom-image replaced with etcd-wrapper. I've added TODOs to ensure that when a newer release of etcd-wrapper is made with the etcd version as v3.4.34, the references will be changed here. Until then, I've changed it to the latest release of etcd-wrapper.

  • .ci/performance_regression_test
  • chart/etcd-backup-restore/values.yaml
  • docs/development/testing_and_dependencies.md

[renormalize]

@seshachalam-yv

I've run etcd-druid e2e tests with changes proposed from this PR, along with the changes proposed in https://github.com/gardener/etcd-wrapper/pull/30.

I overrode the etcd-backup-restore dependency in etcd-druid with the latest commit of this PR.

All tests pass.

make ci-e2e-kind

``` ➜ etcd-druid git:(etcd-v3.4.34) ✗ make ci-e2e-kind Creating local docker registry... bdc16e1971652fe76195d22d055913be04bee78d99fc72ddbc53ead32f85d1f3 Generating kind cluster config... Creating cluster "etcd-druid-e2e" ... ✓ Ensuring node image (kindest/node:v1.29.8) 🖼 ✓ Preparing nodes 📦 ✓ Writing configuration 📜 ✓ Starting control-plane 🕹️ ✓ Installing CNI 🔌 ✓ Installing StorageClass 💾 Set kubectl context to "kind-etcd-druid-e2e" You can now use your cluster with: kubectl cluster-info --context kind-etcd-druid-e2e Have a nice day! 👋 Initializing local docker registry... configmap/local-registry-hosting created 📌 NOTE: To target the newly created KinD cluster, please run the following command: export KUBECONFIG=/Users/renormalize/go/src/github.com/gardener/etcd-druid/hack/kind/kubeconfig node/etcd-druid-e2e-control-plane condition met deployment.apps/localstack created service/localstack created Waiting for deployment "localstack" rollout to finish: 0 of 1 updated replicas are available... deployment "localstack" successfully rolled out pod/localstack-7f769fd7dc-p6j5c condition met namespace/e2e-test created ------------------- Setting up environment ------------------- namespace/e2e-test unchanged No tags generated Starting test... Starting deploy... Starting pre-deploy hooks... Deleting previous job No resources found Completed pre-deploy hooks - configmap/aws-infra-common created - configmap/aws-run-script created - secret/aws-infra-secret created - job.batch/aws-infra-job created Starting post-deploy hooks... Wait until job is ready job.batch/aws-infra-job condition met Completed post-deploy hooks You can also run [skaffold run --tail] to get the logs ------------------- Deploying Druid ------------------- Generating tags... - europe-docker.pkg.dev/gardener-project/public/gardener/etcd-druid -> localhost:5001/europe-docker_pkg_dev_gardener-project_public_gardener_etcd-druid:v0.23.0-8-gd2047f4a-dirty Checking cache... - europe-docker.pkg.dev/gardener-project/public/gardener/etcd-druid: Not found. Building Starting build... Found [kind-etcd-druid-e2e] context, using local docker daemon. Building [europe-docker.pkg.dev/gardener-project/public/gardener/etcd-druid]... Target platforms: [linux/arm64] Using base gcr.io/distroless/static-debian11:nonroot@sha256:63ebe035fbdd056ed682e6a87b286d07d3f05f12cb46f26b2b44fc10fc4a59ed for github.com/gardener/etcd-druid Using build config europe-docker.pkg.dev/gardener-project/public/gardener/etcd-druid for github.com/gardener/etcd-druid Building github.com/gardener/etcd-druid for linux/arm64/v8 Publishing localhost:5001/europe-docker_pkg_dev_gardener-project_public_gardener_etcd-druid:v0.23.0-8-gd2047f4a-dirty Published localhost:5001/europe-docker_pkg_dev_gardener-project_public_gardener_etcd-druid:v0.23.0-8-gd2047f4a-dirty@sha256:e4fa4b60fa3143d6dd5c3118332bfecef1162071370af880358c9a9de1ef7c8b Build [europe-docker.pkg.dev/gardener-project/public/gardener/etcd-druid] succeeded Starting test... Tags used in deployment: - europe-docker.pkg.dev/gardener-project/public/gardener/etcd-druid -> localhost:5001/europe-docker_pkg_dev_gardener-project_public_gardener_etcd-druid:v0.23.0-8-gd2047f4a-dirty@sha256:e4fa4b60fa3143d6dd5c3118332bfecef1162071370af880358c9a9de1ef7c8b Starting deploy... Helm release etcd-druid not installed. Installing... NAME: etcd-druid LAST DEPLOYED: Thu Nov 7 14:46:32 2024 NAMESPACE: e2e-test STATUS: deployed REVISION: 1 TEST SUITE: None Waiting for deployments to stabilize... - e2e-test:deployment/etcd-druid is ready. Deployments stabilized in 2.106 seconds You can also run [skaffold run --tail] to get the logs ------------------- Running e2e tests ------------------- Running Suite: e2e Suite - /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e ======================================================================================== Random Seed: 1730971000 Will run 8 of 8 specs Running in parallel across 11 processes ------------------------------ [BeforeSuite] PASSED [0.045 seconds] [BeforeSuite] /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:67 Timeline >> {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating namespace","namespace":"shoot"} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}} << Timeline ------------------------------ [BeforeSuite] PASSED [0.048 seconds] [BeforeSuite] /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:67 Timeline >> {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating namespace","namespace":"shoot"} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}} << Timeline ------------------------------ [BeforeSuite] PASSED [0.048 seconds] [BeforeSuite] /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:67 Timeline >> {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating namespace","namespace":"shoot"} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}} << Timeline ------------------------------ [BeforeSuite] PASSED [0.048 seconds] [BeforeSuite] /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:67 Timeline >> {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating namespace","namespace":"shoot"} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}} << Timeline ------------------------------ [BeforeSuite] PASSED [0.048 seconds] [BeforeSuite] /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:67 Timeline >> {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating namespace","namespace":"shoot"} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}} << Timeline ------------------------------ [BeforeSuite] PASSED [0.049 seconds] [BeforeSuite] /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:67 Timeline >> {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating namespace","namespace":"shoot"} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}} << Timeline ------------------------------ [BeforeSuite] PASSED [0.048 seconds] [BeforeSuite] /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:67 Timeline >> {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating namespace","namespace":"shoot"} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}} << Timeline ------------------------------ [BeforeSuite] PASSED [0.049 seconds] [BeforeSuite] /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:67 Timeline >> {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating namespace","namespace":"shoot"} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}} << Timeline ------------------------------ [BeforeSuite] PASSED [0.049 seconds] [BeforeSuite] /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:67 Timeline >> {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating namespace","namespace":"shoot"} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}} << Timeline ------------------------------ [BeforeSuite] PASSED [0.049 seconds] [BeforeSuite] /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:67 Timeline >> {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating namespace","namespace":"shoot"} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}} << Timeline ------------------------------ [SynchronizedAfterSuite] PASSED [0.000 seconds] [SynchronizedAfterSuite] /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:117 ------------------------------ [SynchronizedAfterSuite] PASSED [0.000 seconds] [SynchronizedAfterSuite] /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:117 ------------------------------ [BeforeSuite] PASSED [0.068 seconds] [BeforeSuite] /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:67 Timeline >> {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating namespace","namespace":"shoot"} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-server-cert-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-client-tls-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}} << Timeline ------------------------------ [SynchronizedAfterSuite] PASSED [0.001 seconds] [SynchronizedAfterSuite] /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:117 ------------------------------ • [157.535 seconds] Etcd Backup when single-node etcd is configured with provider aws Should create, test backup and delete etcd with backup /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_backup_test.go:58 Timeline >> STEP: Purge snapstore @ 11/07/24 14:46:50.621 STEP: Create etcd @ 11/07/24 14:46:50.634 {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:17+05:30","msg":"etcd is ready","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:17+05:30","msg":"Checking statefulset","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:17+05:30","msg":"Checking configmap","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:17+05:30","msg":"Checking client service","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}} STEP: Create debug pod @ 11/07/24 14:48:17.081 STEP: Check initial snapshot is available @ 11/07/24 14:48:27.123 STEP: Put keys into etcd @ 11/07/24 14:48:27.177 {"level":"info","ts":"2024-11-07T14:48:27+05:30","msg":"populating etcd with sequential key-value pairs","fromKey":"foo-1","fromValue":"bar-1","toKey":"foo-10","toValue":"bar-10"} {"level":"info","ts":"2024-11-07T14:48:31+05:30","msg":"put key-value successful","key":"foo-1","value":"bar-1"} {"level":"info","ts":"2024-11-07T14:48:31+05:30","msg":"put key-value successful","key":"foo-2","value":"bar-2"} {"level":"info","ts":"2024-11-07T14:48:32+05:30","msg":"put key-value successful","key":"foo-3","value":"bar-3"} {"level":"info","ts":"2024-11-07T14:48:32+05:30","msg":"put key-value successful","key":"foo-4","value":"bar-4"} {"level":"info","ts":"2024-11-07T14:48:32+05:30","msg":"put key-value successful","key":"foo-5","value":"bar-5"} {"level":"info","ts":"2024-11-07T14:48:33+05:30","msg":"put key-value successful","key":"foo-6","value":"bar-6"} {"level":"info","ts":"2024-11-07T14:48:33+05:30","msg":"put key-value successful","key":"foo-7","value":"bar-7"} {"level":"info","ts":"2024-11-07T14:48:33+05:30","msg":"put key-value successful","key":"foo-8","value":"bar-8"} {"level":"info","ts":"2024-11-07T14:48:33+05:30","msg":"put key-value successful","key":"foo-9","value":"bar-9"} {"level":"info","ts":"2024-11-07T14:48:34+05:30","msg":"put key-value successful","key":"foo-10","value":"bar-10"} STEP: Check snapshot after putting data into etcd @ 11/07/24 14:48:34.277 STEP: Trigger on-demand full snapshot @ 11/07/24 14:48:34.361 STEP: Put additional data into etcd @ 11/07/24 14:48:34.414 {"level":"info","ts":"2024-11-07T14:48:34+05:30","msg":"populating etcd with sequential key-value pairs","fromKey":"foo-11","fromValue":"bar-11","toKey":"foo-15","toValue":"bar-15"} {"level":"info","ts":"2024-11-07T14:48:38+05:30","msg":"put key-value successful","key":"foo-11","value":"bar-11"} {"level":"info","ts":"2024-11-07T14:48:38+05:30","msg":"put key-value successful","key":"foo-12","value":"bar-12"} {"level":"info","ts":"2024-11-07T14:48:38+05:30","msg":"put key-value successful","key":"foo-13","value":"bar-13"} {"level":"info","ts":"2024-11-07T14:48:38+05:30","msg":"put key-value successful","key":"foo-14","value":"bar-14"} {"level":"info","ts":"2024-11-07T14:48:39+05:30","msg":"put key-value successful","key":"foo-15","value":"bar-15"} STEP: Trigger on-demand delta snapshot @ 11/07/24 14:48:39.286 STEP: Test cluster restoration by deleting data directory @ 11/07/24 14:48:39.335 {"level":"info","ts":"2024-11-07T14:48:39+05:30","msg":"waiting for sts to become unready","statefulSetName":"etcd-single-node-aws"} {"level":"info","ts":"2024-11-07T14:49:03+05:30","msg":"sts is unready","statefulSetName":"etcd-single-node-aws"} {"level":"info","ts":"2024-11-07T14:49:03+05:30","msg":"waiting for sts to become ready again","statefulSetName":"etcd-single-node-aws"} {"level":"info","ts":"2024-11-07T14:49:19+05:30","msg":"sts is ready","statefulSetName":"etcd-single-node-aws"} {"level":"info","ts":"2024-11-07T14:49:19+05:30","msg":"fetching etcd key-value pairs"} {"level":"info","ts":"2024-11-07T14:49:19+05:30","msg":"fetched key-value pair from etcd","key":"foo-1","value":"bar-1"} {"level":"info","ts":"2024-11-07T14:49:20+05:30","msg":"fetched key-value pair from etcd","key":"foo-2","value":"bar-2"} {"level":"info","ts":"2024-11-07T14:49:20+05:30","msg":"fetched key-value pair from etcd","key":"foo-3","value":"bar-3"} {"level":"info","ts":"2024-11-07T14:49:20+05:30","msg":"fetched key-value pair from etcd","key":"foo-4","value":"bar-4"} {"level":"info","ts":"2024-11-07T14:49:21+05:30","msg":"fetched key-value pair from etcd","key":"foo-5","value":"bar-5"} {"level":"info","ts":"2024-11-07T14:49:21+05:30","msg":"fetched key-value pair from etcd","key":"foo-6","value":"bar-6"} {"level":"info","ts":"2024-11-07T14:49:21+05:30","msg":"fetched key-value pair from etcd","key":"foo-7","value":"bar-7"} {"level":"info","ts":"2024-11-07T14:49:21+05:30","msg":"fetched key-value pair from etcd","key":"foo-8","value":"bar-8"} {"level":"info","ts":"2024-11-07T14:49:22+05:30","msg":"fetched key-value pair from etcd","key":"foo-9","value":"bar-9"} {"level":"info","ts":"2024-11-07T14:49:22+05:30","msg":"fetched key-value pair from etcd","key":"foo-10","value":"bar-10"} {"level":"info","ts":"2024-11-07T14:49:22+05:30","msg":"fetched key-value pair from etcd","key":"foo-11","value":"bar-11"} {"level":"info","ts":"2024-11-07T14:49:23+05:30","msg":"fetched key-value pair from etcd","key":"foo-12","value":"bar-12"} {"level":"info","ts":"2024-11-07T14:49:23+05:30","msg":"fetched key-value pair from etcd","key":"foo-13","value":"bar-13"} {"level":"info","ts":"2024-11-07T14:49:23+05:30","msg":"fetched key-value pair from etcd","key":"foo-14","value":"bar-14"} {"level":"info","ts":"2024-11-07T14:49:24+05:30","msg":"fetched key-value pair from etcd","key":"foo-15","value":"bar-15"} STEP: Deleting debug pod @ 11/07/24 14:49:24.12 STEP: Deleting etcd @ 11/07/24 14:49:24.125 {"level":"info","ts":"2024-11-07T14:49:24+05:30","msg":"Checking if etcd is gone","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:49:28+05:30","msg":"Checking if statefulset is gone","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:49:28+05:30","msg":"Checking if configmap is gone","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:49:28+05:30","msg":"Checking client service is gone","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:49:28+05:30","msg":"Deleting PVCs"} << Timeline ------------------------------ [SynchronizedAfterSuite] PASSED [0.000 seconds] [SynchronizedAfterSuite] /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:117 ------------------------------ • [164.388 seconds] Etcd Compaction when compaction is enabled for single-node etcd with provider aws should test compaction on backup /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_compaction_test.go:53 Timeline >> STEP: Purge snapstore @ 11/07/24 14:46:50.621 STEP: Create etcd @ 11/07/24 14:46:50.639 {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:35+05:30","msg":"etcd is ready","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:35+05:30","msg":"Checking statefulset","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:35+05:30","msg":"Checking configmap","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:35+05:30","msg":"Checking client service","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}} STEP: Create debug pod @ 11/07/24 14:48:35.169 STEP: Check initial snapshot is available @ 11/07/24 14:48:39.181 STEP: Put keys into etcd @ 11/07/24 14:48:39.23 {"level":"info","ts":"2024-11-07T14:48:39+05:30","msg":"populating etcd with sequential key-value pairs","fromKey":"foo-1","fromValue":"bar-1","toKey":"foo-10","toValue":"bar-10"} {"level":"info","ts":"2024-11-07T14:48:42+05:30","msg":"put key-value successful","key":"foo-1","value":"bar-1"} {"level":"info","ts":"2024-11-07T14:48:43+05:30","msg":"put key-value successful","key":"foo-2","value":"bar-2"} {"level":"info","ts":"2024-11-07T14:48:43+05:30","msg":"put key-value successful","key":"foo-3","value":"bar-3"} {"level":"info","ts":"2024-11-07T14:48:43+05:30","msg":"put key-value successful","key":"foo-4","value":"bar-4"} {"level":"info","ts":"2024-11-07T14:48:44+05:30","msg":"put key-value successful","key":"foo-5","value":"bar-5"} {"level":"info","ts":"2024-11-07T14:48:44+05:30","msg":"put key-value successful","key":"foo-6","value":"bar-6"} {"level":"info","ts":"2024-11-07T14:48:44+05:30","msg":"put key-value successful","key":"foo-7","value":"bar-7"} {"level":"info","ts":"2024-11-07T14:48:45+05:30","msg":"put key-value successful","key":"foo-8","value":"bar-8"} {"level":"info","ts":"2024-11-07T14:48:45+05:30","msg":"put key-value successful","key":"foo-9","value":"bar-9"} {"level":"info","ts":"2024-11-07T14:48:45+05:30","msg":"put key-value successful","key":"foo-10","value":"bar-10"} STEP: Check snapshot after putting data into etcd @ 11/07/24 14:48:45.763 STEP: Put additional data into etcd @ 11/07/24 14:48:45.85 {"level":"info","ts":"2024-11-07T14:48:45+05:30","msg":"populating etcd with sequential key-value pairs","fromKey":"foo-11","fromValue":"bar-11","toKey":"foo-15","toValue":"bar-15"} {"level":"info","ts":"2024-11-07T14:48:49+05:30","msg":"put key-value successful","key":"foo-11","value":"bar-11"} {"level":"info","ts":"2024-11-07T14:48:49+05:30","msg":"put key-value successful","key":"foo-12","value":"bar-12"} {"level":"info","ts":"2024-11-07T14:48:50+05:30","msg":"put key-value successful","key":"foo-13","value":"bar-13"} {"level":"info","ts":"2024-11-07T14:48:50+05:30","msg":"put key-value successful","key":"foo-14","value":"bar-14"} {"level":"info","ts":"2024-11-07T14:48:50+05:30","msg":"put key-value successful","key":"foo-15","value":"bar-15"} STEP: Trigger on-demand delta snapshot @ 11/07/24 14:48:50.694 {"level":"info","ts":"2024-11-07T14:48:50+05:30","msg":"waiting for compaction job to become successful"} {"level":"info","ts":"2024-11-07T14:49:25+05:30","msg":"compaction job is successful"} STEP: Put additional data into etcd @ 11/07/24 14:49:25.932 {"level":"info","ts":"2024-11-07T14:49:25+05:30","msg":"populating etcd with sequential key-value pairs","fromKey":"foo-16","fromValue":"bar-16","toKey":"foo-20","toValue":"bar-20"} {"level":"info","ts":"2024-11-07T14:49:29+05:30","msg":"put key-value successful","key":"foo-16","value":"bar-16"} {"level":"info","ts":"2024-11-07T14:49:29+05:30","msg":"put key-value successful","key":"foo-17","value":"bar-17"} {"level":"info","ts":"2024-11-07T14:49:30+05:30","msg":"put key-value successful","key":"foo-18","value":"bar-18"} {"level":"info","ts":"2024-11-07T14:49:30+05:30","msg":"put key-value successful","key":"foo-19","value":"bar-19"} {"level":"info","ts":"2024-11-07T14:49:30+05:30","msg":"put key-value successful","key":"foo-20","value":"bar-20"} STEP: Trigger next on-demand delta snapshot @ 11/07/24 14:49:30.868 STEP: Verify that there are new delta snapshots as compaction is not triggered yet because delta events have not reached next 15 revision @ 11/07/24 14:49:30.914 STEP: Deleting debug pod @ 11/07/24 14:49:30.957 STEP: Deleting etcd @ 11/07/24 14:49:30.96 {"level":"info","ts":"2024-11-07T14:49:30+05:30","msg":"Checking if etcd is gone","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:49:34+05:30","msg":"Checking if statefulset is gone","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:49:34+05:30","msg":"Checking if configmap is gone","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:49:34+05:30","msg":"Checking client service is gone","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:49:34+05:30","msg":"Deleting PVCs"} << Timeline ------------------------------ • [229.038 seconds] Etcd Single-node etcd configuration Scaling up from single-node to multi-node without TLS should scale a single-node etcd (TLS not enabled for peerUrl) to a multi-node etcd cluster (TLS not enabled for peerUrl) /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_multi_node_test.go:138 Timeline >> STEP: Purge snapstore @ 11/07/24 14:46:50.62 STEP: Creating a single-node etcd @ 11/07/24 14:46:50.636 {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:33+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:33+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:33+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:33+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}} STEP: Scaling up a healthy cluster (from 1 to 3 replicas) @ 11/07/24 14:48:33.137 {"level":"info","ts":"2024-11-07T14:50:11+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:35+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:35+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:35+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:35+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}} STEP: Deleting etcd @ 11/07/24 14:50:35.622 {"level":"info","ts":"2024-11-07T14:50:35+05:30","msg":"Checking if etcd is gone","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:39+05:30","msg":"Checking if statefulset is gone","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:39+05:30","msg":"Checking if configmap is gone","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:39+05:30","msg":"Checking client service is gone","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:39+05:30","msg":"Deleting PVCs"} << Timeline ------------------------------ [SynchronizedAfterSuite] PASSED [0.000 seconds] [SynchronizedAfterSuite] /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:117 ------------------------------ • [261.226 seconds] Etcd Single-node etcd configuration Scaling down from single-node to zero and back up without TLS executes scaling down to 0 and back up to 3 replicas without TLS /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_multi_node_test.go:189 Timeline >> STEP: Purge snapstore @ 11/07/24 14:46:50.62 STEP: Creating a single-node etcd @ 11/07/24 14:46:50.631 {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:27+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:27+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:27+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:27+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} STEP: Scaling down a healthy cluster (from 1 to 0 replica) @ 11/07/24 14:48:27.134 {"level":"info","ts":"2024-11-07T14:48:29+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:31+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:31+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:31+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:31+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} STEP: Scaling up cluster (from 0 to 1 replica) @ 11/07/24 14:48:31.161 {"level":"info","ts":"2024-11-07T14:48:33+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:57+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:57+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:57+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:57+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} STEP: Scaling up a healthy cluster (from 1 to 3 replica) @ 11/07/24 14:48:57.244 {"level":"info","ts":"2024-11-07T14:50:35+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:57+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:57+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:57+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:57+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} STEP: Deleting etcd @ 11/07/24 14:50:57.734 {"level":"info","ts":"2024-11-07T14:50:57+05:30","msg":"Checking if etcd is gone","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:51:11+05:30","msg":"Checking if statefulset is gone","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:51:11+05:30","msg":"Checking if configmap is gone","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:51:11+05:30","msg":"Checking client service is gone","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:51:11+05:30","msg":"Deleting PVCs"} << Timeline ------------------------------ [SynchronizedAfterSuite] PASSED [0.000 seconds] [SynchronizedAfterSuite] /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:117 ------------------------------ • [373.654 seconds] Etcd when multi-node is configured should perform etcd operations excluding zero downtime maintenance /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_multi_node_test.go:51 Timeline >> STEP: Purge snapstore @ 11/07/24 14:46:50.616 STEP: Creating etcd @ 11/07/24 14:46:50.63 {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:07+05:30","msg":"etcd is ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:07+05:30","msg":"Checking statefulset","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:07+05:30","msg":"Checking configmap","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:07+05:30","msg":"Checking client service","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} STEP: Hibernating etcd (Scaling down from 3 replicas to 0) @ 11/07/24 14:50:07.485 {"level":"info","ts":"2024-11-07T14:50:07+05:30","msg":"Waiting for statefulset spec to reflect change in replicas to 0","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:09+05:30","msg":"Checking etcd","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:17+05:30","msg":"Checking statefulset","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:17+05:30","msg":"etcd is hibernated","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} STEP: Waking up etcd (Scaling up from 0 to 3 replicas) @ 11/07/24 14:50:17.532 {"level":"info","ts":"2024-11-07T14:50:19+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:51:47+05:30","msg":"etcd is ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:51:47+05:30","msg":"Checking statefulset","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:51:47+05:30","msg":"Checking configmap","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:51:47+05:30","msg":"Checking client service","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} STEP: Member restart with data-dir/pvc intact @ 11/07/24 14:51:47.927 {"level":"info","ts":"2024-11-07T14:51:47+05:30","msg":"Deleting one member pod","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:51:47+05:30","msg":"waiting for sts to become unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:51:49+05:30","msg":"sts is unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:51:49+05:30","msg":"Waiting for at least one etcd member to become unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:51:53+05:30","msg":"at least one etcd member is unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:51:53+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:14+05:30","msg":"etcd is ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:14+05:30","msg":"Checking statefulset","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:14+05:30","msg":"Checking configmap","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:14+05:30","msg":"Checking client service","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} STEP: Single member restoration @ 11/07/24 14:52:14.023 {"level":"info","ts":"2024-11-07T14:52:14+05:30","msg":"Creating debug pod","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:18+05:30","msg":"Deleting member directory of one member pod","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:18+05:30","msg":"waiting for sts to become unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:34+05:30","msg":"sts is unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:34+05:30","msg":"Waiting for at least one etcd member to become unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:38+05:30","msg":"at least one etcd member is unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:38+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:54+05:30","msg":"etcd is ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:54+05:30","msg":"Checking statefulset","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:54+05:30","msg":"Checking configmap","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:54+05:30","msg":"Checking client service","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} STEP: Deleting debug pod @ 11/07/24 14:52:54.225 STEP: Deleting etcd @ 11/07/24 14:52:54.227 {"level":"info","ts":"2024-11-07T14:52:54+05:30","msg":"Checking if etcd is gone","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:53:04+05:30","msg":"Checking if statefulset is gone","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:53:04+05:30","msg":"Checking if configmap is gone","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:53:04+05:30","msg":"Checking client service is gone","etcd":{"name":"multi-node-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:53:04+05:30","msg":"Deleting PVCs"} << Timeline ------------------------------ [SynchronizedAfterSuite] PASSED [0.000 seconds] [SynchronizedAfterSuite] /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:117 ------------------------------ • [379.657 seconds] Etcd Single-node etcd configuration Scaling up from single-node to multi-node with TLS executes scaling up with TLS /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_multi_node_test.go:164 Timeline >> STEP: Purge snapstore @ 11/07/24 14:46:50.62 STEP: Creating a single-node etcd @ 11/07/24 14:46:50.632 {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:27+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:27+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:27+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:27+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}} STEP: Scaling up a healthy cluster (from 1 to 3 replicas) with TLS enabled for peerUrl @ 11/07/24 14:48:27.134 {"level":"info","ts":"2024-11-07T14:51:13+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:54+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:54+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:54+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:54+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}} STEP: Deleting etcd @ 11/07/24 14:52:54.204 {"level":"info","ts":"2024-11-07T14:52:54+05:30","msg":"Checking if etcd is gone","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:53:10+05:30","msg":"Checking if statefulset is gone","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:53:10+05:30","msg":"Checking if configmap is gone","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:53:10+05:30","msg":"Checking client service is gone","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:53:10+05:30","msg":"Deleting PVCs"} << Timeline ------------------------------ [SynchronizedAfterSuite] PASSED [0.000 seconds] [SynchronizedAfterSuite] /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:117 ------------------------------ • [415.862 seconds] Etcd Single-node etcd configuration Scaling down from single-node to zero and back up with TLS should scale down a single-node etcd to 0 replica, then scale up from 0->1 replica and then from 1->3 replicas with TLS enabled for cluster peerUrl /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_multi_node_test.go:224 Timeline >> STEP: Purge snapstore @ 11/07/24 14:46:50.621 STEP: Creating a single-node etcd @ 11/07/24 14:46:50.636 {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:35+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:35+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:35+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:35+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} STEP: Scaling down a healthy cluster (from 1 to 0 replica) @ 11/07/24 14:48:35.156 {"level":"info","ts":"2024-11-07T14:48:37+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:39+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:39+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:39+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:48:39+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} STEP: Scaling up cluster (from 0 to 1 replica) @ 11/07/24 14:48:39.197 {"level":"info","ts":"2024-11-07T14:48:41+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:49:05+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:49:05+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:49:05+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:49:05+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} STEP: Scaling up a healthy cluster (from 1 to 3 replica) with TLS enabled for peerUrl @ 11/07/24 14:49:05.294 {"level":"info","ts":"2024-11-07T14:51:51+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:53:32+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:53:32+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:53:32+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:53:32+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} STEP: Deleting etcd @ 11/07/24 14:53:32.364 {"level":"info","ts":"2024-11-07T14:53:32+05:30","msg":"Checking if etcd is gone","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:53:46+05:30","msg":"Checking if statefulset is gone","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:53:46+05:30","msg":"Checking if configmap is gone","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:53:46+05:30","msg":"Checking client service is gone","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:53:46+05:30","msg":"Deleting PVCs"} << Timeline ------------------------------ [SynchronizedAfterSuite] PASSED [0.000 seconds] [SynchronizedAfterSuite] /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:117 ------------------------------ • [436.551 seconds] Etcd when multi-node is configured should perform zero downtime maintenance operation: defragmentation /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/etcd_multi_node_test.go:94 Timeline >> STEP: Purge snapstore @ 11/07/24 14:46:50.62 STEP: Creating etcd @ 11/07/24 14:46:50.636 {"level":"info","ts":"2024-11-07T14:46:50+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:01+05:30","msg":"etcd is ready","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:01+05:30","msg":"Checking statefulset","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:01+05:30","msg":"Checking configmap","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:50:01+05:30","msg":"Checking client service","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} STEP: Deploying etcd zero downtime validator job @ 11/07/24 14:50:01.461 {"level":"info","ts":"2024-11-07T14:50:01+05:30","msg":"Creating job to ensure etcd zero downtime","job":"etcd-zero-down-time-validator-rolling-update"} {"level":"info","ts":"2024-11-07T14:50:15+05:30","msg":"Job is ready","job":"etcd-zero-down-time-validator-rolling-update"} STEP: Conducting zero downtime rolling updates @ 11/07/24 14:50:15.527 {"level":"info","ts":"2024-11-07T14:50:57+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:51:17+05:30","msg":"etcd is ready","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:51:17+05:30","msg":"Checking statefulset","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:51:17+05:30","msg":"Checking configmap","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:51:17+05:30","msg":"Checking client service","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:51:17+05:30","msg":"Etcd Cluster is healthy and there is no downtime","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} STEP: Performing zero downtime maintenance operation: defragmentation @ 11/07/24 14:51:17.853 {"level":"info","ts":"2024-11-07T14:51:17+05:30","msg":"Configuring defragmentation schedule for every 1 minute","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:00+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:22+05:30","msg":"etcd is ready","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:22+05:30","msg":"Checking statefulset","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:22+05:30","msg":"Checking configmap","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:22+05:30","msg":"Checking client service","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:22+05:30","msg":"Etcd Cluster is healthy and there is no downtime","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:52:22+05:30","msg":"Waiting for defragmentation to finish","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:54:01+05:30","msg":"Defragmentation is finished","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:54:01+05:30","msg":"Checking for any Etcd downtime","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:54:01+05:30","msg":"Etcd Cluster is healthy and there is no downtime","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} STEP: Deleting etcd @ 11/07/24 14:54:01.094 {"level":"info","ts":"2024-11-07T14:54:01+05:30","msg":"Checking if etcd is gone","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:54:07+05:30","msg":"Checking if statefulset is gone","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:54:07+05:30","msg":"Checking if configmap is gone","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:54:07+05:30","msg":"Checking client service is gone","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}} {"level":"info","ts":"2024-11-07T14:54:07+05:30","msg":"Deleting PVCs"} << Timeline ------------------------------ [SynchronizedAfterSuite] PASSED [0.000 seconds] [SynchronizedAfterSuite] /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:117 ------------------------------ [SynchronizedAfterSuite] PASSED [316.375 seconds] [SynchronizedAfterSuite] /Users/renormalize/go/src/github.com/gardener/etcd-druid/test/e2e/suite_test.go:117 Timeline >> {"level":"info","ts":"2024-11-07T14:54:07+05:30","msg":"Checking for Etcd resources before deleting namespace","namespace":"shoot","namespace":"shoot"} {"level":"info","ts":"2024-11-07T14:54:07+05:30","msg":"No Etcd resources found; proceeding with namespace deletion","namespace":"shoot","namespace":"shoot"} << Timeline ------------------------------ Ran 8 of 8 Specs in 480.822 seconds SUCCESS! -- 8 Passed | 0 Failed | 0 Pending | 0 Skipped Ginkgo ran 1 suite in 8m10.919516666s Test Suite Passed Exporting logs for cluster "etcd-druid-e2e" to: /tmp/etcd-druid-e2e Deleting kind cluster... Deleting cluster "etcd-druid-e2e" ... Deleted nodes: ["etcd-druid-e2e-control-plane"] Stopping running container kind-registry... Removing container kind-registry... ```

[anveshreddy18]

/assign