ydb-platform / ydb-kubernetes-operator

YDB Operator allows you to deploy your own YDB cluster in Kubernetes
Other
46 stars 16 forks source link

bug:   Развернул 4х2 конфигурацию через оператор и при создании таблицы "database doesn't have storage pools at all to create tablet channels to storage pool binding by profile id" #185

Closed dangerousvasil closed 6 months ago

dangerousvasil commented 6 months ago

Bug Report

YDB Kubernetes Operator version:

latest

Environment

kubernetes

Current behavior:

click operator + ydb operator

Expected behavior:

click and ydb work fine

Steps to reproduce:

deploy click deploy ydb

Related code:

2024-03-04T15:05:59Z    INFO    running step checkStorageFrozen for Storage {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2"}
2024-03-04T15:05:59Z    INFO    running step handlePauseResume for Storage  {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2"}
2024-03-04T15:05:59Z    INFO    running step handleResourcesSync    {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2"}
2024-03-04T15:05:59Z    INFO    resource sync complete  {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2"}
2024-03-04T15:05:59Z    INFO    running step syncNodeSetSpecInline  {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2"}
2024-03-04T15:05:59Z    INFO    syncNodeSetSpecInline complete  {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2"}
2024-03-04T15:05:59Z    INFO    running step setInitialStatus   {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2"}
2024-03-04T15:05:59Z    INFO    running step waitForStatefulSetToScale for Storage  {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2"}
2024-03-04T15:05:59Z    INFO    running step getYDBCredentials  {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2"}
2024-03-04T15:05:59Z    INFO    running step runSelfCheck   {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2"}
2024-03-04T15:05:59Z    INFO    running step runInitScripts {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2"}
2024-03-04T15:05:59Z    DEBUG   events  SelfCheck result: GOOD, issues found: 8 {"type": "Normal", "object": {"kind":"Storage","namespace":"stage","name":"valun-ydb","uid":"be602273-3b00-4e61-b1cc-3f723a488666","apiVersion":"ydb.tech/v1alpha1","resourceVersion":"85119616"}, "reason": "SelfCheck"}
2024-03-04T15:05:59Z    ERROR   unexpected Sync error   {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2", "error": "failed to stream execution results back, stdout:\n\"\"stderr:\n\"\": container ydb-storage is not valid for pod chi-backend-shard-0-0-0", "errorVerbose": "container ydb-storage is not valid for pod chi-backend-shard-0-0-0\nfailed to stream execution results back, stdout:\n\"\"stderr:\n\"\"\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/exec.InPod\n\t/workspace/internal/exec/exec.go:59\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*Reconciler).initializeStorage\n\t/workspace/internal/controllers/storage/init.go:167\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*Reconciler).handleFirstStart\n\t/workspace/internal/controllers/storage/sync.go:579\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*Reconciler).Sync\n\t/workspace/internal/controllers/storage/sync.go:61\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*Reconciler).Reconcile\n\t/workspace/internal/controllers/storage/controller.go:71\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.1/pkg/internal/controller/controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.1/pkg/internal/controller/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.1/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.1/pkg/internal/controller/controller.go:235\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1594"}
github.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*Reconciler).Reconcile
    /workspace/internal/controllers/storage/controller.go:73
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.1/pkg/internal/controller/controller.go:122
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.1/pkg/internal/controller/controller.go:323
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.1/pkg/internal/controller/controller.go:274
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.1/pkg/internal/controller/controller.go:235
2024-03-04T15:05:59Z    ERROR   Reconciler error    {"controller": "storage", "controllerGroup": "ydb.tech", "controllerKind": "Storage", "Storage": {"name":"valun-ydb","namespace":"stage"}, "namespace": "stage", "name": "valun-ydb", "reconcileID": "23c1ab8e-0bb2-4bb4-8506-fc0367f4f6b2", "error": "failed to stream execution results back, stdout:\n\"\"stderr:\n\"\": container ydb-storage is not valid for pod chi-backend-shard-0-0-0", "errorVerbose": "container ydb-storage is not valid for pod chi-backend-shard-0-0-0\nfailed to stream execution results back, stdout:\n\"\"stderr:\n\"\"\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/exec.InPod\n\t/workspace/internal/exec/exec.go:59\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*Reconciler).initializeStorage\n\t/workspace/internal/controllers/storage/init.go:167\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*Reconciler).handleFirstStart\n\t/workspace/internal/controllers/storage/sync.go:579\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*Reconciler).Sync\n\t/workspace/internal/controllers/storage/sync.go:61\ngithub.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*Reconciler).Reconcile\n\t/workspace/internal/controllers/storage/controller.go:71\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.1/pkg/internal/controller/controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.1/pkg/internal/controller/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.1/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.1/pkg/internal/controller/controller.go:235\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1594"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.1/pkg/internal/controller/controller.go:329
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.1/pkg/internal/controller/controller.go:274
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.1/pkg/internal/controller/controller.go:235
Logs from 04.03.2024, 14:12:38

Other information:

Пытаюсь создать таблицу НО

database doesn't have storage pools at all to create tablet channels to storage pool binding by profile id

kobzonega commented 6 months ago

Ошибка известная, происходила из-за бага, которую занесли в одном из последних обновлений. В качестве label selector для подов передавалось значение labels из объекта Storage. Исправили передачей значений из podTemplate https://github.com/ydb-platform/ydb-kubernetes-operator/pull/184

Попробуйте использовать новую версию helm чарта (начиная с 0.4.42)