ydb-platform / ydb

YDB is an open source Distributed SQL Database that combines high availability and scalability with strong consistency and ACID transactions
https://ydb.tech
Apache License 2.0
3.96k stars 543 forks source link

bug: CrashLoopBackOff after Storage creation #23

Open fireharp opened 2 years ago

fireharp commented 2 years ago

Bug Report

YDB Kubernetes Operator version:

NAME            NAMESPACE   REVISION    UPDATED                                 STATUS      CHART                           APP VERSION
prom            default     1           2022-04-20 22:49:30.209714 +0300 MSK    deployed    kube-prometheus-stack-34.10.0   0.55.0     
ydb-operator    default     1           2022-04-20 22:49:57.786045 +0300 MSK    deployed    operator-0.4.5                  0.4.9      

Environment

Minikube (MacOS 10.15.7), GKE, GKE autopilot

Current behavior:

This happens after any cluster setup based on https://ydb.tech/en/docs/getting_started/self_hosted/ydb_minikube or https://ydb.tech/en/docs/deploy/orchestrated/yc_managed_kubernetes

panic: assignment to entry in nil map after initial Storage setup

kubectl get storages.ydb.tech shows always PENDING

Screenshot 2022-04-21 at 11 58 53

ydb-operator get into CrashLoopBackOff

Screenshot 2022-04-21 at 11 59 10

Steps to reproduce:

follow this guide https://ydb.tech/en/docs/getting_started/self_hosted/ydb_minikube

After storage create step

kubectl create -f - <<EOF
apiVersion: ydb.tech/v1alpha1
kind: Storage
metadata: 
  name: storage-sample
spec: 
  dataStore: 
    - 
      accessModes: 
        - ReadWriteOnce
      resources: 
        requests: 
          storage: 10Gi
      volumeMode: Block
  erasure: none
  nodes: 4
  version: "21.4.30"
EOF

Related code:

Full error code

panic: assignment to entry in nil map

goroutine 361 [running]:
github.com/ydb-platform/ydb-kubernetes-operator/internal/configuration.Build(0xc0005234c8, 0xc0005234c8)
    /workspace/internal/configuration/configuration.go:76 +0xea
github.com/ydb-platform/ydb-kubernetes-operator/internal/resources.(*StorageClusterBuilder).GetResourceBuilders(0xc0000b2498)
    /workspace/internal/resources/storage.go:59 +0xd9
github.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*StorageReconciler).handleResourcesSync(0xc0000b9810, {0x19c3838, 0xc000996e70}, 0xc0000b2498)
    /workspace/internal/controllers/storage/sync.go:168 +0x7a
github.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*StorageReconciler).Sync(0xc0000b94f0, {0x19c3838, 0xc000996e70}, 0xc000250879)
    /workspace/internal/controllers/storage/sync.go:69 +0xc5
github.com/ydb-platform/ydb-kubernetes-operator/internal/controllers/storage.(*StorageReconciler).Reconcile(0xc0000b9810, {0x19c3838, 0xc000996e70}, {{{0xc000250879, 0x1682520}, {0xc000250860, 0xc0002a7300}}})
    /workspace/internal/controllers/storage/controller.go:64 +0x1eb
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0xc00024d680, {0x19c3838, 0xc000996de0}, {{{0xc000250879, 0x1682520}, {0xc000250860, 0xc0004bc8c0}}})
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.10.0/pkg/internal/controller/controller.go:114 +0x222
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc00024d680, {0x19c3790, 0xc0005bdd80}, {0x1608f40, 0xc00097a020})
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.10.0/pkg/internal/controller/controller.go:311 +0x2f2
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc00024d680, {0x19c3790, 0xc0005bdd80})
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.10.0/pkg/internal/controller/controller.go:266 +0x205
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.10.0/pkg/internal/controller/controller.go:227 +0x85
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.10.0/pkg/internal/controller/controller.go:223 +0x354
Screenshot 2022-04-21 at 11 58 00 Screenshot 2022-04-21 at 11 59 57
mvgorbunov commented 2 years ago

We've update documentation - try to use configs examples for minicube and managed k8s. We will also add configs validation in operator, thanks for your report.