search

tkestack / tke

Native Kubernetes container management platform supporting multi-tenant and multi-cluster
Other
1.47k stars 335 forks source link

cluster controller panic when gc deleted cluster #1091

Closed zizhuoy closed 3 years ago

zizhuoy commented 3 years ago

What happened: platform-controller panic when GC deleted cluster

panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x1a8 pc=0x1c360a5]

goroutine 571 [running]:
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
        /data/gopath/pkg/mod/k8s.io/apimachinery@v0.18.2/pkg/util/runtime/runtime.go:55 +0x105
panic(0x1ebef20, 0x36e2700)
        /data/go/src/runtime/panic.go:679 +0x1b2
tkestack.io/tke/pkg/platform/types/v1.(*Cluster).RESTConfig(0xc0008178f0, 0xc0009f96b8, 0x0, 0x0, 0x0)
        /data/tke/pkg/platform/types/v1/types.go:140 +0x55
tkestack.io/tke/pkg/platform/types/v1.(*Cluster).Clientset(0xc0008178f0, 0xc0009f9928, 0x184a8bd, 0xc001138000, 0x44e718)
        /data/tke/pkg/platform/types/v1/types.go:81 +0x88
tkestack.io/tke/pkg/platform/provider/imported/cluster.(*Provider).EnsureCleanClusterMark(0xc00056cf28, 0x2592500, 0xc0009823c0, 0xc0008178f0, 0xfd0ee8f, 0x28cee664ed37ed)
        /data/tke/pkg/platform/provider/imported/cluster/delete.go:29 +0x2b
tkestack.io/tke/pkg/platform/provider/cluster.(*DelegateProvider).OnDelete(0xc0007be7e0, 0x2592500, 0xc0005488a0, 0xc0008178f0, 0x0, 0x0)
        /data/tke/pkg/platform/provider/cluster/interface.go:229 +0x1e3
tkestack.io/tke/pkg/platform/controller/cluster/deletion.deleteClusterProvider(0x2592500, 0xc0005488a0, 0xc0003d1ec0, 0xc0005b0000, 0x0, 0x0)
        /data/tke/pkg/platform/controller/cluster/deletion/cluster_deleter.go:398 +0x139
tkestack.io/tke/pkg/platform/controller/cluster/deletion.(*clusterDeleter).deleteAllContent(0xc0003d1ec0, 0x2592500, 0xc0005488a0, 0xc0005b0000, 0xc000a7dbb0, 0xc0005b0000)
        /data/tke/pkg/platform/controller/cluster/deletion/cluster_deleter.go:258 +0x121
tkestack.io/tke/pkg/platform/controller/cluster/deletion.(*clusterDeleter).Delete(0xc0003d1ec0, 0x2592500, 0xc0005488a0, 0xc0002b0330, 0xc, 0x0, 0x0)
        /data/tke/pkg/platform/controller/cluster/deletion/cluster_deleter.go:122 +0x25e
tkestack.io/tke/pkg/platform/controller/cluster.(*Controller).reconcile(0xc00098e480, 0x2592500, 0xc000548840, 0xc0002b0330, 0xc, 0xc000546000, 0x0, 0x0)
        /data/tke/pkg/platform/controller/cluster/cluster_controller.go:243 +0x36a
tkestack.io/tke/pkg/platform/controller/cluster.(*Controller).syncCluster(0xc00098e480, 0xc0002b0330, 0xc, 0x0, 0x0)
        /data/tke/pkg/platform/controller/cluster/cluster_controller.go:228 +0x3ab
tkestack.io/tke/pkg/platform/controller/cluster.(*Controller).processNextWorkItem(0xc00098e480, 0x40ad00)
        /data/tke/pkg/platform/controller/cluster/cluster_controller.go:192 +0xfb
tkestack.io/tke/pkg/platform/controller/cluster.(*Controller).worker(0xc00098e480)
        /data/tke/pkg/platform/controller/cluster/cluster_controller.go:181 +0x2b
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc000a80330)
        /data/gopath/pkg/mod/k8s.io/apimachinery@v0.18.2/pkg/util/wait/wait.go:155 +0x5e
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc000a80330, 0x2534de0, 0xc000548780, 0xc000c76001, 0xc0004177a0)
        /data/gopath/pkg/mod/k8s.io/apimachinery@v0.18.2/pkg/util/wait/wait.go:156 +0xa3
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000a80330, 0x3b9aca00, 0x0, 0x1, 0xc0004177a0)
        /data/gopath/pkg/mod/k8s.io/apimachinery@v0.18.2/pkg/util/wait/wait.go:133 +0xe2
k8s.io/apimachinery/pkg/util/wait.Until(0xc000a80330, 0x3b9aca00, 0xc0004177a0)
        /data/gopath/pkg/mod/k8s.io/apimachinery@v0.18.2/pkg/util/wait/wait.go:90 +0x4d
created by tkestack.io/tke/pkg/platform/controller/cluster.(*Controller).Run
        /data/tke/pkg/platform/controller/cluster/cluster_controller.go:164 +0x203

What you expected to happen: controller working fine.

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment:

zizhuoy commented 3 years ago

/assign