search

tsuna / gohbase

Pure-Go HBase client
Apache License 2.0
736 stars 213 forks source link

gohbase doesn't handle active hbasemaster going away/changing #265

Open dethi opened 3 months ago

dethi commented 3 months ago

This sometimes cause the hbase-k8s-operator to get stuck in gohbase retry loop forever when the active master is restarted

https://github.com/timoha/hbase-k8s-operator/issues/19

2024-01-29 11:08:27.276    goroutine 406 [sync.Cond.Wait, 46 minutes]:
...
2024-01-29 11:08:27.194    [github.com/timoha/hbase-k8s-operator/controllers.(*HBaseReconciler).pickRegionServerToDelete(0xc0007f1880](http://github.com/timoha/hbase-k8s-operator/controllers.%28*HBaseReconciler%29.pickRegionServerToDelete%280xc0007f1880), {0x1a91cb8?, 0xc001616b70?}, {0xc001d42c00, 0x4?, 0x4?}, {0x0, 0x0, 0xc0004ff590?})
2024-01-29 11:08:27.194        /go/pkg/mod/[github.com/tsuna/gohbase@v0.0.0-20230329174459-f15274ad1309/admin_client.go:277](http://github.com/tsuna/gohbase@v0.0.0-20230329174459-f15274ad1309/admin_client.go:277) +0x27 fp=0xc0000eccd8 sp=0xc0000eccb0 pc=0x14acd07
2024-01-29 11:08:27.194    [github.com/tsuna/gohbase.(*client).SetBalancer(0x11](http://github.com/tsuna/gohbase.%28*client%29.SetBalancer%280x11)?, 0x1?)
2024-01-29 11:08:27.194        /go/pkg/mod/[github.com/tsuna/gohbase@v0.0.0-20230329174459-f15274ad1309/rpc.go:100](http://github.com/tsuna/gohbase@v0.0.0-20230329174459-f15274ad1309/rpc.go:100) +0x31e fp=0xc0000eccb0 sp=0xc0000ecb38 pc=0x14b04de
2024-01-29 11:08:27.194    [github.com/tsuna/gohbase.(*client).SendRPC(0xc000c2da00](http://github.com/tsuna/gohbase.%28*client%29.SendRPC%280xc000c2da00), {0x1a982c0, 0xc000a65380})
2024-01-29 11:08:27.194        /go/pkg/mod/[github.com/tsuna/gohbase@v0.0.0-20230329174459-f15274ad1309/rpc.go:602](http://github.com/tsuna/gohbase@v0.0.0-20230329174459-f15274ad1309/rpc.go:602) +0x96 fp=0xc0000ecb38 sp=0xc0000ecac8 pc=0x14b5096
2024-01-29 11:08:27.194    [github.com/tsuna/gohbase.sleepAndIncreaseBackoff({0x1a91cb8](http://github.com/tsuna/gohbase.sleepAndIncreaseBackoff%28%7B0x1a91cb8), 0xc000a3de90}, 0x7ba65ba00)
2024-01-29 11:08:27.194        /usr/local/go/src/runtime/select.go:328 +0x7bc fp=0xc0000ecac8 sp=0xc0000ec988 pc=0x44afdc
2024-01-29 11:08:27.194    runtime.selectgo(0xc0000ecb08, 0xc0000ecaf8, 0xc000a65380?, 0x0, 0x1a982c0?, 0x1)
2024-01-29 11:08:27.194        /usr/local/go/src/runtime/proc.go:363 +0xd6 fp=0xc0000ec988 sp=0xc0000ec968 pc=0x43bb76
2024-01-29 11:08:27.194    runtime.gopark(0xc0000ecb08?, 0x2?, 0x0?, 0x30?, 0xc0000ecafc?)