encounter the panic

[linchuan4028]

 6  0x000000000044e76a in runtime.crash
    at /usr/local/go/src/runtime/signal_unix.go:821
 7  0x00000000004377f4 in runtime.fatalpanic
    at /usr/local/go/src/runtime/panic.go:1216
 8  0x000000000043718d in runtime.gopanic
    at /usr/local/go/src/runtime/panic.go:1064
 9  0x0000000000434f33 in runtime.goPanicIndex
    at /usr/local/go/src/runtime/panic.go:88
10  0x000000000083d507 in github.com/aerospike/aerospike-client-go.(*singleConnectionHeap).DropIdleTail
    at /root/go/pkg/mod/github.com/aerospike/aerospike-client-go@v2.9.0+incompatible/connection_heap.go:118
11  0x000000000083dcc1 in github.com/aerospike/aerospike-client-go.(*connectionHeap).DropIdle
    at /root/go/pkg/mod/github.com/aerospike/aerospike-client-go@v2.9.0+incompatible/connection_heap.go:226
12  0x000000000085b7d3 in github.com/aerospike/aerospike-client-go.(*Node).dropIdleConnections
    at /root/go/pkg/mod/github.com/aerospike/aerospike-client-go@v2.9.0+incompatible/node.go:487
13  0x0000000000858df8 in github.com/aerospike/aerospike-client-go.(*Node).Refresh
    at /root/go/pkg/mod/github.com/aerospike/aerospike-client-go@v2.9.0+incompatible/node.go:138
14  0x0000000000887927 in github.com/aerospike/aerospike-client-go.(*Cluster).tend.func1
    at /root/go/pkg/mod/github.com/aerospike/aerospike-client-go@v2.9.0+incompatible/cluster.go:258
15  0x000000000046a691 in runtime.goexit
    at /usr/local/go/src/runtime/asm_amd64.s:1373

[khaf]

Can you please provide a bit more information about the environment in which this happened? Was it tests? Did you compile via -race switch?

[linchuan4028]

It's on prd env. No compile via -race.

[khaf]

Can you paste the whole panic stack trace? Especially the error message.

[linchuan4028]

the code segment in connenction_heap.go

// DropIdleTail closes idle connection in tail.
// It will return true if tail connection was idle and dropped
func (h *singleConnectionHeap) DropIdleTail() bool {
    h.mutex.Lock()
    defer h.mutex.Unlock()

    // if heap is not empty
    if h.full || (h.tail != h.head) {
        conn := h.data[(h.tail+1)%h.size]

below is the stack value

*github.com/aerospike/aerospike-client-go.singleConnectionHeap {
    head: 0,
    tail: 0,
    data: []*github.com/aerospike/aerospike-client-go.Connection len: 0, cap: 0, nil,
    size: 32,
    full: true,
    mutex: sync.Mutex {state: 0, sema: 0},}

this line conn := h.data[(h.tail+1)%h.size] show index out of range

[khaf]

Thanks for the report, I'm on it.

[khaf]

This should be fixed in the latest v3.3.1 release. Thanks for your patience.