apache / servicecomb-service-center

A standalone service center to allow services to register their instance information and to discover providers of a given service
Apache License 2.0
1.34k stars 342 forks source link

function StringToBytesWithNoCopy(s string) in pkg/util/util.go have some problems in GC scenarios #1432

Open colin-si opened 9 months ago

colin-si commented 9 months ago

Describe the bug The return result of the StringToBytesWithNoCopy function will be recycled by GC when a large number of objects are created.

To Reproduce The following demo can reproduce this problem

var ResultKey []byte

func main() {
    key:=[]byte("123456789")
    ResultKey = StringToBytesWithNoCopy(string(key))
    // Result is normal before gc
    log.Printf("before runtime.gc and makeEscape: %v\n", string(ResultKey))
    runtime.GC()
    for i:=1;i<5000;i++{
        makeEscape(time.Now().String())
    }
    // Result has been recycled after gc
    log.Printf("after runtime.gc and makeEscape: %v\n",string(ResultKey))
}

func makeEscape(input string)string  {
    return  input
}

// method copy from servicecomb-service-center/pkg/util/util.go
func StringToBytesWithNoCopy(s string) []byte {
    x := (*[2]uintptr)(unsafe.Pointer(&s))
    h := [3]uintptr{x[0], x[1], x[1]}
    return *(*[]byte)(unsafe.Pointer(&h))
}

// use this method can solve the problem
func StringToBytesWithNoCopyNew(str string) []byte {
    return *(*[]byte)(unsafe.Pointer(
        &struct {
            string
            Cap int
        }{str, len(str)},
    ))
}

Expected behavior When we executed the above demo, we found that the returned results were recycled after GC. This was unexpected. The memory where ResultKey is located is recycled and reallocated for use elsewhere.

2023/12/13 10:57:20 before runtime.gc and makeEscape: 123456789
2023/12/13 10:57:20 after runtime.gc and makeEscape:  m=+0.017

Platform And Runtime (please complete the following information):

Platform

Runtime

Additional context Add any other context about the problem here.

hubeoo commented 3 months ago

reflect.StringHeader is unsafe

type StringHeader struct {
    Data uintptr
    Len  int
}

Garbage Collector (GC) : Go's garbage collector automatically tracks all valid Pointers to ensure that the memory they point to is not reclaimed. This means that as long as there is a valid pointer to a memory region, that memory region will not be collected by the garbage collector.

StringHeader which uses uintptr is a typeless integer that can be used to hold pointer values, but it is not really a pointer type. Therefore, the garbage collector will not regard uintptr as a valid pointer to track. Causes StringHeader to be affected.