gnolang / gno

Gno: An interpreted, stack-based Go virtual machine to build succinct and composable apps + gno.land: a blockchain for timeless code and fair open-source.
https://gno.land/
Other
900 stars 377 forks source link

Memory Leaks #2713

Open notJoon opened 3 months ago

notJoon commented 3 months ago

Overview

By modifying the print-runtime-metrics option of the test flag to analyze memory allocations. I discovered that memory is not fully released and continue to accumulate even after the lifecycle of the objects/variables ends.

This analysis primarily compared uint64, uint256 and math/big's bigint types.

Test Methodology

  1. Main comparison targets: uint64, uint256, math/big's bigint
  2. Additional verification: strings package's Builder
  3. Method: Comparison of memory usage for identical operations
  4. Tool: Modified print-runtime-metrics flag (may differ from actual usage)

Modify the Allocator type as follows to see how much memory is allocated for each item when the flag is used.

type Allocator struct {
    maxBytes int64
    bytes    int64
    opAllocs map[string]uint64
    mu sync.Mutex
}

Problem Analysis

1. Memory Leaks in Nested Scopes

 func TestAcc10Uint256(t *testing.T) {
    {
        res := Zero()
        for i := 0; i < 10; i++ {
            res.Add(res, One())
        }
        println(res.ToString()) // res: 124.2kb alloc
    }  // <- res is deallocated here

    res2 := Zero()
    for i := 0; i < maxLoop; i++ {
        res2.Add(res2, One())
    }
    println(res2.ToString()) // res2: 123.5kb allocs
} // total 197.9k allocs (+74.4kb)
func TestAcc10BigInt(t *testing.T) {
    {
        res := big.NewInt(0)
        for i := 0; i < maxLoop; i++ {
            res.Add(res, big.NewInt(1))
        }
        println(res.String()) //  res: 61.8kb alloc
    } // <- res is deallocated here

    res2 := big.NewInt(0)
    for i := 0; i < maxLoop; i++ {
        res2.Add(res2, big.NewInt(1))
    }
    println(res2.String()) // res2: 61.3kb alloc
}  // total 73.6kb allocs (+12.3kb)

2. Memory Accumulations in Loops

Iterations Allocation (kb)
10 1.7
20 3.1
30 4.5
40 6.0
50 7.4

Memory Usage Comparison by Type (uint: kb)

Iterations uint64 uint256 bigint
0 [^1] 50.5 56.9 51.4
10 52.9 198.7 62.8
20 52.9 324.6 72.2
30 52.9 450.5 81.6
40 52.9 576.4 90.9
50 52.9 702.3 100.3

The main cause of memory leaks in loops are estimated to be as follows:

  1. Object Creation and Absence of GC

    • In uint256 and bigint operations, new objects are likely to be created to store the result for each operation.
  2. Accumulation of Temporarty Objects

    • Temporary objects created in each iteration accumulate in heap memory without being immediately released.
    • In environments without GC, these objects are not automatically cleaned up. 스크린샷 2024-08-20 오후 5 21 39

3. Memory Management Characteristics by Type

I also checked for similar behaviour in stdlib, such as the strings package, and this was also experiencing the same issue.

func TestAccumulateStrings(t *testing.T) {
    {
        var builder strings.Builder
        for i := 0; i < maxLoop; i++ {
            builder.WriteString("Hello")
        }
        result := builder.String()
        println(len(result))
    } // 103.2k

    var builder2 strings.Builder
    for i := 0; i < maxLoop; i++ {
        builder2.WriteString("World")
    }
    result2 := builder2.String()
    println(len(result2)) // 102.7k
} // 156.4k

Conclusion

Memory leaks occur when using uint256, bigint and standard library objects like strings.Builder in environment without GC or other memory management systems.

Looking at the ownership.go file, it appears that a reference counting method is applied to manage objects, but it seems to have limitations.

This can lead to performance degradation and increase gas cost, necessitating the adding appropriate memory management strategies. we might consider RAII, or GC as suggested previously.

Related

266

1788

[^1]: State after object creation only.

zivkovicmilos commented 3 months ago

cc @petar-dambovaliev for visibility