[Bug]: example gnet for echo tcp uses too much memory

[kolinfluence]

Actions I've taken before I'm here

• [X] I've thoroughly read the documentations on this issue but still have no clue.
• [X] I've searched the Github Issues but didn't find any duplicate issues that have been resolved.
• [X] I've searched the internet for this issue but didn't find anything helpful.

What happened?

ran iperf on this twice and the memory used is too great

iperf -c 127.0.0.1 -p 9000

why does it use so much memory and how to resolve this without using gomemlimit? memory is not reduced after iperf finished immediately.

https://github.com/gnet-io/gnet-examples/blob/v2/echo_tcp/echo.go

Major version of gnet

v2

Specific version of gnet

latest

Operating system

Linux

OS version

ubuntu 22.04

Go version

1.22

Relevant log output

information as attached.

Code snippets (optional)

No response

How to Reproduce

get the tcp echo file from example, build it, run iperf command as above, check memory with top and see it over uses memory available.

Does this issue reproduce with the latest release?

It can reproduce with the latest release

[panjf2000]

Use pprof to find out where the majority of the memory is allocated.

[kolinfluence]

@panjf2000 is this really ok?

nothing extra added and using iperf... memory is bytesliced gotten from internal mechanism... if i do gomemlimit, the whole thing becomes too slow. any fix for this?

basically, it seemed easy to crash with oom by iperfing the server if it's not guarded by firewall or something. i would prefer to hope u can have a fix for this or at least give us some suggestion on workarounds to ensure it's not using memory this way other than gomemlimit

package main

import (
        "flag"
        "fmt"
        "log"
        "net/http"
        _ "net/http/pprof"

        "github.com/panjf2000/gnet/v2"
)

type echoServer struct {
        gnet.BuiltinEventEngine

        eng       gnet.Engine
        addr      string
        multicore bool
}

func (es *echoServer) OnBoot(eng gnet.Engine) gnet.Action {
        es.eng = eng
        log.Printf("echo server with multi-core=%t is listening on %s\n", es.multicore, es.addr)
        return gnet.None
}

func (es *echoServer) OnTraffic(c gnet.Conn) gnet.Action {
        buf, _ := c.Next(-1)
        c.Write(buf)
        return gnet.None
}

func main() {
        var port int
        var multicore bool
        var pprofPort int

        // Example command: go run main.go --port 9000 --multicore=true --pprofPort=6060
        flag.IntVar(&port, "port", 9000, "Set the port for the TCP server")
        flag.BoolVar(&multicore, "multicore", false, "Enable multicore processing")
        flag.IntVar(&pprofPort, "pprofPort", 6060, "Set the port for the pprof HTTP server")
        flag.Parse()

        // Start pprof server
        go func() {
                log.Printf("Starting pprof server on http://localhost:%d/debug/pprof/\n", pprofPort)
                log.Println(http.ListenAndServe(fmt.Sprintf("localhost:%d", pprofPort), nil))
        }()

        // Setup and start TCP echo server
        echo := &echoServer{addr: fmt.Sprintf("tcp://:%d", port), multicore: multicore}
        log.Fatal(gnet.Run(echo, echo.addr, gnet.WithMulticore(multicore)))
}

[kolinfluence]

@panjf2000 i can confirm this is a bug, running the example tcp echo using

iperf -c 127.0.0.1 -p 8080 -t 3600

u can crash it. with GOMEMLIMIT=6GiB ./echoserver you get 3000 req/s without GOMEMLIMIT, u get 100k req/s with iperf

[panjf2000]

I've never run into this problem when benchmarking, and I think it is expected to allocate plenty of memory when the server is overloaded during benchmark testing, this happens to all network frameworks. Have you tried WithEdgeTriggeredIO?

[panjf2000]

Besides, you need to prove that gnet has this issue while other network frameworks don't before you call it a bug.

[kolinfluence]

@panjf2000 , is this how WithEdgeTriggeredIO is used?

same, the memory is used til oom, iperf -c 127.0.0.1 -p 9000 -t 3600

package main

import (
        "flag"
        "fmt"
        "log"
        "net/http"
        _ "net/http/pprof"

        "github.com/panjf2000/gnet/v2"
)

type echoServer struct {
        gnet.BuiltinEventEngine

        eng       gnet.Engine
        addr      string
        multicore bool
}

func (es *echoServer) OnBoot(eng gnet.Engine) gnet.Action {
        es.eng = eng
        log.Printf("echo server with multi-core=%t is listening on %s\n", es.multicore, es.addr)
        return gnet.None
}

func (es *echoServer) OnTraffic(c gnet.Conn) gnet.Action {
        buf, _ := c.Next(-1)
        c.Write(buf)
        return gnet.None
}

func main() {
        var port int
        var multicore bool
        var pprofPort int

        // Example command: go run main.go --port 9000 --multicore=true --pprofPort=6060
        flag.IntVar(&port, "port", 9000, "Set the port for the TCP server")
        flag.BoolVar(&multicore, "multicore", false, "Enable multicore processing")
        flag.IntVar(&pprofPort, "pprofPort", 6060, "Set the port for the pprof HTTP server")
        flag.Parse()

        // Start pprof server
        go func() {
                log.Printf("Starting pprof server on http://localhost:%d/debug/pprof/\n", pprofPort)
                log.Println(http.ListenAndServe(fmt.Sprintf("localhost:%d", pprofPort), nil))
        }()

        // Setup and start TCP echo server
        echo := &echoServer{addr: fmt.Sprintf("tcp://:%d", port), multicore: multicore}
        log.Fatal(gnet.Run(echo, echo.addr, gnet.WithMulticore(multicore), gnet.WithEdgeTriggeredIO(true)))
}

[kolinfluence]

@panjf2000 sorry my bad, it's working as it should. thx.