dominant-strategies / go-quai

Official Go Implementation of the Quai Network
GNU General Public License v3.0
2.36k stars 456 forks source link

Investigate goroutine leaks - affecting memory #1879

Closed mibuono closed 2 weeks ago

mibuono commented 2 weeks ago

HandleMessage doesn't complete some of the times and causes the goroutines to bloom.

kiltsonfire commented 2 weeks ago

Showing top 20 nodes out of 52 flat flat% sum% cum cum% 7572 99.93% 99.93% 7572 99.93% runtime.gopark 0 0% 99.93% 65 0.86% bufio.(Reader).Peek 0 0% 99.93% 65 0.86% bufio.(Reader).fill 0 0% 99.93% 193 2.55% github.com/dominant-strategies/go-quai/common.ReadMessageFromStream 0 0% 99.93% 45 0.59% github.com/dominant-strategies/go-quai/node.(httpServer).ServeHTTP 0 0% 99.93% 175 2.31% github.com/dominant-strategies/go-quai/p2p/node.(P2PNode).Start.func1 0 0% 99.93% 44 0.58% github.com/dominant-strategies/go-quai/p2p/node/pubsubManager.(PubsubManager).Subscribe.func1 0 0% 99.93% 440 5.81% github.com/dominant-strategies/go-quai/p2p/node/pubsubManager.(PubsubManager).Subscribe.func1.2 0 0% 99.93% 193 2.55% github.com/dominant-strategies/go-quai/p2p/protocol.QuaiProtocolHandler 0 0% 99.93% 4991 65.87% github.com/dominant-strategies/go-quai/p2p/protocol.QuaiProtocolHandler.func2 0 0% 99.93% 144 1.90% github.com/dominant-strategies/go-quai/quai.(Quai).startBloomHandlers.func1 0 0% 99.93% 45 0.59% github.com/dominant-strategies/go-quai/rpc.(Client).dispatch 0 0% 99.93% 45 0.59% github.com/dominant-strategies/go-quai/rpc.(Client).read 0 0% 99.93% 45 0.59% github.com/dominant-strategies/go-quai/rpc.(Server).ServeCodec 0 0% 99.93% 45 0.59% github.com/dominant-strategies/go-quai/rpc.(Server).WebsocketHandler.func1 0 0% 99.93% 45 0.59% github.com/dominant-strategies/go-quai/rpc.(jsonCodec).readBatch 0 0% 99.93% 45 0.59% github.com/dominant-strategies/go-quai/rpc.(websocketCodec).pingLoop 0 0% 99.93% 45 0.59% github.com/gorilla/websocket.(Conn).NextReader 0 0% 99.93% 45 0.59% github.com/gorilla/websocket.(Conn).ReadJSON 0 0% 99.93% 45 0.59% github.com/gorilla/websocket.(Conn).advanceFrame (pprof) list github.com/dominant-strategies/go-quai/p2p/protocol.QuaiProtocolHan dler.func2 Total: 7577 ROUTINE ======================== github.com/dominant-strategies/go-quai/p2p/protocol.QuaiProtocolHandler.func2 in github.com/dominant-strategies/go-quai/p2p/protocol/handler.go 0 4991 (flat, cum) 65.87% of Total . . 49: go func() { . . 50: for { . . 51: select { . 4991 52: case message := <-msgChan: . . 53: handleMessage(message, stream, node) . . 54: } . . 55: } . . 56: }() . . 57: (pprof) list github.com/dominant-strategies/go-quai/p2p/node/pubsubManager.(*P ubsubManager).Subscribe.func1.2 Total: 7577 ROUTINE ======================== github.com/dominant-strategies/go-quai/p2p/node/pubsubManager.(*PubsubManager).Subscribe.func1.2 in github.com/dominant-strategies/go-quai/p2p/node/pubsubManager/gossipsub.go 0 440 (flat, cum) 5.81% of Total . . 136: msgWorker = func(location common.Location) { . . 137: defer func() { . . 138: if r := recover(); r != nil { . . 139: log.Global.WithFields(log.Fields{ . . 140: "error": r, . . 141: "stacktrace": string(debug.Stack()), . . 142: "location": location.Name(), . . 143: }).Errorf("Go-Quai Panicked") . . 144: } . . 145: go msgWorker(location) // If this worker exits, start a new one . . 146: }() . 440 147: for msg := range msgChan { // This should exit when msgChan is closed . . 148: data := types.ObjectPool.Get() . . 149: data = nil . . 150: // unmarshal the received data depending on the topic's type . . 151: err = pb.UnmarshalAndConvert(msg.Data, location, &data, datatype) . . 152: if err != nil { ![Uploading profile001.png…]()