schollz / peerdiscovery

Pure-Go library for cross-platform local peer discovery using UDP multicast :woman: :repeat: :woman:
MIT License
644 stars 56 forks source link

Memory leak #11

Closed kylepritchard closed 1 year ago

kylepritchard commented 4 years ago

I have been using peerdiscovery in an app for work and it is fantastic. It makes finding other machines and making a distributed cluster quick and easy. Good work!!! I have noticed that when using it in a loop for continuous discovery and building/rebuilding a client list, the application memory usage was increasing. Using the example and running pprof it seems that there is a memory leak with the for loop which scans the socket made. This is running in goroutine which from what I can see doesn't exit.


Type: inuse_space
Time: Feb 10, 2020 at 7:40pm (EST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top20
Showing nodes accounting for 1.61MB, 100% of 1.61MB total
      flat  flat%   sum%        cum   cum%
    1.61MB   100%   100%     1.61MB   100%  github.com/schollz/peerdiscovery.(*peerDiscovery).listen
(pprof) list listen
Total: 1.61MB
ROUTINE ======================== github.com/schollz/peerdiscovery.(*peerDiscovery).listen in D:\goprogs\src\github.com\schollz\peerdiscovery\peerdiscovery.go
    1.61MB     1.61MB (flat, cum)   100% of Total
         .          .    322:           }
         .          .    323:   }
         .          .    324:
         .          .    325:   // Loop forever reading from the socket
         .          .    326:   for {
    1.61MB     1.61MB    327:           buffer := make([]byte, maxDatagramSize)
         .          .    328:           var (
         .          .    329:                   n       int
         .          .    330:                   src     net.Addr
         .          .    331:                   errRead error
         .          .    332:           )
(pprof)`

The above heap profile doesn't show a great deal of memory usage, however over the long run it all adds up and I had it crash the app due to memory restrictions. I resolved this by adding a bool variable into the peerdiscovery struct which I set to true at the end of the Discover method. In the socket for loop I run a check of the bool variable and break the loop if true.

Crude solution but maybe something to consider if making more updates
schollz commented 4 years ago

@kylepritchard Would you mind contributing your solution as a PR? Do not worry about it being "crude" :) I can merge it and refine as needed