search

maddyblue / go-dsp

Digital Signal Processing for Go
http://godoc.org/github.com/mjibson/go-dsp
ISC License
858 stars 89 forks source link

Possible memory leak #23

Open pawel0987 opened 6 years ago

pawel0987 commented 6 years ago

Invoking fft.FFT() many times for large wav files (~10MB) couses program to panic because of out of memory. I have tested my program with and without that single line. Without computing fft it occupies ~300MB for very long time. When I enable fft computing it will be 3GB of RAM occupied pretty fast.

maddyblue commented 6 years ago

Certainly possible. I don't use this anymore and it was maybe the first Go code I wrote, so it is likely very bad at memory. I'm definitely not going to work on fixing it but I would review a PR.

J1ng06 commented 4 years ago

@pawel0987 I am using this library as well and my app got to 40GB memory usage after running for a few days. The issue is from the fft/bluestein.go: there are two map that caches all the bluesteinFactors and bluesteinInvFactors, which gets larger over time. In my use case, my data set is large as well with different input_len most of the time. Matt's implementation was to speed up the calculation by memorizing the bluesteinFactors and bluesteinInvFactors, but it's not memory friendly. I did some modification for my use case and the calculation slows down, but I don't need to worry about high memory usage anymore.

i0x71 commented 3 years ago

@J1ng06 Would you be able to share the modifications that you made to avoid the memory leak ? Or is it implemented in your fork ?

Much appreciated