search

ggerganov / whisper.cpp

Port of OpenAI's Whisper model in C/C++
MIT License
35.38k stars 3.61k forks source link

memory leak when deallocating and reallocating WhisperContext #2467

Open yousifa opened 3 weeks ago

yousifa commented 3 weeks ago

Deallocating WhisperContext (from swift) will leave memory allocated (1-10mb)

To replicate: Call WhisperContext() and then set it to nil in the demo whisper swiftui app repeatedly. Memory will continue growing.

chnbr commented 3 weeks ago

I can confirm this. Sometimes the memory loss is up to 30MB.

yousifa commented 3 weeks ago

Yep, the amount it retains seems inconsistent and random

ggerganov commented 3 weeks ago

Thanks for reporting. I'll take a look in the next days. Let me know if you find what is the resource that leaks or the commit that caused the regression.

yousifa commented 3 weeks ago

Thanks, I'm looking into it too. This issue has existed for at least 6 months, as that was the version I was using when i first noticed it (and it still exists in the latest).

yousifa commented 1 week ago

@ggerganov any updates here?

ggerganov commented 1 week ago

Nope, haven't looked into this yet. Did you find anything on your end?

ggerganov commented 1 week ago

I'm not very good with Swift: would this deinit function get called when you set the WhisperContext = nil?

https://github.com/ggerganov/whisper.cpp/blob/a5abfe6a90495f7bf19fe70d016ecc255e97359c/examples/whisper.swiftui/whisper.cpp.swift/LibWhisper.swift#L15-L19

I suppose if this is like a destructor in C++, then it would get called.

Josscii commented 1 week ago

if there is no strong reference to the WhisperContext object, it should call the deinit

ggerganov commented 1 week ago

Thanks @Josscii. So @yousifa and @chnbr, make sure that this whisper_free is called in your code. If it is and it still leaks, then provide the most basic example Swift program that reproduces the issue and instructions how to run it.

chnbr commented 1 week ago

Thanks @Josscii. So @yousifa and @chnbr, make sure that this whisper_free is called in your code.

At least on my side it definitely is. In Xcode debugger it is easy to monitor the memory consumption. One of the first things I learned is to call whisper_free. Otherwise we would leak a few hundred MB of memory depending on the model.

The thing is however that after calling whisper_free the consumption does not drop to the amount of before initialising whisper. I would expect that some allocs during operation are not free'd properly.

yousifa commented 1 week ago

Yes, whisper_free is called in the deinit and i call it manually as well. We'll get you a sample project shortly.