search

alanngnet / CoverHunterMPS

Fork of Liu Feng's CoverHunter to run on a single computer, plus more features and documentation.
7 stars 2 forks source link

Memory leak during CQT generation #32

Closed alanngnet closed 1 month ago

alanngnet commented 5 months ago

Something is causing gradually increasing memory consumption for each python instance running during CQT generation, at least in MPS context. Possibly a bug in nnAudio to report those authors, or already a known bug? Or something we can address in our project?

alanngnet commented 5 months ago

Impact is significant for large datasets. Throughput rapidly decreases after a few thousand samples, as dealing with overflowing memory takes more and more resources. Eventually I have to abort the run and restart the extract_csi_features script.

alanngnet commented 1 month ago

Added workaround to just do CQT processing in batches and then run garbage collection. Batch size=5000 worked well on my system. It does noticeably bog down before 5,000, but running gc more often also adds significant delay even just to run through lines that don't need new CQT generation.