MTG / gaia

C++ library to apply similarity measures and classifications on the results of audio analysis, including Python bindings. Together with Essentia it can be used to compute high-level descriptions of music.
http://essentia.upf.edu
GNU Affero General Public License v3.0
271 stars 66 forks source link

Memory leaks in dataset training #96

Open alastair opened 4 years ago

alastair commented 4 years ago

During dataset training, when cluster_mode=False, the multiprocessing code starts n new processes, and then runs jobs in each process. There is a memory leak during this process, and so eventually the processes get killed (and turn into zombies). Running with cluster_mode=True, a new process is started for the actual training process, and so any leaked memory is recovered at the end of each job.

Attached is a valgrind report of potential memory leaks. Gaia was built with debug mode, and so symbol names and filenames are included (search for gaia2::) gaia-svmclassification-valgrind.txt

alastair commented 4 years ago

https://github.com/MTG/gaia/blob/3c07f6fd964b9cb51f868959f4c119acb4695c79/src/3rdparty/libsvm/gaiasvmutils.cpp#L121-L122