hitachi-speech / EEND

End-to-End Neural Diarization
MIT License
360 stars 57 forks source link

High memory usage #14

Open francoishernandez opened 3 years ago

francoishernandez commented 3 years ago

Hi there, Thanks for the nice papers and implementation! As part of a few set of experiments I'm trying to run this on a custom dataset with around 1500 recordings, accounting for a total of 100 hours of speech. Though, it seems to use way too much RAM. It filled 64G of RAM, and almost 64G of swap, and I had to stop to prevent the system to freeze. Is this a known behavior? Are there some workarounds to limit memory usage? Thanks!

Note: the mini_librispeech recipe seems to run properly though, using around 2G of memory. There might be some issues with my setup/data.