We provide a PyTorch implementation of the paper Voice Separation with an Unknown Number of Multiple Speakers In which, we present a new method for separating a mixed audio sequence, in which multiple voices speak simultaneously. The new method employs gated neural networks that are trained to separate the voices at multiple processing steps, while maintaining the speaker in each output channel fixed. A different model is trained for every number of possible speakers, and the model with the largest number of speakers is employed to select the actual number of speakers in a given sample. Our method greatly outperforms the current state of the art, which, as we show, is not competitive for more than two speakers.
Other
1.26k
stars
181
forks
source link
When executing "svoice.separate", "CUDA out of memory" occurs #47
Hi，I have a large file to be separated, 22M. When I execute "svoice.separate", "CUDA out of memory" occurs. How can I solve it?
When the separated files are relatively small, there is no error.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:00<00:00, 90.37it/s]
0%| | 0/5 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/usr/local/python3/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/local/python3/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/data/home/test/svoice/svoice/separate.py", line 133, in
separate(args, local_out_dir=args.out_dir)
File "/data/home/test/svoice/svoice/separate.py", line 123, in separate
estimate_sources = model(mixture)[-1]
File "/data/home/test/svoice/venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, kwargs)
File "/data/home/test/svoice/svoice/models/swave.py", line 253, in forward
output_all = self.separator(mixture_w)
File "/data/home/test/svoice/venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, *kwargs)
File "/data/home/test/svoice/svoice/models/swave.py", line 214, in forward
output_all = self.rnn_model(enc_segments)
File "/data/home/test/svoice/venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(input, kwargs)
File "/data/home/test/svoice/svoice/models/swave.py", line 108, in forward
row_output = self.rows_grnni
File "/data/home/test/svoice/venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, *kwargs)
File "/data/home/test/svoice/svoice/models/swave.py", line 43, in forward
rnnoutput, = self.rnn(output)
File "/data/home/test/svoice/venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(input, **kwargs)
File "/data/home/test/svoice/venv/lib/python3.8/site-packages/torch/nn/modules/rnn.py", line 576, in forward
result = _VF.lstm(input, hx, self._flat_weights, self.bias, self.num_layers,
RuntimeError: CUDA out of memory. Tried to allocate 5.42 GiB (GPU 0; 15.78 GiB total capacity; 9.66 GiB already allocated; 3.62 GiB free; 11.00 GiB reserved in total by PyTorch)
Hi @srdfjy,
At the moment I can see two possible options:
1) try to disable gradients with respect to the input sequence using x.require_grad=False it might solve it.
2) split the large file into several small ones
The code loads all the audio file at once, so processing large files can be problematic
adiyoss
commented
3 years ago 
srdfjy
Hi，I have a large file to be separated, 22M. When I execute "svoice.separate", "CUDA out of memory" occurs. How can I solve it?
When the separated files are relatively small, there is no error.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:00<00:00, 90.37it/s] 0%| | 0/5 [00:00<?, ?it/s] Traceback (most recent call last): File "/usr/local/python3/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/local/python3/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/data/home/test/svoice/svoice/separate.py", line 133, in
separate(args, local_out_dir=args.out_dir)
File "/data/home/test/svoice/svoice/separate.py", line 123, in separate
estimate_sources = model(mixture)[-1]
File "/data/home/test/svoice/venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, kwargs)
File "/data/home/test/svoice/svoice/models/swave.py", line 253, in forward
output_all = self.separator(mixture_w)
File "/data/home/test/svoice/venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, *kwargs)
File "/data/home/test/svoice/svoice/models/swave.py", line 214, in forward
output_all = self.rnn_model(enc_segments)
File "/data/home/test/svoice/venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(input, kwargs)
File "/data/home/test/svoice/svoice/models/swave.py", line 108, in forward
row_output = self.rows_grnni
File "/data/home/test/svoice/venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, *kwargs)
File "/data/home/test/svoice/svoice/models/swave.py", line 43, in forward
rnnoutput, = self.rnn(output)
File "/data/home/test/svoice/venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(input, **kwargs)
File "/data/home/test/svoice/venv/lib/python3.8/site-packages/torch/nn/modules/rnn.py", line 576, in forward
result = _VF.lstm(input, hx, self._flat_weights, self.bias, self.num_layers,
RuntimeError: CUDA out of memory. Tried to allocate 5.42 GiB (GPU 0; 15.78 GiB total capacity; 9.66 GiB already allocated; 3.62 GiB free; 11.00 GiB reserved in total by PyTorch)