search

facebookresearch / svoice

We provide a PyTorch implementation of the paper Voice Separation with an Unknown Number of Multiple Speakers In which, we present a new method for separating a mixed audio sequence, in which multiple voices speak simultaneously. The new method employs gated neural networks that are trained to separate the voices at multiple processing steps, while maintaining the speaker in each output channel fixed. A different model is trained for every number of possible speakers, and the model with the largest number of speakers is employed to select the actual number of speakers in a given sample. Our method greatly outperforms the current state of the art, which, as we show, is not competitive for more than two speakers.
Other
1.23k stars 178 forks source link

AssertionError when training for 3 speakers #54

Closed umerjamil16 closed 2 years ago

umerjamil16 commented 2 years ago

Hello,

I am training model on Librimix 3 speakers dataset. I've also modified the make_debug.sh file for generating .json file in svoice/egs/debug/tr/ directory. However I am facing error as shown in below code (The model train perfectly for dataset of 2 speakers).

Would be greatful if someone can point me in a direction towards addressing the issue. 

[2021-11-15 13:55:47,256][__main__][ERROR] - Some error happened
Traceback (most recent call last):
  File "train.py", line 118, in main
    _main(args)
  File "train.py", line 112, in _main
    run(args)
  File "train.py", line 93, in run
    solver.train()
  File "/workspace/svoice/svoice/solver.py", line 122, in train
    train_loss = self._run_one_epoch(epoch)
  File "/workspace/svoice/svoice/solver.py", line 210, in _run_one_epoch
    sources, est_src, lengths)
  File "/workspace/svoice/svoice/models/sisnr_loss.py", line 23, in cal_loss
    source_lengths)
  File "/workspace/svoice/svoice/models/sisnr_loss.py", line 39, in cal_si_snr_with_pit
    assert source.size() == estimate_source.size()
AssertionError
adiyoss commented 2 years ago

Hi @umerjamil16, Did you set svoice.C=3? you should specify to how many channels you want to separate the input mixture.

umerjamil16 commented 2 years ago

Got it, its working fine now. Thanks for the quick reply.

qalabeabbas49 commented 2 years ago

@umerjamil16 can you share how did the training with librimix go ? with 3 speakers.

I trying to do the same.

Also, I would appreciate if you could share the config.yaml for 3speakers(librimix).

Thanks